Some operating systems don't ship which nproc, which causes impala-config.sh to fail. This
change alleviates the problem by checking if nproc exists, and setting a reasonable
default if it fails.
Change-Id: Ic6e4d0fbce57eedc82163cfa17f71bdccbc38b51
Reviewed-on: http://gerrit.ent.cloudera.com:8080/3208
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
Currently, we launch #nproc processes to run tests locally. This patch changes the default
to #proc/2, to not overload the system.
Change-Id: I8bca23eb7462a0c497df93f82a60d85835bedbe9
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2972
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
This updates Impala to use Sentry v1.3 instead of Sentry v1.2. No major functionality
changed between Sentry versions, but some Sentry classes were moved and APIs changed.
Change-Id: I3765748d2cdbe00f59eefa3c971558efede38ebd
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2319
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: jenkins
This just updates the versions, it doesn't touch anything in /thirdparty.
Change parquet version to append SNAPSHOT
Added hadoop-hbase-compat jar in AUX_CLASSPATH and mapreduce/*.jar to HDFS
Change-Id: I4471ef4476997371cf49a9d54cfa63f2fda126e4
Changes include:
- version changes in impala-config
- version changes in various loading scripts
- hbase jars are no longer in hive/lib
- mini-llama script changes
- updates due to sentry api changes
- JDBC tests disabled
- unsupported types tests disabled.
Change-Id: If8cf1b7ad8e22aa4d23094b9a4b1047f7e9d93ee
Impala reserves resources from YARN via Llama and handles resources
preemptions by cancelling affected queries. Adds the Impala Resource
Broker for interacting with Llama. Refactors scheduler and coordinator
to move fragment-to-host assignment logic into scheduler. Local test
setup uses MiniLLama.
Change-Id: Ic7b0fe43de52d30f4207b4e65cce7e6a294e54e1
This change cleans up our FE pom.xml file by removing unneeded
dependencies and system dependencies (system dependencies are now pulled in
from the Maven release repository).
The upside is that our pom is cleaner and it will also help reduce the likelihood of
broken dependencies since Maven will pull in the right versions. The downside
is that we now pull in quite a few more JARs.
Note: I was unable to find release artifacts for Sentry and Parquet so I leaving
those as "system" for now.
Change-Id: I0b917b09a02243d78d89747591ab6bccacf7cf38
Saving changes
Change-Id: I3697a7b44884c40e077b3e354fef76625e1b881d
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1011
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: jenkins
The following changes have been made:
-- Update hbase
-- Update hive
-- Update hadoop
-- Update the parquet version to 1.2.5
Change-Id: Id6ceaef0e9eebab27ffd408160116fa84ed300fb
We now maintain our own internal version of the Mongoose webserver,
renamed to 'Squeasel' for differentiation. This patch imports the new
code, and swaps all mentions of mongoose or mg_ for squeasel / sq_.
In the future, we might consider making Squeasel a git subproject so
that we can pull in changes more easily.
Change-Id: I83b595dc336a32f2c8aba59eee420b71274b681b
Reviewed-on: http://gerrit.ent.cloudera.com:8080/485
Tested-by: jenkins
Reviewed-by: Henry Robinson <henry@cloudera.com>
This changes adds support for SQL statement authorization in Impala. The authorization
works by updating the Catalog API to require a User + Privilege when getting Table/Db
objects (and in the future can be extended to cover columns as well).
If the user doesn't have permission to access the object, an AuthorizationException is
thrown. The authorization checks are done during analysis as new Catalog objects are
encountered.
These changes build on top of the Hive Access code which handles the actually
processing of authorization requests. The authorization is currently based
on a "policy file" which will be stored in HDFS. This policy file is read once
on startup and then reloaded every 5 minutes. It can also be reloaded on a
specific impalad by executing a "refresh" command.
Authorization is enabled by setting:
--server_name='server1'
and then pointing the impalad to the policy file using the flag:
--authorization_policy_file=/path/to/policy/file
any authorization configuration problems will result in impalad failing to
start.
From the Java documentation:
"Signal-chaining enables the Java Platform to better interoperate with
native code that installs its own signal handlers... The signal-chaining facility offers:
A. Support for pre-installed signal handlers when the HotSpot VM is created.
B. Support for signal handler installation after the HotSpot VM is created, inside JNI
code or from another native thread."
This change adds support for auxiliary worksloads, tests, and datasets. This is useful
to augment the regular test runs with some additional tests that do not belong in the
main Impala repo.
Impala detects the HDFS version by reading the Namenode web UI and run
the corresponding check.
On 4.1, Impala tries to check the datanode (server side) config by reading
the datanode web UI.
