Python tests and infra scripts will now use "python" from the virtualenv
via $IMPALA_HOME/bin/impala-python. Some scripts could be simplified now
that python 2.6 and a dependable set of third-party libraries are
available but that is not done as part of this commit.
Change-Id: If1cf96898d6350e78ea107b9026b12ba63a4162f
Reviewed-on: http://gerrit.cloudera.org:8080/603
Reviewed-by: Taras Bobrovytsky <tbobrovytsky@cloudera.com>
Tested-by: Internal Jenkins
cgroups.py was using unsupported "except <Exception> as <var>" syntax.
generate_metrics.py was using the json module which is not available
in Python 2.4, but contains simplejson which provides the same
functionality.
Change-Id: If2c176c15a9573dd2a2acf5ee459ff24ce891ce3
Reviewed-on: http://gerrit.cloudera.org:8080/396
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: Matthew Jacobs <mj@cloudera.com>
Allows the base cgroup hierarchy path used by the impala
test cluster to be specified with the environment variable
IMPALA_CGROUP_BASE_PATH. This is needed to support older
kernels that do not use the proper default cgroup path
and do not even support finding the hierarchy via mount.
This will be used in jenkins test runs with RM enabled
which run on Centos6 images.
Change-Id: I30984a58fbcf990410f75f7feb5c1d549afa6ddd
Reviewed-on: http://gerrit.cloudera.org:8080/397
Reviewed-by: Casey Ching <casey@cloudera.com>
Tested-by: Internal Jenkins
Context managers are not supported before Python 2.7. Removes the
use of the 'with' clause in cgroups.py because this code is
executed on Centos 6 packaging boxes with an older version of
Python.
Change-Id: Ic6bcf161086f671ec2010df16f9bb23534c57697
Reviewed-on: http://gerrit.cloudera.org:8080/385
Reviewed-by: Martin Grund <mgrund@cloudera.com>
Tested-by: Matthew Jacobs <mj@cloudera.com>
Adds a flag to start-impala-cluster.py (--enable_rm) to set up the
mini Impala cluster using Yarn and Llama. This hides a number of
flags that must be set on the impalads:
-enable_rm
-llama_addressess: set to the local llama service
-fair_scheduler_allocation_path: set to the path of the fair-scheduler.xml
in each node's hadoop conf directory
-cgroup_hierarchy_path: set to a path in the CPU cgroup hierarchy which
has the correct permissions for Impala to manage a child cgroup. The
path comes from cgroups.py.
The new module cgroups.py was added to contain cgroups-related
utilities. Right now it provides paths to the CPU controller
hierarchy root and a path within the hierarchy that can be used
for impalads (i.e. have the proper permissions, one for each
cluster node).
Change-Id: Ic2181ec5613c180f240958c84f885c6b136a64d4
Reviewed-on: http://gerrit.cloudera.org:8080/369
Tested-by: Internal Jenkins
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
