mirror of
https://github.com/apache/impala.git
synced 2026-01-29 21:00:18 -05:00
236b9194d3
This patch does the work to load data and run some end-to-end query tests on a dockerised cluster. Changes were required in start-impala-cluster.py/ImpalaCluster and in some configuration files. ImpalaCluster is used for various things, including discovering service ports and testing for cluster readiness. This patch adds basic support and uses it from start-impala-cluster.py to check for cluster readiness. Some logic is moved from start-impala-cluster.py to ImpalaCluster. Limitations: * We're fairly inconsistent about whether services listen only on a single interface (e.g. loopback, traditionally) or whether it listens on all interfaces. This doesn't fix all of those issues. E.g. HDFS datanodes listen on all interfaces to work around some issues. * Many tests don't pass yet, particularly those using ImpalaCluster(), which isn't initialised with the appropriate docker arguments. Testing: Did a full data load locally using a dockerised Impala cluster: START_CLUSTER_ARGS="--docker_network=impala-cluster" \ TEST_START_CLUSTER_ARGS="--docker_network=impala-cluster" \ ./buildall.sh -format -testdata -ninja -notests -skiptests -noclean Ran a selection of end-to-end tests touching HDFS, Kudu and HBase tables after I loaded data locally. Ran exhaustive tests with non-dockerised impala cluster. Change-Id: I98fb9c4f5a3a3bb15c7809eab28ec8e5f63ff517 Reviewed-on: http://gerrit.cloudera.org:8080/12189 Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>