mirror of
https://github.com/apache/impala.git
synced 2026-01-03 06:00:52 -05:00
75a97d3d7e
This is the first iteration of a kerberized development environment. All the daemons start and use kerberos, with the sole exception of the hive metastore. This is sufficient to test impala authentication. When buildall.sh is run using '-kerberize', it will stop before loading data or attempting to run tests. Loading data into the cluster is known to not work at this time, the root causes being that Beeline -> HiveServer2 -> MapReduce throws errors, and Beeline -> HiveServer2 -> HBase has problems. These are left for later work. However, the impala daemons will happily authenticate using kerberos both from clients (like the impala shell) and amongst each other. This means that if you can get data into the mini-cluster, you could query it. Usage: * Supply a '-kerberize' option to buildall.sh, or * Supply a '-kerberize' option to create-test-configuration.sh, then 'run-all.sh -format', re-source impala-config.sh, and then start impala daemons as usual. You must reformat the cluster because kerberizing it will change all the ownership of all files in HDFS. Notable changes: * Added clean start/stop script for the llama-minikdc * Creation of Kerberized HDFS - namenode and datanodes * Kerberized HBase (and Zookeeper) * Kerberized Hive (minus the MetaStore) * Kerberized Impala * Loading of data very nearly working Still to go: * Kerberize the MetaStore * Get data loading working * Run all tests * The unknown unknowns * Extensive testing Change-Id: Iee3f56f6cc28303821fc6a3bf3ca7f5933632160 Reviewed-on: http://gerrit.sjc.cloudera.com:8080/4019 Reviewed-by: Michael Yoder <myoder@cloudera.com> Tested-by: jenkins
38 lines
1.2 KiB
Bash
Executable File
38 lines
1.2 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Copyright (c) 2012 Cloudera, Inc. All rights reserved.
|
|
|
|
. ${IMPALA_HOME}/bin/impala-config.sh
|
|
if ${CLUSTER_DIR}/admin is_kerberized; then
|
|
KERB_ARGS="--use_kerberos"
|
|
fi
|
|
|
|
# Split hbasealltypesagg and hbasealltypessmall and assign their splits
|
|
cd $IMPALA_HOME/testdata
|
|
mvn clean package
|
|
mvn dependency:copy-dependencies
|
|
|
|
. ${IMPALA_HOME}/bin/set-classpath.sh
|
|
export CLASSPATH=$IMPALA_HOME/testdata/target/impala-testdata-0.1-SNAPSHOT.jar:$CLASSPATH
|
|
|
|
RESULT=1
|
|
RETRY_COUNT=0
|
|
while [ $RESULT -ne 0 ] && [ $RETRY_COUNT -le 10 ]; do
|
|
java ${JAVA_KERBEROS_MAGIC} \
|
|
com.cloudera.impala.datagenerator.HBaseTestDataRegionAssigment \
|
|
functional_hbase.alltypesagg functional_hbase.alltypessmall
|
|
RESULT=$?
|
|
|
|
if [ $RESULT -ne 0 ]; then
|
|
((RETRY_COUNT++))
|
|
# If the split failed, force reload the hbase tables before trying the next split
|
|
$IMPALA_HOME/bin/start-impala-cluster.py
|
|
$IMPALA_HOME/bin/load-data.py -w functional-query \
|
|
--table_names=alltypesagg,alltypessmall --table_formats=hbase/none --force \
|
|
${KERB_ARGS} --principal=${MINIKDC_PRINC_HIVE}
|
|
$IMPALA_HOME/tests/util/compute_table_stats.py --db_names=functional_hbase \
|
|
--table_names=alltypesagg,alltypessmall ${KERB_ARGS}
|
|
fi
|
|
done
|
|
|
|
exit $RESULT
|