mirror of
https://github.com/apache/impala.git
synced 2026-01-07 09:02:19 -05:00
75a97d3d7e
This is the first iteration of a kerberized development environment. All the daemons start and use kerberos, with the sole exception of the hive metastore. This is sufficient to test impala authentication. When buildall.sh is run using '-kerberize', it will stop before loading data or attempting to run tests. Loading data into the cluster is known to not work at this time, the root causes being that Beeline -> HiveServer2 -> MapReduce throws errors, and Beeline -> HiveServer2 -> HBase has problems. These are left for later work. However, the impala daemons will happily authenticate using kerberos both from clients (like the impala shell) and amongst each other. This means that if you can get data into the mini-cluster, you could query it. Usage: * Supply a '-kerberize' option to buildall.sh, or * Supply a '-kerberize' option to create-test-configuration.sh, then 'run-all.sh -format', re-source impala-config.sh, and then start impala daemons as usual. You must reformat the cluster because kerberizing it will change all the ownership of all files in HDFS. Notable changes: * Added clean start/stop script for the llama-minikdc * Creation of Kerberized HDFS - namenode and datanodes * Kerberized HBase (and Zookeeper) * Kerberized Hive (minus the MetaStore) * Kerberized Impala * Loading of data very nearly working Still to go: * Kerberize the MetaStore * Get data loading working * Run all tests * The unknown unknowns * Extensive testing Change-Id: Iee3f56f6cc28303821fc6a3bf3ca7f5933632160 Reviewed-on: http://gerrit.sjc.cloudera.com:8080/4019 Reviewed-by: Michael Yoder <myoder@cloudera.com> Tested-by: jenkins
42 lines
949 B
Python
42 lines
949 B
Python
# Copyright (c) Cloudera 2012.
|
|
|
|
from subprocess import Popen, PIPE
|
|
import time
|
|
import re
|
|
import sys
|
|
|
|
now = time.time()
|
|
TIMEOUT_SECONDS = 30.0
|
|
ZK_CLASS="org.apache.zookeeper.ZooKeeperMain"
|
|
|
|
print "Waiting for HBase Master"
|
|
|
|
while time.time() - now < TIMEOUT_SECONDS:
|
|
sys.stdout.write(".")
|
|
sys.stdout.flush()
|
|
|
|
p = Popen(["java",
|
|
ZK_CLASS,
|
|
"-server",
|
|
"localhost:2181",
|
|
"get",
|
|
"/hbase/rs"], stderr=PIPE, stdout=PIPE)
|
|
out, err = p.communicate()
|
|
if re.match(".*" + ZK_CLASS + "\w*$", err):
|
|
print "Failure"
|
|
print err
|
|
print "CLASSPATH does not contain " + ZK_CLASS
|
|
print "Please check your CLASSPATH"
|
|
exit(1)
|
|
|
|
if "numChildren" in err:
|
|
print "Success"
|
|
print "HBase master is up, found in %2.1fs" % (time.time() - now,)
|
|
exit(0)
|
|
|
|
time.sleep(0.5)
|
|
|
|
print "Failure"
|
|
print "Hbase master did NOT write /hbase/rs in %2.1fs" % (time.time() - now,)
|
|
exit(1)
|