mirror of
https://github.com/apache/impala.git
synced 2026-01-04 00:00:56 -05:00
75a97d3d7e
This is the first iteration of a kerberized development environment. All the daemons start and use kerberos, with the sole exception of the hive metastore. This is sufficient to test impala authentication. When buildall.sh is run using '-kerberize', it will stop before loading data or attempting to run tests. Loading data into the cluster is known to not work at this time, the root causes being that Beeline -> HiveServer2 -> MapReduce throws errors, and Beeline -> HiveServer2 -> HBase has problems. These are left for later work. However, the impala daemons will happily authenticate using kerberos both from clients (like the impala shell) and amongst each other. This means that if you can get data into the mini-cluster, you could query it. Usage: * Supply a '-kerberize' option to buildall.sh, or * Supply a '-kerberize' option to create-test-configuration.sh, then 'run-all.sh -format', re-source impala-config.sh, and then start impala daemons as usual. You must reformat the cluster because kerberizing it will change all the ownership of all files in HDFS. Notable changes: * Added clean start/stop script for the llama-minikdc * Creation of Kerberized HDFS - namenode and datanodes * Kerberized HBase (and Zookeeper) * Kerberized Hive (minus the MetaStore) * Kerberized Impala * Loading of data very nearly working Still to go: * Kerberize the MetaStore * Get data loading working * Run all tests * The unknown unknowns * Extensive testing Change-Id: Iee3f56f6cc28303821fc6a3bf3ca7f5933632160 Reviewed-on: http://gerrit.sjc.cloudera.com:8080/4019 Reviewed-by: Michael Yoder <myoder@cloudera.com> Tested-by: jenkins
31 lines
1.3 KiB
Bash
Executable File
31 lines
1.3 KiB
Bash
Executable File
#!/bin/bash
|
|
# Copyright 2012 Cloudera Inc.
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
#
|
|
# Check if a reference githash exists in hdfs; If it does, diff the directories that
|
|
# contain the schemas against the current HEAD.
|
|
# This script exits with 0 and 1 as the returncodes
|
|
# - 0 implies that the schema diff is emppty, or that a reference githash was not found.
|
|
# - 1 implies that the schemas have changed.
|
|
|
|
. ${IMPALA_HOME}/bin/impala-config.sh
|
|
set -ex
|
|
|
|
# If /test-warehouse/githash.txt does not exist, exit with a 0
|
|
hdfs dfs -test -e /test-warehouse/githash.txt || { exit 0; }
|
|
GIT_HASH=$(echo $(hdfs dfs -cat /test-warehouse/githash.txt))
|
|
# Check whether a non-empty diff exists.
|
|
# TODO: Make this more granular (on the level of a dataset)
|
|
git diff --exit-code ${GIT_HASH}..HEAD ${IMPALA_HOME}/testdata/datasets
|