jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-25 02:03:09 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-9218: Add support for locally compiled Hive

Browse Source 
- Add HIVE_VERSION_OVERRIDE, HIVE_STORAGE_API_VERSION_OVERRIDE,
  HIVE_METASTORE_THRIFT_DIR_OVERRIDE, HIVE_HOME_OVERRIDE environment
  variable support to impala-config.sh
- When used together with HIVE_SRC_DIR_OVERRIDE allows a user to
  specify a locally compiled version of Hive for development and the
  minicluster
- Hive jars are expected to have been installed into the local maven
  repository
- Currently only version 3 of Hive is supported due to the absence of
  API shims for Hive 4.0
Example:
  ~/hive $ mvn package install -Pdist -DskipTests

Example configuration:
export HIVE_VERSION_OVERRIDE=3.1.0-SNAPSHOT
export HIVE_STORAGE_API_VERSION_OVERRIDE=2.6.0
export HIVE_HOME_OVERRIDE=\
~/hive/packaging/target/apache-hive-3.1.0-SNAPSHOT-bin/apache-hive-3.1.0-SNAPSHOT-bin
export HIVE_SRC_DIR_OVERRIDE=~/hive
export HIVE_METASTORE_THRIFT_DIR_OVERRIDE=~/hive/standalone-metastore/src/main/thrift/

Change-Id: I21892c153c445e3a5d93f2bc8f5e0b799929dd34
Reviewed-on: http://gerrit.cloudera.org:8080/17094
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
John Sherman
2021-02-20 17:04:24 +00:00
committed by Impala Public Jenkins
parent 2039746ebe
commit a29d06db53
4 changed files with 26 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
bin/bootstrap_toolchain.py
View File

@@ -466,7 +466,14 @@ def get_hadoop_downloads():
unpack_directory_tmpl="hive-${version}")
tez = CdpComponent("tez", archive_basename_tmpl="tez-${version}-minimal",
makedir=True)
cluster_components.extend([hadoop, hbase, hive, hive_src, tez])
use_override_hive = \
"HIVE_VERSION_OVERRIDE" in os.environ and os.environ["HIVE_VERSION_OVERRIDE"] != ""
# If we are using a locally built Hive we do not have a need to pull hive as a
# dependency
if use_override_hive:
cluster_components.extend([hadoop, hbase, tez])
else:
cluster_components.extend([hadoop, hbase, hive, hive_src, tez])
# Ranger is always CDP
cluster_components.append(CdpComponent("ranger",
archive_basename_tmpl="ranger-${version}-admin"))