jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-19 18:12:08 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-7381: Prevent build failure after switching to new CDH_BUILD_NUMBER

Browse Source 
Switching to a new CDH_BUILD_NUMBER requires downloading new CDH
components as well as forcing Maven to update its local repository.
This patch updates the CDH_COMPONENTS_HOME to include the
CDH_BUILD_NUMBER which will automatically download the new CDH
components after switching to a new CDH_BUILD_NUMBER. When running
a build if it detects that a new CDH_BUILD_NUMBER has changed, the
build will force an update to the local Maven repository. This helps
to prevent build failure even on a fresh Git clone due to stale local
Maven repository.

Testing:
- Manually tested by running buildall.sh with different CDH_BUILD_NUMBER

Change-Id: Ib0ad9c2258663d3bd7470e6df921041d1ca0c0be
Reviewed-on: http://gerrit.cloudera.org:8080/11099
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Fredy Wijaya
2018-07-31 20:06:29 -07:00
committed by Impala Public Jenkins
parent 479c83cf72
commit d5ada970c6
4 changed files with 16 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@@ -13,6 +13,7 @@ org.eclipse.jdt.ui.prefs
load-*-generated.sql
bin/version.info
bin/impala-config-local.sh
.cdh
# distcc options
.impala_compiler_opts

3
README.md
View File

@@ -65,7 +65,8 @@ can do so through the environment variables and scripts listed below.
| IMPALA_HOME | | Top level Impala directory |
| IMPALA_TOOLCHAIN | "${IMPALA_HOME}/toolchain" | Native toolchain directory (for compilers, libraries, etc.) |
| SKIP_TOOLCHAIN_BOOTSTRAP | "false" | Skips downloading the toolchain any python dependencies if "true" |
| CDH_COMPONENTS_HOME | "${IMPALA_HOME}/toolchain/cdh_components" OR "${IMPALA_HOME}/thirdparty" (if detected) | If a thirdparty directory is present, components found here will override anything in IMPALA_TOOLCHAIN. |
| CDH_BUILD_NUMBER | | Identifier to indicate the CDH build number
| CDH_COMPONENTS_HOME | "${IMPALA_HOME}/toolchain/cdh_components-${CDH_BUILD_NUMBER}" OR "${IMPALA_HOME}/thirdparty" (if detected) | If a thirdparty directory is present, components found here will override anything in IMPALA_TOOLCHAIN. |
| CDH_MAJOR_VERSION | "5" | Identifier used to uniqueify paths for potentially incompatible component builds. |
| IMPALA_CONFIG_SOURCED | "1" | Set by ${IMPALA_HOME}/bin/impala-config.sh (internal use) |
| JAVA_HOME | "/usr/lib/jvm/${JAVA_VERSION}" | Used to locate Java |

2
bin/impala-config.sh
View File

@@ -475,7 +475,7 @@ export PATH="$IMPALA_HOME/bin:$IMPALA_TOOLCHAIN/cmake-$IMPALA_CMAKE_VERSION/bin/
# The directory in which all the thirdparty CDH components live.
if [ "${DOWNLOAD_CDH_COMPONENTS}" = true ]; then
export CDH_COMPONENTS_HOME="$IMPALA_TOOLCHAIN/cdh_components"
export CDH_COMPONENTS_HOME="$IMPALA_TOOLCHAIN/cdh_components-$CDH_BUILD_NUMBER"
else
export CDH_COMPONENTS_HOME="$IMPALA_HOME/thirdparty"
fi

12
buildall.sh
View File

@@ -465,6 +465,18 @@ create_log_dirs
bootstrap_dependencies
# Create .cdh file that contains the CDH_BUILD_NUMBER. If the content
# of the file is different than the one in the environment variable,
# append -U into IMPALA_MAVEN_OPTION to force Maven to update its local
# cache.
CDH_FILE="${IMPALA_HOME}/.cdh"
if [[ -f ${CDH_FILE} ]]; then
if [[ $(cat ${CDH_FILE}) != ${CDH_BUILD_NUMBER} ]]; then
export IMPALA_MAVEN_OPTIONS="${IMPALA_MAVEN_OPTIONS} -U"
fi
fi
echo "${CDH_BUILD_NUMBER}" > ${CDH_FILE}
if [[ "$BUILD_FE_ONLY" -eq 1 ]]; then
build_fe
exit 0