jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-19 18:12:08 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-9760: Add IMPALA_TOOLCHAIN_PACKAGES_HOME to prepare for GCC7

Browse Source 
The locations for native-toolchain packages in IMPALA_TOOLCHAIN
currently do not include the compiler version. This means that
the toolchain can't distinguish between native-toolchain packages
built with gcc 4.9.2 versus gcc 7.5.0. The collisions can cause
issues when switching back and forth between branches.

This introduces the IMPALA_TOOLCHAIN_PACKAGES_HOME environment
variable, which is a location inside IMPALA_TOOLCHAIN that would
hold native-toolchain packages. Currently, it is set to the same
as IMPALA_TOOLCHAIN, so there is no difference in behavior.
This lays the groundwork to add the compiler version to this
path when switching to GCC7.

Testing:
 - The only impediment to building with
   IMPALA_TOOLCHAIN_PACKAGES_HOME=$IMPALA_TOOLCHAIN/test is
   Impala-lzo. With a custom Impala-lzo, compilation succeeds.
   Either Impala-lzo will be fixed or it will be removed.
 - Core tests

Change-Id: I1ff641e503b2161baf415355452f86b6c8bfb15b
Reviewed-on: http://gerrit.cloudera.org:8080/15991
Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Joe McDonnell
2020-05-27 13:32:43 -07:00
parent f474b03dce
commit 56ee90c598
24 changed files with 105 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
infra/python/bootstrap_virtualenv.py
View File

@@ -199,10 +199,10 @@ def download_toolchain_python():
only the presence of the Python executable is checked in the toolchain location.
'''
toolchain_root = os.environ.get("IMPALA_TOOLCHAIN")
if not toolchain_root:
raise Exception(
"Impala environment not set up correctly, make sure $IMPALA_TOOLCHAIN is set.")
toolchain_packages_home = os.environ.get("IMPALA_TOOLCHAIN_PACKAGES_HOME")
if not toolchain_packages_home:
raise Exception("Impala environment not set up correctly, make sure "
"$IMPALA_TOOLCHAIN_PACKAGES_HOME is set.")
package = ToolchainPackage("python")
if not (os.environ.get(SKIP_TOOLCHAIN_BOOTSTRAP) == 'true'):
@@ -224,12 +224,13 @@ def install_deps():
def have_toolchain():
'''Return true if the Impala toolchain is available'''
return "IMPALA_TOOLCHAIN" in os.environ
return "IMPALA_TOOLCHAIN_PACKAGES_HOME" in os.environ
def toolchain_pkg_dir(pkg_name):
'''Return the path to the toolchain package'''
pkg_version = os.environ["IMPALA_" + pkg_name.upper() + "_VERSION"]
return os.path.join(os.environ["IMPALA_TOOLCHAIN"], pkg_name + "-" + pkg_version)
return os.path.join(os.environ["IMPALA_TOOLCHAIN_PACKAGES_HOME"],
pkg_name + "-" + pkg_version)
def install_compiled_deps_if_possible():
'''Install dependencies that require compilation with toolchain GCC, if the toolchain