jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2026-01-04 18:00:57 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-13551: Produce the shell tarball by pip installing impala-shell

Browse Source 
Currently, the shell tarball maintains its own packaging code
and directory layout. This is very complicated and currently has
several Python packages directly checked into our repository.

To simplify it, this changes the shell tarball to be based on
pip installing the pypi package. Specifically, the new directory
structure for an unpack shell tarball is:
impala-shell-4.5.0-SNAPSHOT/
  impala-shell
  install_py${PYTHON_VERSION}/
  install_py${ANOTHER_PYTHON_VERSION}/
For example, install_py2.7 is the Python 2.7 pip install of impala-shell.
install_py3.8 is a Python 3.8 pip install of impala-shell. This means
that the impala-shell script simply picks the install for the
specified version of python and uses that pip install directory.
To make this more consistent across different Linux distributions, this
upgrades pip in the virtualenv to the latest.

With this, ext-py and pkg_resources.py can be removed.

This requires rearranging the shell build code. Specifically, this splits
out the code that generates impala_build_version.py so that it can run
before generating the pypi package. The shell tarball now has a dependency
on the pypi package and must run after it.

This builds on Michael Smith's work from IMPALA-11399.

Testing:
 - Ran shell tests locally
 - Built on Centos 7, Redhat 8 & 9, Ubuntu 20 & 22, SLES 15

Change-Id: Ifbb66ab2c5bc7180221f98d9bf5e38d62f4ac036
Reviewed-on: http://gerrit.cloudera.org:8080/20171
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Joe McDonnell
2023-07-02 15:44:41 -07:00
committed by Impala Public Jenkins
parent 5b4afb4f8f
commit aefd1b0920
179 changed files with 137 additions and 49422 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
shell/packaging/make_python_package.sh
View File

@@ -27,8 +27,8 @@
# Until the thrift-generated python files in ${IMPALA_HOME}/shell/gen-py
# have been created by the build process, this script will not work.
# It also relies upon the impala_build_version.py file created by the
# parent packaging script, ${IMPALA_HOME}/shell/make_shell_tarball.sh,
# which needs to be run before this script will work.
# ${IMPALA_HOME}/shell/gen_impala_build_version.sh script, which needs
# to run before this script will work.
#
# After those files exist, however, this script can be run again at will.