mirror of
https://github.com/apache/impala.git
synced 2025-12-19 18:12:08 -05:00
1142c7b58e
Bootstrapping the impala-python virtualenv requires multiple rounds of pip installs with different sets of requirements. This consolidates the requirements.txt, stage2-requirements.txt, and compiled-requirements.txt into a single requirements.txt. This will make it easier to upgrade python packages. This also splits out setuptools into its own setuptools-requirements.txt. Setuptools is used during the pip install for several of the dependencies. Recent versions of setuptools do not support Python 2, but some of the install tools (like easy_install) don't know how to pick a version of setuptools that works with Python 2. Splitting it out to its own requirements file lets us pin the version. To make review easier, this does not change any of the versions of the dependencies. It also leaves the stage2-requirements.txt and compiled-requirements.txt split out in separate sections of requirements.txt. These will later be turned into a single alphabetical list. Testing: - Tested impala-python locally - Ran GVO Change-Id: I8e920e5a257f1e0613065685078624a50d59bf2e Reviewed-on: http://gerrit.cloudera.org:8080/17226 Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com> Tested-by: Joe McDonnell <joemcdonnell@cloudera.com>
To install new packages: 1) Add your package to deps/requirements.txt, or deps/compiled-requirements.txt if the the package needs a C/C++ compiler to build . You should specify the version number using the "foo == x.y.z" notation so future upgrades can be done automatically. 2) Run deps/download_requirements, it will download the package to the deps dir. 3) Run the "impala-python" command, this should detect that requirements.txt changed and automatically rebuild the virtualenv. 4) Now in the python prompt, you should be able to import the new module. To upgrade a package: 1) Edit deps/requirement.txt to use the version you need. 2) Go to step 2 above.