mirror of
https://github.com/apache/impala.git
synced 2025-12-19 18:12:08 -05:00
5071f54a4c
impala-python currently gets its Thrift from the toolchain by adding the appropriate Thrift toolchain directories to the PYTHONPATH. This is a problem when switching to Python 3, because the toolchain Thrift was built with Python 2 and this can produce complicated bugs. In general, it is also not a good idea to get Python dependencies from the toolchain. This switches to installing Thrift into the impala-python virtualenv, which lets the different Python versions have their own copy of compiled files. Testing: - Ran a core job Change-Id: Ib36e8a1ce8d446b69b08e81ea458f95c158e28f5 Reviewed-on: http://gerrit.cloudera.org:8080/21046 Reviewed-by: Michael Smith <michael.smith@cloudera.com> Reviewed-by: Wenzhe Zhou <wzhou@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
To install new packages: 1) Add your package to deps/requirements.txt, or deps/compiled-requirements.txt if the the package needs a C/C++ compiler to build . You should specify the version number using the "foo == x.y.z" notation so future upgrades can be done automatically. 2) Run deps/download_requirements, it will download the package to the deps dir. 3) Run the "impala-python" command, this should detect that requirements.txt changed and automatically rebuild the virtualenv. 4) Now in the python prompt, you should be able to import the new module. To upgrade a package: 1) Edit deps/requirement.txt to use the version you need. 2) Go to step 2 above.