mirror of
https://github.com/apache/impala.git
synced 2025-12-19 09:58:28 -05:00
1913ab46ed
To remove the dependency on Python 2, existing scripts need to use python3 rather than python. These commands find those locations (for impala-python and regular python): git grep impala-python | grep -v impala-python3 | grep -v impala-python-common | grep -v init-impala-python git grep bin/python | grep -v python3 This removes or switches most of these locations by various means: 1. If a python file has a #!/bin/env impala-python (or python) but doesn't have a main function, it removes the hash-bang and makes sure that the file is not executable. 2. Most scripts can simply switch from impala-python to impala-python3 (or python to python3) with minimal changes. 3. The cm-api pypi package (which doesn't support Python 3) has been replaced by the cm-client pypi package and interfaces have changed. Rather than migrating the code (which hasn't been used in years), this deletes the old code and stops installing cm-api into the virtualenv. The code can be restored and revamped if there is any interest in interacting with CM clusters. 4. This switches tests/comparison over to impala-python3, but this code has bit-rotted. Some pieces can be run manually, but it can't be fully verified with Python 3. It shouldn't hold back the migration on its own. 5. This also replaces locations of impala-python in comments / documentation / READMEs. 6. kazoo (used for interacting with HBase) needed to be upgraded to a version that supports Python 3. The newest version of kazoo requires upgrades of other component versions, so this uses kazoo 2.8.0 to avoid needing other upgrades. The two remaining uses of impala-python are: - bin/cmake_aux/create_virtualenv.sh - bin/impala-env-versioned-python These will be removed separately when we drop Python 2 support completely. In particular, these are useful for testing impala-shell with Python 2 until we stop supporting Python 2 for impala-shell. The docker-based tests still use /usr/bin/python, but this can be switched over independently (and doesn't impact impala-python) Testing: - Ran core job - Ran build + dataload on Centos 7, Redhat 8 - Manual testing of individual scripts (except some bitrotted areas like the random query generator) Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc Reviewed-on: http://gerrit.cloudera.org:8080/23468 Reviewed-by: Michael Smith <michael.smith@cloudera.com> Tested-by: Michael Smith <michael.smith@cloudera.com>
91 lines
2.5 KiB
Plaintext
91 lines
2.5 KiB
Plaintext
Contents
|
|
|
|
I. Note
|
|
II. Oracle Database server setup
|
|
III. Oracle client setup
|
|
IV. Oracle connectivity, cx_Oracle installation
|
|
TODO: config/setup
|
|
TODO: data load/migrate
|
|
|
|
I. Note
|
|
|
|
Due to licensing, we cannot distribute Oracle client binaries, and of
|
|
course we will not distribute Oracle Database. Please refer to these
|
|
guidelines if you need to use Oracle as a reference database for the
|
|
Random Query Generator.
|
|
|
|
|
|
II. Oracle Database server setup
|
|
|
|
There are lots of choices for Oracle Database. What follows are two
|
|
suggestions:
|
|
|
|
http://www.oracle.com/technetwork/database/database-technologies/express-edition/downloads/index.html
|
|
|
|
https://hub.docker.com/r/wnameless/oracle-xe-11g/
|
|
|
|
Follow their instructions for installation and setup. If you can use
|
|
sqlplus or some other shell to successfully "select 1 from dual;", then
|
|
you're good to go here.
|
|
|
|
|
|
III. Oracle client setup
|
|
|
|
Use cx_Oracle.
|
|
|
|
http://cx-oracle.sourceforge.net/
|
|
|
|
Note: Installing this via pip or similar means won't work until you have
|
|
first installed prerequisites.
|
|
|
|
1. Install Oracle instant-client
|
|
|
|
http://www.oracle.com/technetwork/database/features/instant-client/index-097480.html
|
|
|
|
After you choose your architecture, you need
|
|
|
|
1. the basic client
|
|
2. the SDK
|
|
|
|
You will likely have to register on the Oracle site to download the
|
|
instant client binaries. As of this writing, install instructions are at
|
|
the bottom of the page once you make your architecture choice.
|
|
|
|
2. Additional dependent packages in distribution
|
|
|
|
It's possible either when trying to install cx_Oracle, import it, or use
|
|
it, you will get some linker error. I cannot predict what libraries you
|
|
do and don't have. Here's a list of Linux distributions and known
|
|
package dependencies:
|
|
|
|
Ubuntu 14.04.2 LTS
|
|
libaio1
|
|
|
|
3. Ensure impala-python3 is working. It may be enough to simply run:
|
|
|
|
$ impala-python3
|
|
|
|
Getting it working is out of scope of this document.
|
|
|
|
4. Run these commands to install cx_Oracle into the impala-python3
|
|
virtual environment:
|
|
|
|
$ source "${IMPALA_HOME}"/infra/python/env-gcc${IMPALA_GCC_VERSION}/bin/activate
|
|
(env)$ pip install cx_Oracle==5.2.1
|
|
(env)$ deactivate
|
|
|
|
If pip install fails, double check the following:
|
|
* ORACLE_HOME environment variable
|
|
* LD_LIBRARY_PATH environment variable
|
|
|
|
BUILD.txt packaged with cx_Oracle has more instructions for installing
|
|
cx_Oracle.
|
|
|
|
|
|
IV. Oracle connectivity, cx_Oracle installation
|
|
|
|
Verify cx_Oracle can connect to your Oracle Database. Edit appopriately
|
|
and then run:
|
|
|
|
$ "${IMPALA_HOME}"/tests/comparison/util/verify-oracle-connection.py
|