IMPALA-10608: Update kudu-python version and remove some unused packages

This updates kudu-python to version 1.14.0 (from 1.2.0).
As part of this, it disables ccache for bootstrap_virtualenv.py.
ccache wasn't working anyway, because pip install uses random
temporary directories. It also needs to copy a few files to
the build directory for the Kudu install. The advantage to
upgrading is that the new version no longer has a numpy dependency.

Additionally, this modifies a few minor packages:
 - virtualenv moves to the latest version prior to the rewrite
   that accompanied version 20 (i.e. 16.10.7).
 - setuptools moves to the last version that supports python 2.7 (44.1.1)
 - remove botos3, ipython, and ordereddict

These changes speed up installing the virtualenv
Before:
real	3m11.956s
user	2m49.620s
sys	0m14.266s
After:
real    1m38.798s
user    1m33.591s
sys     0m8.112s

Testing:
 - Hand tests, GVO run

Change-Id: Ib47770df9e46de448fe2bffef7abe2c3aa942fb9
Reviewed-on: http://gerrit.cloudera.org:8080/17231
Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Joe McDonnell
2021-03-24 20:47:12 -07:00
parent 9670c1455d
commit e7fc18c4ea
7 changed files with 15 additions and 60 deletions

View File

@@ -108,16 +108,6 @@ def exec_cmd(args, **kwargs):
return output
def use_ccache():
'''Returns true if ccache is available and should be used'''
if 'DISABLE_CCACHE' in os.environ: return False
try:
exec_cmd(['ccache', '-V'])
return True
except:
return False
def select_cc():
'''Return the C compiler command that should be used as a string or None if the
compiler is not available '''
@@ -127,7 +117,6 @@ def select_cc():
toolchain_gcc_dir = toolchain_pkg_dir("gcc")
cc = os.path.join(toolchain_gcc_dir, "bin/gcc")
if not os.path.exists(cc): return None
if use_ccache(): cc = "ccache %s" % cc
return cc
@@ -284,6 +273,10 @@ def install_kudu_client_if_possible():
env = dict(os.environ)
env["KUDU_HOME"] = fake_kudu_build_dir
kudu_client_dir = find_kudu_client_install_dir()
# Copy the include directory to the fake build directory
kudu_include_dir = os.path.join(kudu_client_dir, "include")
shutil.copytree(kudu_include_dir,
os.path.join(fake_kudu_build_dir, "build", "latest", "src"))
env["CPLUS_INCLUDE_PATH"] = os.path.join(kudu_client_dir, "include")
env["LIBRARY_PATH"] = os.path.pathsep.join([os.path.join(kudu_client_dir, 'lib'),
os.path.join(kudu_client_dir, 'lib64')])