IMPALA-10608 followup: Detect the virtualenv tarball version

When rebasing from an older commit, the version change
in virtualenv can cause there to be multiple virtualenv
tarballs of different versions in the infra/python/deps
directory. bootstrap_virtualenv.py currently doesn't
handle this gracefully, because it is looking for all
virtualenv*.tar.gz files and fails when it finds more
than one.

This changes bootstrap_virtualenv.py to get the virtualenv
version from the requirements.txt file and only look
for the tarball with that version. If it fails to get
the version, it falls back to the old method.

Testing:
 - Copied virtualenv-16.7.10.tar.gz to virtualenv-16.7.9.tar.gz
   and verified that bootstrap_virtualenv.py works

Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40
Reviewed-on: http://gerrit.cloudera.org:8080/17249
Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com>
Tested-by: Joe McDonnell <joemcdonnell@cloudera.com>

infra/python/bootstrap_virtualenv.py

@@ -80,10 +80,34 @@ def delete_virtualenv_if_exist():
     shutil.rmtree(ENV_DIR)
 
 
+def detect_virtualenv_version():
+  with open(REQS_PATH, "r") as reqs_file:
+    for line in reqs_file:
+      line = line.strip()
+      # Ignore blank lines and comments
+      if len(line) == 0 or line[0] == '#':
+        continue
+      if line.find("virtualenv") != -1 and line.find("==") != -1:
+        packagestring, version = [a.strip() for a in line.split("==")]
+        if packagestring == "virtualenv":
+          LOG.debug("Detected virtualenv version {0}".format(version))
+          return version
+  # If the parsing didn't work, don't raise an exception.
+  return None
+
+
 def create_virtualenv():
   LOG.info("Creating python virtualenv")
   build_dir = tempfile.mkdtemp()
-  file = tarfile.open(find_file(DEPS_DIR, "virtualenv*.tar.gz"), "r:gz")
+  # Try to find the virtualenv version by parsing the requirements file
+  # Default to "*" if we can't figure it out.
+  virtualenv_version = detect_virtualenv_version()
+  if virtualenv_version is None:
+    virtualenv_version = "*"
+  # Open the virtualenv tarball
+  virtualenv_tarball = \
+      find_file(DEPS_DIR, "virtualenv-{0}.tar.gz".format(virtualenv_version))
+  file = tarfile.open(virtualenv_tarball, "r:gz")
   for member in file.getmembers():
     file.extract(member, build_dir)
   file.close()