IMPALA-9627: Update utility scripts for Python 3 (part 2)

We're starting to see environments where the system Python ('python') is
Python 3. Updates utility and build scripts to work with Python 3, and
updates check-pylint-py3k.sh to check scripts that use system python.

Fixes other issues found during a full build and test run with Python
3.8 as the default for 'python'.

Fixes a impala-shell tip that was supposed to have been two tips (and
had no space after period when they were printed).

Removes out-of-date deploy.py and various Python 2.6 workarounds.

Testing:
- Full build with /usr/bin/python pointed to python3
- run-all-tests passed with python pointed to python3
- ran push_to_asf.py

Change-Id: Idff388aff33817b0629347f5843ec34c78f0d0cb
Reviewed-on: http://gerrit.cloudera.org:8080/19697
Reviewed-by: Michael Smith <michael.smith@cloudera.com>
Tested-by: Michael Smith <michael.smith@cloudera.com>
This commit is contained in:
Michael Smith
2023-03-24 09:31:22 -07:00
parent 5b0864d7b4
commit 0a42185d17
39 changed files with 115 additions and 650 deletions

View File

@@ -21,12 +21,13 @@ A script for generating arbitrary junit XML reports while building Impala.
These files will be consumed by jenkins.impala.io to generate reports for
easier triaging of build and setup errors.
"""
from __future__ import print_function
from __future__ import absolute_import, division, print_function
import argparse
import codecs
import errno
import os
import re
import sys
import textwrap
from xml.dom import minidom
from xml.etree import ElementTree as ET
@@ -167,7 +168,10 @@ class JunitReport(object):
junit_log_file = os.path.join(junitxml_logdir, filename)
with codecs.open(junit_log_file, encoding="UTF-8", mode='w') as f:
f.write(unicode(self))
if sys.version_info.major < 3:
f.write(unicode(self))
else:
f.write(str(self))
return junit_log_file
@@ -212,7 +216,10 @@ class JunitReport(object):
else:
# This is a string passed in on the command line. Make sure to return it as
# a unicode string.
content = unicode(file_or_string, encoding="UTF-8")
if sys.version_info.major < 3:
content = unicode(file_or_string, encoding="UTF-8")
else:
content = file_or_string
return JunitReport.remove_ansi_escape_sequences(content)
def __unicode__(self):
@@ -223,6 +230,10 @@ class JunitReport(object):
root_node_dom = minidom.parseString(root_node_unicode)
return root_node_dom.toprettyxml(indent=' ' * 4)
def __str__(self):
if sys.version_info.major < 3:
return unicode(self).encode('utf-8')
return self.__unicode__()
def get_options():
"""Parse and return command line options."""

View File

@@ -21,6 +21,7 @@
Some tests that produce JUnitXML include tests that did not run (i.e. status="notrun").
This script walks through the JUnitXML and removes these elements.
"""
from __future__ import absolute_import, division, print_function
from optparse import OptionParser
from xml.etree import ElementTree as ET
from xml.dom import minidom