mirror of
https://github.com/apache/impala.git
synced 2026-02-03 09:00:39 -05:00
78ec9903cc
This patch upgrades pytest version from 2.9.2 to 6.2.5, the highest pytest version available without the need to upgrade setuptools. The pytest requirement is moved to py3-requirements.txt. We can not go back to test with Python2 + pytest-2.9.2 after this patch because our test script need to be adjusted as well. This is OK since we have default testing using Python3 since IMPALA-14333 and stop building impala-shell for Python2 since IMPALA-14606. The adjustment are follows: - Replace deprecated @pytest.yield_fixture with plain @pytest.fixture. - Replace --resultlog parameter (removed in version 6.0) with --report-log from pytest-reportlog plugin. - Make impala-shell.sh bootstrap Python3 venv (infra/python/env-gcc10.4.0-py3/) by default. Python2 venv (infra/python/env-gcc10.4.0/) is not bootstrapped automatically anymore. - Upgrade execnet to version 1.9.0. This is required by pytest-xdist==2.4.0. - Remove unused pytest-runner plugin - Add -Wonce argument in pytest.ini to slightly suppress warnings. - Add "junit_logging = system-err" option at pytest.ini to continue logging stderr output to junit xml file. - Fix rootdir and pytest.ini path programmatically at run-tests.py because individual pytest-xdist worker often does not pick this up correctly during parallel run. With pytest-xdist==2.4.0, parallel EE tests does not show verbose individual test names. It only shows pytest progress marker like following line ...ss....s................................s............................. This is a known limitation in pytest-xdist because execnet, the underlying library used for communication between master and workers, does not support transferring stdout/stderr from workers. https://pytest-xdist.readthedocs.io/en/stable/known-limitations.html Read following links for more detail about the deprecation notes: https://docs.pytest.org/en/stable/deprecations.html Change SKIP_SSL_MSG default to empty string because skipif does not accept None reason anymore. Removed run-process-failure-tests.sh (unused after IMPALA-5534) and unused pytest fixtures. Fixed erroneous log formatting in test_restart_services.py. Fixed small warnings found by pytest-6.2.5 that help stabilize exhaustive tests run at: - test_automatic_invalidation.py - test_calcite_planner.py - test_events_custom_configs.py - test_session_expiration.py - test_shell_interactive.py - auto_scaler.py - concurrent_workload.py - hdfs_util.py Most of the remaining warnings are about not closing resources properly at the end of test. These warnings should be addressed in follow up patches. Testing: - Pass exhaustive tests. Change-Id: Ic3812fe976ef09ac48753dee30151714f4752c24 Reviewed-on: http://gerrit.cloudera.org:8080/23842 Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>