jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-23 21:08:39 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-12908: (Addendum) use RUNTIME_FILTER_WAIT_TIME_MS for tuple cache TPC testing

Browse Source 
When runtime filters arrive after tuple caching has occurred, they
can't filter the cached results. This can lead to larger tuple caching
result sets than expected, causing correctness check failures in TPC
tests.

While other solutions may exist, extending RUNTIME_FILTER_WAIT_TIME_MS
is a simple fix by ensuring runtime filters are applied before tuple
caching.

Also set the query option enable_tuple_cache_verification to false
by default, as the filter arrival time may affect the correctness
check. To avoid flaky tests, change to use a more conservative
approach and only enable the correctness check when explicitly
specified by the testcase.

Tests:
Verified TPC tests pass correctness checks with increased runtime
filter wait time.

Change-Id: Ie70a87344c436ce8e2073575df5c5bf762ef562d
Reviewed-on: http://gerrit.cloudera.org:8080/21898
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Yida Wu
2024-10-03 14:09:55 -07:00
committed by Impala Public Jenkins
parent 19678ae65c
commit 0767ae065a
3 changed files with 12 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
tests/query_test/test_tuple_cache_tpc_queries.py
View File

@@ -19,6 +19,7 @@
from __future__ import absolute_import, division, print_function
import pytest
from tests.common.environ import IS_TUPLE_CACHE_CORRECT_CHECK
from tests.common.impala_test_suite import ImpalaTestSuite
from tests.common.skip import SkipIf
from tests.common.test_dimensions import create_single_exec_option_dimension
@@ -29,6 +30,11 @@ MT_DOP_VALUES = [0, 4]
def run_tuple_cache_test(self, vector, query, mtdop):
vector.get_value('exec_option')['enable_tuple_cache'] = True
# Use a long runtime filter wait time (1 minute) to ensure filters arrive before
# generating the tuple cache for correctness check.
if IS_TUPLE_CACHE_CORRECT_CHECK:
vector.get_value('exec_option')['runtime_filter_wait_time_ms'] = 60000
vector.get_value('exec_option')['enable_tuple_cache_verification'] = True
vector.get_value('exec_option')['mt_dop'] = mtdop
# Run twice to test write and read the tuple cache.
self.run_test_case(query, vector)