jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-19 18:12:08 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-11213: Fixed impala-shell strict hs2 mode for large fetches

Browse Source 
The strict hs2 protocol mode is broken when fetching large results.
The FetchResults.hasMoreRows field is always returned as false.  When
there are no more results, Hive returns an empty batch with no rows.
HIVE-26108 has been filed to support the hasMoreRows field.

Added a framework test that retrieves 1M rows from tpcds. The default
number of rows returned from Hive is 10K so this should be more than
enough to ensure that multiple fetches are done.

Change-Id: Ife436d91e7fe0c30bf020024e20a5d8ad89faa24
Reviewed-on: http://gerrit.cloudera.org:8080/18370
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Reviewed-by: Wenzhe Zhou <wzhou@cloudera.com>
Reviewed-by: Csaba Ringhofer <csringhofer@cloudera.com>
This commit is contained in:
Steve Carlin
2022-03-30 15:55:38 -07:00
committed by Wenzhe Zhou
parent 7281718121
commit c1f99d1369
2 changed files with 19 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
tests/shell/test_shell_commandline.py
View File

@@ -1099,6 +1099,17 @@ class TestImpalaShell(ImpalaTestSuite):
assert "4\t4\t40.4" in result.stdout, result.stdout
def test_large_fetch(self, vector):
query = "select ss_sold_time_sk from tpcds.store_sales limit 50000"
output = run_impala_shell_cmd(vector, ['-q', query, '-B', '--output_delimiter=;'])
assert "Fetched 50000 row(s)" in output.stderr
def test_single_null_fetch(self, vector):
query = "select null"
output = run_impala_shell_cmd(vector, ['-q', query, '-B', '--output_delimiter=;'])
assert "NULL" in output.stdout
assert "Fetched 1 row(s)" in output.stderr
def test_fetch_size(self, vector):
"""Test the --fetch_size option with and without result spooling enabled."""
if vector.get_value('strict_hs2_protocol'):