wzhou-code a5a99adcd2 IMPALA-12376: DataSourceScanNode drop some returned rows ...

DataSourceScanNode does not handle eos properly in function
DataSourceScanNode::GetNext(). Rows, which are returned from
external data source, could be dropped if data_source_batch_size
is set with value which is greater than default value 1024.

Testing:
 - Added end-to-end test with data_source_batch_size as 2048.
   The test failed without fixing, passed with fixing.
   Also added test with data_source_batch_size as 512.
 - Passed core tests.

Change-Id: I978d0a65faa63a47ec86a0127c0bee8dfb79530b
Reviewed-on: http://gerrit.cloudera.org:8080/20636
Reviewed-by: Abhishek Rawat <arawat@cloudera.com>
Tested-by: Wenzhe Zhou <wzhou@cloudera.com>

2023-11-03 00:05:54 +00:00

5.6 KiB

Raw Blame History

View Raw