impala

mirror of https://github.com/apache/impala.git synced 2025-12-25 02:03:09 -05:00

Files

wzhou-code d318f1c992 IMPALA-12377: Improve count(*) performance for jdbc external table

Backend function DataSourceScanNode::GetNext() handles count query
inefficiently. Even when there are no column data returned from
external data source, it still tries to materialize rows and add
rows to RowBatch one by one up to the number of row count. It also
call GetNextInputBatch() multiple times (count / batch_size), while
GetNextInputBatch() invokes JNI function in external data source.

This patch improves the DataSourceScanNode::GetNext() and
JdbcDataSource.getNext() to avoid unnecessary function calls.

Testing:
 - Ran query_test/test_ext_data_sources.py which consists count
   queries for jdbc external table.
 - Passed core-tests.

Change-Id: I9953dca949eb773022f1d6dcf48d8877857635d6
Reviewed-on: http://gerrit.cloudera.org:8080/20653
Reviewed-by: Abhishek Rawat <arawat@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>

2023-11-14 04:48:23 +00:00