IMPALA-12607: Bump the GBN and fetch events specific to the db/table

from the metastore

Bump the GBN to 49623641 to leverage HIVE-27499, so that Impala can
directly fetch the latest events specific to the db/table from the
metastore, instead of fetching the events from metastore and then
filtering in the cache matching the DbName/TableName.

Implementation Details:
Currently when a DDL/DML is performed in Impala, we fetch all the
events from metastore based on current eventId and then filter them in
Impala which can be a bottleneck if the events count is huge. This can
be optimized by including db name and/or table name in the notification
event request object and then filter by event type in impala. This can
provide performance boost on tables that generate a lot of events.

Note:
Also included ShowUtils class in hive-minimal-exec jar as it is
required in the current build version

Testing:
1) Did some tests in local cluster
2) Added a test case in MetaStoreEventsProcessorTest

Change-Id: I6aecd5108b31c24e6e2c6f9fba6d4d44a3b00729
Reviewed-on: http://gerrit.cloudera.org:8080/20979
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Sai Hemanth Gantasala
2024-02-01 12:34:00 -08:00
committed by Impala Public Jenkins
parent 09d2f10f4d
commit 68f8a6a1df
7 changed files with 213 additions and 33 deletions

View File

@@ -92,6 +92,8 @@ the same dependencies
<include>org/apache/hive/common/util/TxnIdUtils*</include>
<!-- Needed to support describe formatted command compat with Hive -->
<include>org/apache/hadoop/hive/ql/metadata/**/*</include>
<!-- Needed to support describe formatted command since HIVE-24509 -->
<include>org/apache/hadoop/hive/ql/ddl/ShowUtils.class</include>
<include>org/apache/hadoop/hive/ql/parse/SemanticException.class</include>
<!-- Needed to support Hive udfs -->
<include>org/apache/hadoop/hive/ql/exec/*UDF*</include>