jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-19 18:12:08 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-14562: Enable Hierarchical event processing by default

Browse Source 
IMPALA-12709 Added support for hierarchical metastore event processing.
This commit enables hierarchical event processing by default.

hms_event_polling_interval_s can now be set to decimal value (eg: 0.5)
to support millisecond precision interval. Along with that others
configs can be fine tuned, such as:
num_db_event_executors: To set the number of database level event
executors.
num_table_event_executors_per_db_event_executor: To set the number of
table level event executors within a database event executor.
min_event_processor_idle_ms: To set the minimum time to retain idle db
processors and table processors.
max_outstanding_events_on_executors: To set the limit of maximum
outstanding events to process on event executors.

Testing:
- All the testing required to enable this flag is done in IMPALA-12709
and IMPALA-13801.

Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Reviewed-on: http://gerrit.cloudera.org:8080/23687
Reviewed-by: <k.venureddy2103@gmail.com>
Reviewed-by: Quanlong Huang <huangquanlong@gmail.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Sai Hemanth Gantasala
2025-11-18 17:02:36 -08:00
committed by Impala Public Jenkins
parent b77cd73c19
commit bff814f079
2 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
be/src/catalog/catalog-server.cc
View File

@@ -258,7 +258,7 @@ DEFINE_string(common_hms_event_types, "ADD_PARTITION,ALTER_PARTITION,DROP_PARTIT
"ALTER_SCHEMA_VERSION, DROP_SCHEMA_VERSION, CREATE_CATALOG, ALTER_CATALOG,"
"DROP_CATALOG, CREATE_DATACONNECTOR, ALTER_DATACONNECTOR, DROP_DATACONNECTOR.");
DEFINE_bool(enable_hierarchical_event_processing, false,
DEFINE_bool(enable_hierarchical_event_processing, true,
"This configuration is used to enable hierarchical event processing. The default "
"value is false. When enabled, events are fetched, dispatched and processed in "
"different threads.");

2
be/src/common/global-flags.cc
View File

@@ -288,7 +288,7 @@ DEFINE_bool(invalidate_tables_on_memory_pressure, false, "Configure catalogd to
"invalidate_table_timeout_s. To enable this feature, a true flag must be applied to "
"both catalogd and impalad.");
DEFINE_double(hms_event_polling_interval_s, 1,
DEFINE_double(hms_event_polling_interval_s, 0.5,
"Configure catalogd to refresh cached table metadata based on metastore events. "
"These metastore events could be generated by external systems like Apache Hive or "
"a different Impala cluster using the same Hive metastore server as this one. "