mirror of
https://github.com/apache/impala.git
synced 2025-12-19 18:12:08 -05:00
64abca481f
After IMPALA-14074, the passive catalogd can have a warmed up metadata cache during failover (with catalogd_ha_reset_metadata_on_failover=false and a non-empty warmup_tables_config_file). However, it could still use a stale metadata cache when some pending HMS events generated by the previous active catalogd are not applied yet. This patch adds a wait during HA failover to ensure HMS events before the failover happens are all applied on the new active catalogd. The timeout is configured by a new flag which defaults to 300 (5 minutes): catalogd_ha_failover_catchup_timeout_s. When timeout happens, by default catalogd will fallback to resetting all metadata. Users can decide whether to reset or continue using the current cache. This is configured by another flag, catalogd_ha_reset_metadata_on_failover_catchup_timeout. Since the passive catalogd depends on HMS event processing to keep its metadata up-to-date with the active catalogd, this patch adds validation to avoid starting catalogd with catalogd_ha_reset_metadata_on_failover set to false and hms_event_polling_interval_s <= 0. This patch also makes catalogd_ha_reset_metadata_on_failover a non-hidden flag so it's shown in the /varz web page. Tests: - Ran test_warmed_up_metadata_after_failover 200 times. Without the fix, it usually fails in several runs. - Added new tests for the new flags. Change-Id: Icf4fcb0e27c14197f79625749949b47c033a5f31 Reviewed-on: http://gerrit.cloudera.org:8080/23174 Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>