mirror of
https://github.com/apache/impala.git
synced 2025-12-19 18:12:08 -05:00
ec31324eb5
Tables that are in unloaded state are represented as IncompleteTable. Table level metrics of them won't be used at all but occupy around 7KB of memory for each table. This is a significant amount comparing to the table name strings. This patch skips initializing these metrics for IncompleteTable to save memory usage. This reduces the initial memory requirement to launch catalogd. To avoid other codes unintentionally add new metrics to IncompleteTable, overrides all Table methods that use metrics_ to return simple results, e.g. IncompleteTable.getMedianTableLoadingTime() always returns 0. IncompleteTable.getMetrics() shouldn't be used. Added a Precondition check for this. Tests: - Verified in a heap dump file after loading 1.3M IncompleteTables that the heap usage reduces to 2GB and only few instances of com.codahale.metrics.Timer are created. Previously catalogd OOM in a heap size of 18GB when running global IM, and the number of com.codahale.metrics.Timer instances is similar to the number of IncompleteTables. - Passed CORE tests. Change-Id: If0fcfeab99bbfbefe618d0abf7f2482a0cc5ef9f Reviewed-on: http://gerrit.cloudera.org:8080/23547 Reviewed-by: Riza Suminto <riza.suminto@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Reviewed-by: Michael Smith <michael.smith@cloudera.com>