mirror of
https://github.com/apache/impala.git
synced 2026-02-02 06:00:36 -05:00
2d3289027c
If an Iceberg table is frequently updated/written to in small batches, a lot of small files are created. This decreases read performance. Similarly, frequent row-level deletes contribute to this problem by creating delete files, which have to be merged on read. So far INSERT OVERWRITE (rewriting the table with itself) has been used to compact Iceberg tables. However, it comes with some RESTRICTIONS: - The table should not have multiple partition specs/partition evolution. - The table should not contain complex types. The OPTIMIZE statement offers a new syntax and a solution limited to Iceberg tables to enhance read performance for subsequent operations. See IMPALA-12293 for details. Syntax: OPTIMIZE TABLE <table_name>; This first patch introduces the new syntax, temporarily as an alias for INSERT OVERWRITE. Note that executing OPTIMIZE TABLE requires ALL privileges. Testing: - negative tests - FE planner test - Ranger test - E2E tests Change-Id: Ief42537499ffe64fafdefe25c8d175539234c4e7 Reviewed-on: http://gerrit.cloudera.org:8080/20405 Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>