mirror of
https://github.com/apache/impala.git
synced 2026-01-31 09:00:19 -05:00
d2f866f9a1
With IMPALA-10627 we switched to use standard Iceberg table properties: https://iceberg.apache.org/configuration/ E.g. we switched from 'iceberg.file_format' to 'write.format.default'. For backward compatibility we also support 'iceberg.file_format'. Though the support is not perfect as it causes a crash in some cases. Impala crashes when the following conditions met: * local catalog mode is being used * Iceberg table is being queried * the data file format is ORC * 'iceberg.file_format' is set instead of 'write.format.default' table property * Query is "select count(*) from t;" Impala wrongly assumes that PARQUET is being used and tries to apply the count star optimization. It is not implemented for the ORC scanner and causes it to crash. This patch fixes the wrong assumption. Also it fixes the HdfsOrcScanner, so it won't crash in release mode but raise an error. This patch also enables UNSETting the file format table property for Iceberg tables. This table property was already enabled for modifications (changing the value via SET TBLPROPERTIES). Testing: * added e2e test for the above conditions Change-Id: Iafd9baef1c124d7356a14ba24c571567629a5e50 Reviewed-on: http://gerrit.cloudera.org:8080/17877 Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>