jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2026-02-02 06:00:36 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-11964: Make sure Impala returns error for Iceberg tables with equality deletes

Browse Source 
Impala only supports position deletes currently. It should raise an
error when equality deletes are encountered.

We already had a check for this when the query was planned by Iceberg.
But when we were using cached metadata the check was missing. This means
that Impala could return bogus results in the presence of equality
delete files. This patch adds check for the latter case as well.

Tables with equality delete files are still loadable by Impala, and
users can still query snapshots of it if they don't have equality
deletes.

Testing:
 * added e2e tests

Change-Id: I14d7116692c0e47d0799be650dc323811e2ee0fb
Reviewed-on: http://gerrit.cloudera.org:8080/19601
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Zoltan Borok-Nagy
2023-03-07 18:43:53 +01:00
committed by Impala Public Jenkins
parent fe29a356b9
commit f2cb2c9ceb
18 changed files with 269 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test vendored
View File

@@ -662,3 +662,24 @@ select * from functional_parquet.iceberg_alltypes_part for system_time as of '20
---- CATCH
IllegalArgumentException: Cannot find a snapshot older than 2000-01-01 01:02:03
====
---- QUERY
# Querying a table with equality deletes is not allowed.
# We don't use time-travel, so we plan the query from cached metadata.
select * from functional_parquet.iceberg_v2_delete_equality;
---- CATCH
ImpalaRuntimeException: Iceberg table functional_parquet.iceberg_v2_delete_equality has EQUALITY delete file which is currently not supported by Impala
====
---- QUERY
# Querying a table with equality deletes is not allowed.
# Use time-travel based on snapshot id.
select * from functional_parquet.iceberg_v2_delete_equality for system_version as of 5725822353600261755;
---- CATCH
ImpalaRuntimeException: Iceberg table functional_parquet.iceberg_v2_delete_equality has EQUALITY delete file which is currently not supported by Impala
====
---- QUERY
# Querying a table with equality deletes is not allowed.
# Use time-travel based on timestamp.
select * from functional_parquet.iceberg_v2_delete_equality for system_time as of now();
---- CATCH
ImpalaRuntimeException: Iceberg table functional_parquet.iceberg_v2_delete_equality has EQUALITY delete file which is currently not supported by Impala
====

10
testdata/workloads/functional-query/queries/QueryTest/iceberg-query.test vendored
View File

@@ -1160,3 +1160,13 @@ where i.action in ('view') and j.id=1 and j.id=i.id;
---- TYPES
int
====
---- QUERY
# We can query a snapshot if it doesn't have equality deletes.
select * from iceberg_v2_delete_equality for system_version as of 6816997371555012807;
---- RESULTS
1,'a'
2,'b'
3,'c'
---- TYPES
BIGINT,STRING
====