Files
impala/testdata/workloads/functional-query/queries/QueryTest
Tamas Mate 8292e4afdd IMPALA-11864: Iceberg LOAD DATA should not load S3 hidden files
Loading data from S3 did not skip hidden files because the
FileSystemUtil.listFiles() call was returning a RemoteIterator, which
compared to RecursingIterator does not filter the hidden files. This
would make a load fail because the hidden files likely have invalid
magic string.

This commit adds an extra condition to skip hidden files when creating
the CREATE subquery.

Testing:
 - Added E2E test
 - Ran E2E test on S3 build

Change-Id: Iffd179383c2bb2529f6f9b5f8bf5cba5f3553652
Reviewed-on: http://gerrit.cloudera.org:8080/19441
Reviewed-by: Daniel Becker <daniel.becker@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Reviewed-by: Noemi Pap-Takacs <npaptakacs@cloudera.com>
Reviewed-by: Zoltan Borok-Nagy <boroknagyz@cloudera.com>
2023-01-27 12:35:52 +00:00
..
2021-07-06 18:35:30 +00:00
2022-02-11 07:01:58 +00:00