Zoltan Borok-Nagy a7e71b4523
IMPALA-10358: Correct Iceberg type mappings ...
The Iceberg format spec defines what types to use for different file
formats, e.g.: https://iceberg.apache.org/spec/#parquet
Impala should follow the specification, so this patch
* annotates strings with UTF8 in Parquet metadata
* removes fixed(L) <-> CHAR(L) mapping
* forbids INSERTs when the Iceberg schema has a TIMESTAMPTZ column
This patch also refactors the type/schema conversions as
Impala => Iceberg conversions were duplicated in
IcebergCatalogOpExecutor and IcebergUtil. I introduced the class
'IcebergSchemaConverter' to contain the code for conversions.
Testing:
* added test to check CHAR and VARCHAR types are not allowed
* test that INSERTs are not allowed when the table has TIMESTMAPTZ
* added test to check that strings are annotated with UTF8
Change-Id: I652565f82708824f5cf7497139153b06f116ccd3
Reviewed-on: http://gerrit.cloudera.org:8080/16851
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com >
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com >
2020-12-15 19:17:51 +00:00
2019-08-13 18:55:09 +00:00
2020-09-01 22:27:27 +00:00
2020-05-12 23:29:04 +00:00
2019-07-27 13:45:51 +00:00
2020-09-19 20:10:48 +00:00
2020-08-12 17:45:50 +00:00
2020-05-18 16:51:42 +00:00
2019-07-27 13:45:51 +00:00
2019-11-07 14:30:47 +00:00
2019-07-03 20:46:45 +00:00
2020-05-20 21:00:44 +00:00
2020-05-20 21:00:44 +00:00
2020-05-20 21:00:44 +00:00
2020-09-02 13:29:25 +00:00
2020-05-20 21:00:44 +00:00
2020-06-10 04:02:16 +00:00
2020-06-10 04:02:16 +00:00
2020-06-10 04:02:16 +00:00
2016-09-13 21:57:36 +00:00
2019-11-26 19:14:24 +00:00
2019-10-16 02:04:12 +00:00
2020-07-08 22:59:57 +00:00
2020-08-13 02:22:31 +00:00
2018-09-29 11:59:03 +00:00
2017-02-14 05:56:33 +00:00
2020-05-12 23:29:04 +00:00
2020-12-11 19:51:28 +00:00
2018-08-08 20:51:45 +00:00
2020-10-21 05:20:33 +00:00
2020-03-31 19:11:20 +00:00
2019-09-27 17:18:35 +00:00
2019-09-27 17:18:35 +00:00
2018-05-15 22:23:14 +00:00
2020-09-26 11:11:47 +00:00
2019-09-19 18:46:19 +00:00
2018-10-09 00:45:10 +00:00
2018-04-11 02:21:48 +00:00
2018-11-17 01:48:05 +00:00
2019-08-13 18:55:09 +00:00
2019-01-22 01:20:31 +00:00
2020-05-12 23:29:04 +00:00
2020-05-12 23:29:04 +00:00
2019-08-06 16:38:07 +00:00
2020-05-12 23:29:04 +00:00
2020-05-12 23:29:04 +00:00
2020-12-11 19:51:28 +00:00
2018-12-08 00:21:55 +00:00
2020-12-11 19:51:28 +00:00
2020-08-13 03:10:16 +00:00
2020-11-11 01:39:01 +00:00
2020-06-24 03:47:18 +00:00
2020-09-12 05:30:45 +00:00
2020-06-24 03:47:18 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2019-05-05 02:11:55 +00:00
2019-04-23 13:33:57 +00:00
2020-11-25 14:33:10 +00:00
2020-09-02 10:49:10 +00:00
2020-06-24 16:51:49 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2020-09-28 14:02:35 +00:00
2020-07-08 19:32:15 +00:00
2020-08-13 18:58:23 +00:00
2018-01-25 04:33:11 +00:00
2019-01-14 09:07:03 +00:00
2019-04-23 13:33:57 +00:00
2018-10-09 00:45:10 +00:00
2019-08-14 06:11:45 +00:00
2018-04-11 02:21:48 +00:00
2018-04-11 02:21:48 +00:00
2020-11-11 01:39:01 +00:00
2020-11-11 01:39:01 +00:00
2020-04-02 12:01:41 +00:00
2020-06-05 17:43:32 +00:00
2019-05-15 22:34:28 +00:00
2018-11-01 07:30:58 +00:00
2018-04-12 22:01:35 +00:00
2018-02-02 01:13:08 +00:00
2020-10-21 05:20:33 +00:00
2020-07-31 17:23:45 +00:00
2018-02-15 10:55:55 +00:00
2018-02-15 10:55:55 +00:00
2020-11-30 22:33:57 +00:00
2018-05-12 01:43:37 +00:00
2019-01-12 04:03:26 +00:00
2019-09-25 01:06:19 +00:00
2020-11-20 17:54:08 +00:00
2020-08-13 02:22:31 +00:00
2020-02-22 02:02:56 +00:00
2020-08-12 17:45:50 +00:00
2020-09-30 21:01:54 +00:00
2020-08-12 17:45:50 +00:00
2020-07-14 12:53:51 +00:00
2018-12-11 02:47:16 +00:00
2020-07-14 03:13:18 +00:00
2019-02-07 23:05:30 +00:00
2020-05-12 23:29:04 +00:00
2020-05-12 23:29:04 +00:00
2019-04-23 13:33:57 +00:00
2019-04-23 13:33:57 +00:00
2020-05-12 23:29:04 +00:00
2017-10-10 01:30:33 +00:00
2017-09-21 17:38:08 +00:00
2020-12-11 19:51:28 +00:00
2018-07-03 23:49:44 +00:00
2019-06-05 14:23:23 +00:00
2018-08-16 21:25:00 +00:00
2017-05-09 01:36:46 +00:00
2016-02-19 00:03:15 -08:00
2016-05-12 14:17:59 -07:00
2020-01-17 18:54:33 +00:00
2019-09-27 17:18:35 +00:00
2020-02-14 04:26:16 +00:00
2020-02-11 15:08:39 +00:00
2020-11-20 21:40:28 +00:00
2020-12-02 22:42:12 +00:00
2020-12-15 19:17:51 +00:00
2020-12-15 19:17:51 +00:00
2020-09-09 13:26:42 +00:00
2020-12-10 19:01:08 +00:00
2020-11-23 21:47:55 +00:00
2018-08-09 19:18:08 +00:00
2020-10-21 05:20:33 +00:00
2020-10-21 05:20:33 +00:00
2020-05-11 19:32:08 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2019-06-05 11:15:04 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2020-05-11 19:32:08 +00:00
2018-11-06 00:00:12 +00:00
2020-12-11 19:51:28 +00:00
2020-07-31 17:23:45 +00:00
2016-02-19 00:03:15 -08:00
2020-09-04 00:55:02 +00:00
2019-10-22 04:21:51 +00:00
2020-02-05 22:00:48 +00:00
2020-11-03 20:05:20 +00:00
2020-04-01 15:48:36 +00:00
2020-04-01 15:48:36 +00:00
2020-04-01 15:48:36 +00:00
2020-11-03 20:05:20 +00:00
2018-05-11 22:41:49 +00:00
2020-06-05 00:40:33 +00:00
2018-04-27 21:55:11 +00:00
2020-03-18 18:05:34 +00:00
2020-03-18 18:05:34 +00:00
2020-04-01 15:48:36 +00:00
2020-04-01 15:48:36 +00:00
2017-05-11 20:55:51 +00:00
2017-05-11 20:55:51 +00:00
2018-09-26 06:51:56 +00:00
2020-04-01 15:48:36 +00:00
2019-10-21 20:23:56 +00:00
2017-07-21 21:49:04 +00:00
2020-05-13 22:01:34 +00:00
2018-04-11 02:21:48 +00:00
2020-10-21 05:20:33 +00:00
2020-10-08 22:49:59 +00:00
2018-12-27 21:02:57 +00:00
2020-12-11 19:51:28 +00:00
2018-04-11 02:21:48 +00:00
2020-02-22 02:02:56 +00:00
2020-06-09 16:26:23 +00:00
2019-03-08 04:39:08 +00:00
2020-07-08 22:59:57 +00:00
2020-10-21 05:20:33 +00:00
2019-02-12 20:37:45 +00:00
2016-11-03 11:59:07 +00:00
2017-03-17 19:33:31 +00:00
2016-12-02 01:46:55 +00:00
2019-11-26 07:28:23 +00:00
2016-12-02 01:46:55 +00:00
2016-12-02 01:46:55 +00:00
2019-09-25 16:58:14 +00:00
2020-12-11 19:51:28 +00:00
2018-01-05 20:44:21 +00:00
2020-10-21 20:30:05 +00:00
2017-11-22 22:00:16 +00:00
2020-07-15 17:10:50 +00:00
2016-04-01 05:06:38 +00:00
2018-11-17 01:48:05 +00:00
2019-01-22 01:20:31 +00:00
2019-03-08 04:39:08 +00:00
2020-04-16 15:45:49 +00:00
2019-03-08 04:39:08 +00:00
2019-03-08 04:39:08 +00:00
2019-03-08 04:39:08 +00:00
2020-01-17 18:54:33 +00:00
2019-10-04 18:36:22 +00:00
2018-10-01 13:20:40 +00:00
2018-10-01 13:20:40 +00:00
2020-02-22 02:02:56 +00:00
2017-08-19 20:56:10 +00:00
2020-09-22 22:19:41 +00:00
2017-03-03 01:43:42 +00:00
2016-05-12 14:17:48 -07:00
2017-03-09 05:07:44 +00:00
2019-03-14 22:43:50 +00:00
2018-03-08 04:48:36 +00:00
2018-01-17 04:18:24 +00:00
2016-08-25 10:20:36 +00:00
2018-09-10 16:09:41 +00:00
2018-09-10 16:09:41 +00:00
2016-06-20 15:37:18 -07:00
2016-06-20 15:37:18 -07:00
2020-06-18 21:00:27 +00:00
2018-08-22 18:08:20 +00:00
2017-05-09 15:47:21 +00:00
2019-08-24 00:32:28 +00:00
2019-05-07 00:36:56 +00:00
2019-04-09 08:54:23 +00:00
2018-02-22 00:57:46 +00:00
2018-11-27 02:01:12 +00:00
2019-05-10 11:46:38 +00:00
2019-05-10 11:46:38 +00:00
2019-05-10 11:46:38 +00:00
2020-11-24 18:35:07 +00:00
2020-12-10 19:01:08 +00:00
2018-03-22 02:47:33 +00:00
2017-07-22 05:53:06 +00:00
2020-05-21 18:40:20 +00:00
2018-08-23 15:55:53 +00:00
2016-10-14 05:41:22 +00:00
2020-08-25 15:42:01 +00:00
2020-12-11 19:51:28 +00:00
2018-04-11 02:21:48 +00:00
2019-05-10 12:06:01 +00:00
2020-05-20 23:03:23 +00:00
2020-05-20 23:03:23 +00:00
2020-05-20 23:03:23 +00:00
2018-08-15 04:15:46 +00:00
2018-08-15 04:15:46 +00:00
2019-02-07 21:41:52 +00:00
2017-12-13 10:04:40 +00:00
2020-12-10 07:15:36 +00:00
2020-04-07 15:10:27 +00:00
2020-04-02 12:01:41 +00:00
2019-08-06 02:35:54 +00:00
2020-05-14 02:52:16 +00:00
2017-11-17 21:33:51 +00:00
2020-12-08 03:15:44 +00:00
2020-04-14 15:42:13 +00:00
2020-04-14 15:42:13 +00:00
2020-05-15 21:46:38 +00:00
2018-11-17 01:48:05 +00:00
2019-02-12 20:37:45 +00:00
2020-07-08 19:32:15 +00:00
2019-01-09 05:26:50 +00:00
2016-08-31 03:12:30 +00:00
2020-12-10 19:01:08 +00:00
2020-04-02 12:01:41 +00:00
2019-09-26 18:35:06 +00:00
2020-11-20 21:40:28 +00:00
2016-09-29 21:14:13 +00:00
2020-05-12 23:29:04 +00:00
2020-04-14 12:18:21 +00:00
2017-08-05 01:03:02 +00:00
2017-08-05 01:03:02 +00:00
2018-04-28 23:41:39 +00:00
2018-02-02 01:13:08 +00:00
2020-10-21 05:20:33 +00:00
2018-10-30 22:17:49 +00:00
2020-03-03 06:12:24 +00:00
2020-03-17 23:29:45 +00:00
2020-08-25 16:11:20 +00:00
2020-04-24 20:56:58 +00:00
2018-04-28 23:41:39 +00:00
2020-08-25 16:11:20 +00:00
2017-08-29 23:01:10 +00:00
2019-01-22 01:20:31 +00:00
2019-01-22 01:20:31 +00:00
2020-05-15 16:49:32 +00:00
2020-08-13 03:10:16 +00:00
2017-03-03 01:43:42 +00:00
2017-03-03 01:43:42 +00:00
2018-11-01 21:27:13 +00:00
2018-11-01 21:27:13 +00:00
2019-08-22 05:03:03 +00:00
2020-08-25 23:08:40 +00:00
2020-10-21 05:20:33 +00:00
2020-12-08 03:15:44 +00:00
2018-04-11 02:21:48 +00:00
2016-02-28 21:31:37 -08:00
2019-09-27 01:54:39 +00:00
2020-10-21 05:20:33 +00:00
2020-05-12 23:29:04 +00:00
2016-11-09 03:27:12 +00:00
2019-04-23 13:33:57 +00:00
2016-11-09 03:27:12 +00:00
2018-04-11 02:21:48 +00:00
2016-12-08 04:53:53 +00:00
2016-12-08 04:53:53 +00:00
2016-11-09 03:27:12 +00:00
2018-11-20 23:49:11 +00:00
2019-05-15 22:34:28 +00:00
2020-10-21 05:20:33 +00:00
2020-06-15 23:42:12 +00:00
2018-07-03 20:34:27 +00:00
2020-09-18 22:12:47 +00:00
2018-02-02 01:13:08 +00:00
2019-05-24 15:08:19 +00:00
2019-05-24 15:08:19 +00:00
2020-07-14 03:13:18 +00:00
2019-04-23 13:33:57 +00:00
2020-10-21 05:20:33 +00:00
a7e71b4523