Vihang Karajgaonkar 6ebea33a9d
IMPALA-9092: Add support for creating external Kudu table ...
In HMS-3 the translation layer converts a managed kudu table into an
external kudu table and adds additional table property
'external.table.purge' to 'true'. This means any installation which
is using HMS-3 (or a Hive version which has HIVE-22158) will always
create Kudu tables as external tables. This is problematic since the
output of show create table will now be different and may confuse
the users.
In order to improve the user experience of such synchronized tables
(external tables with external.table.purge property set to true),
this patch adds support in Impala to create
external Kudu tables. Previous versions of Impala disallowed
creating a external Kudu table if the Kudu table did not exist.
After this patch, Impala will check if the Kudu table exists and if
it does not it will create a Kudu table based on the schema provided
in the create table statement. The command will error out if the Kudu
table already exists. However, this applies to only the synchronized
tables. Previous way to create a pure external table behaves the
same.
Following syntax of creating a synchronized table is now allowed:
CREATE EXTERNAL TABLE foo (
id int PRIMARY KEY,
name string)
PARTITION BY HASH PARTITIONS 8
STORED AS KUDU
TBLPROPERTIES ('external.table.purge'='true')
The syntax is very similar to creating a managed table, except for
the EXTERNAL keyword and additional table property. A synchronized
table will behave similar to managed Kudu tables (drops and renames
are allowed). The output of show create table on a synchronized
table will display the full column and partition spec similar to the
managed tables.
Testing:
1. After the CDP version bump all of the existing Kudu tables now
create synchronized tables so there is good coverage there.
2. Added additional tests which create synchronized tables and
compares the show create table output.
3. Ran exhaustive tests with both CDP and CDH builds.
Change-Id: I76f81d41db0cf2269ee1b365857164a43677e14d
Reviewed-on: http://gerrit.cloudera.org:8080/14750
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com >
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com >
2019-12-13 23:02:13 +00:00
2019-08-13 18:55:09 +00:00
2019-11-07 14:30:47 +00:00
2019-08-16 23:22:53 +00:00
2019-07-27 13:45:51 +00:00
2019-08-15 13:02:55 +00:00
2019-08-27 11:58:07 +00:00
2019-07-27 13:45:51 +00:00
2019-11-07 14:30:47 +00:00
2019-07-03 20:46:45 +00:00
2019-08-27 18:51:55 +00:00
2019-11-07 14:30:47 +00:00
2019-06-21 03:28:43 +00:00
2019-09-27 01:54:39 +00:00
2019-06-19 05:58:10 +00:00
2016-09-13 21:57:36 +00:00
2019-11-26 19:14:24 +00:00
2019-10-16 02:04:12 +00:00
2018-09-29 11:59:03 +00:00
2018-09-29 11:59:03 +00:00
2017-02-14 05:56:33 +00:00
2018-12-08 00:21:55 +00:00
2019-09-26 18:35:06 +00:00
2019-06-03 09:24:38 +00:00
2018-08-08 20:51:45 +00:00
2019-04-23 13:33:57 +00:00
2019-10-04 18:36:22 +00:00
2019-09-27 17:18:35 +00:00
2019-09-27 17:18:35 +00:00
2018-05-15 22:23:14 +00:00
2018-02-13 08:29:03 +00:00
2019-06-26 01:58:10 +00:00
2019-09-19 18:46:19 +00:00
2018-10-09 00:45:10 +00:00
2018-04-11 02:21:48 +00:00
2018-11-17 01:48:05 +00:00
2019-08-13 18:55:09 +00:00
2019-01-22 01:20:31 +00:00
2019-05-02 23:33:32 +00:00
2019-05-02 23:33:32 +00:00
2019-08-06 16:38:07 +00:00
2019-04-23 13:33:57 +00:00
2018-12-08 00:21:55 +00:00
2019-08-15 14:40:04 +00:00
2018-12-08 00:21:55 +00:00
2015-04-21 19:27:50 +00:00
2019-09-27 01:54:39 +00:00
2019-04-16 22:40:13 +00:00
2019-08-29 12:34:56 +00:00
2019-09-26 18:35:06 +00:00
2017-05-12 15:43:30 +00:00
2019-09-26 18:35:06 +00:00
2018-04-11 02:21:48 +00:00
2019-09-26 18:35:06 +00:00
2018-04-11 02:21:48 +00:00
2019-09-26 18:35:06 +00:00
2018-04-11 02:21:48 +00:00
2019-05-05 02:11:55 +00:00
2019-04-23 13:33:57 +00:00
2019-05-15 22:34:28 +00:00
2019-10-04 18:36:22 +00:00
2019-09-27 17:18:35 +00:00
2019-10-04 18:36:22 +00:00
2018-01-25 04:33:11 +00:00
2019-01-14 09:07:03 +00:00
2019-04-23 13:33:57 +00:00
2018-10-09 00:45:10 +00:00
2019-08-14 06:11:45 +00:00
2018-04-11 02:21:48 +00:00
2018-04-11 02:21:48 +00:00
2019-08-07 01:41:15 +00:00
2019-08-07 01:41:15 +00:00
2018-04-11 02:21:48 +00:00
2019-05-15 22:34:28 +00:00
2018-05-18 03:44:46 +00:00
2018-11-01 07:30:58 +00:00
2015-07-16 19:38:17 +00:00
2018-04-12 22:01:35 +00:00
2018-02-02 01:13:08 +00:00
2016-01-27 20:41:45 +00:00
2018-02-15 10:55:55 +00:00
2018-02-15 10:55:55 +00:00
2018-11-21 00:08:25 +00:00
2018-05-12 01:43:37 +00:00
2019-01-12 04:03:26 +00:00
2019-09-25 01:06:19 +00:00
2019-09-25 01:06:19 +00:00
2019-11-07 23:56:06 +00:00
2018-12-11 02:47:16 +00:00
2018-04-11 02:21:48 +00:00
2018-12-11 02:47:16 +00:00
2019-02-07 23:05:30 +00:00
2019-04-23 13:33:57 +00:00
2019-04-23 13:33:57 +00:00
2014-05-08 13:59:00 -07:00
2015-02-23 23:32:41 +00:00
2019-04-23 13:33:57 +00:00
2015-02-23 23:32:41 +00:00
2015-02-23 23:32:41 +00:00
2019-04-23 13:33:57 +00:00
2017-02-22 23:10:34 +00:00
2014-01-08 10:52:36 -08:00
2015-02-23 23:32:41 +00:00
2017-10-10 01:30:33 +00:00
2017-09-21 17:38:08 +00:00
2015-04-21 19:27:50 +00:00
2019-10-18 08:15:00 +00:00
2018-07-03 23:49:44 +00:00
2019-06-05 14:23:23 +00:00
2014-06-24 02:14:27 -07:00
2018-08-16 21:25:00 +00:00
2017-05-09 01:36:46 +00:00
2016-02-19 00:03:15 -08:00
2016-05-12 14:17:59 -07:00
2019-09-27 17:18:35 +00:00
2018-08-09 19:18:08 +00:00
2018-10-09 00:45:10 +00:00
2019-06-21 03:28:43 +00:00
2018-04-11 02:21:48 +00:00
2015-08-22 01:46:26 +00:00
2014-01-08 10:53:51 -08:00
2015-05-27 22:25:12 +00:00
2019-06-05 11:15:04 +00:00
2014-12-02 18:08:09 -08:00
2018-04-11 02:21:48 +00:00
2018-05-11 22:41:49 +00:00
2018-11-06 00:00:12 +00:00
2018-05-11 22:41:49 +00:00
2016-02-19 00:03:15 -08:00
2018-12-27 21:02:57 +00:00
2019-10-22 04:21:51 +00:00
2015-02-23 23:32:41 +00:00
2019-11-26 19:14:24 +00:00
2019-06-06 22:16:27 +00:00
2019-04-13 02:42:32 +00:00
2018-04-11 02:21:48 +00:00
2019-05-03 20:28:48 +00:00
2019-06-06 22:16:27 +00:00
2018-05-11 22:41:49 +00:00
2019-01-24 03:14:35 +00:00
2018-04-27 21:55:11 +00:00
2018-04-11 02:21:48 +00:00
2018-04-11 02:21:48 +00:00
2018-04-11 02:21:48 +00:00
2018-04-11 02:21:48 +00:00
2017-05-11 20:55:51 +00:00
2017-05-11 20:55:51 +00:00
2018-09-26 06:51:56 +00:00
2017-11-29 05:53:15 +00:00
2019-10-21 20:23:56 +00:00
2017-07-21 21:49:04 +00:00
2019-09-27 01:54:39 +00:00
2018-04-11 02:21:48 +00:00
2018-10-09 00:45:10 +00:00
2019-10-21 20:12:47 +00:00
2018-12-27 21:02:57 +00:00
2018-04-11 02:21:48 +00:00
2018-04-11 02:21:48 +00:00
2018-08-03 17:45:25 +00:00
2019-03-08 04:39:08 +00:00
2019-01-14 09:07:03 +00:00
2019-04-23 13:33:57 +00:00
2019-02-12 20:37:45 +00:00
2019-10-22 00:31:01 +00:00
2016-11-03 11:59:07 +00:00
2017-03-17 19:33:31 +00:00
2016-12-02 01:46:55 +00:00
2019-11-26 07:28:23 +00:00
2016-12-02 01:46:55 +00:00
2016-12-02 01:46:55 +00:00
2019-09-25 16:58:14 +00:00
2016-05-31 23:32:11 -07:00
2018-01-05 20:44:21 +00:00
2019-05-10 11:46:38 +00:00
2017-11-22 22:00:16 +00:00
2019-09-20 22:15:41 +00:00
2015-09-02 19:23:54 +00:00
2015-09-02 19:23:54 +00:00
2016-04-01 05:06:38 +00:00
2015-09-02 19:23:54 +00:00
2018-11-17 01:48:05 +00:00
2019-01-22 01:20:31 +00:00
2019-03-08 04:39:08 +00:00
2019-03-08 04:39:08 +00:00
2019-04-09 10:23:13 +00:00
2019-03-08 04:39:08 +00:00
2019-03-08 04:39:08 +00:00
2015-10-07 14:47:40 -07:00
2015-08-22 01:46:26 +00:00
2019-10-04 18:36:22 +00:00
2018-10-01 13:20:40 +00:00
2018-10-01 13:20:40 +00:00
2018-11-17 01:48:05 +00:00
2017-08-19 20:56:10 +00:00
2017-03-03 01:43:42 +00:00
2016-05-12 14:17:48 -07:00
2017-03-09 05:07:44 +00:00
2019-03-14 22:43:50 +00:00
2018-03-08 04:48:36 +00:00
2018-01-17 04:18:24 +00:00
2016-08-25 10:20:36 +00:00
2018-09-10 16:09:41 +00:00
2018-09-10 16:09:41 +00:00
2016-06-20 15:37:18 -07:00
2016-06-20 15:37:18 -07:00
2018-08-02 20:21:12 +00:00
2018-08-22 18:08:20 +00:00
2017-05-09 15:47:21 +00:00
2019-08-24 00:32:28 +00:00
2019-05-07 00:36:56 +00:00
2019-04-09 08:54:23 +00:00
2018-02-22 00:57:46 +00:00
2018-11-27 02:01:12 +00:00
2019-05-10 11:46:38 +00:00
2019-05-10 11:46:38 +00:00
2019-05-10 11:46:38 +00:00
2019-05-10 11:46:38 +00:00
2019-10-05 00:04:08 +00:00
2018-03-22 02:47:33 +00:00
2017-07-22 05:53:06 +00:00
2019-05-07 00:36:56 +00:00
2018-08-23 15:55:53 +00:00
2016-10-14 05:41:22 +00:00
2019-08-24 00:32:28 +00:00
2019-04-12 07:28:26 +00:00
2018-04-11 02:21:48 +00:00
2019-05-10 12:06:01 +00:00
2018-08-15 04:15:46 +00:00
2018-08-15 04:15:46 +00:00
2019-02-07 21:41:52 +00:00
2017-12-13 10:04:40 +00:00
2019-08-06 02:35:54 +00:00
2017-11-17 21:33:51 +00:00
2017-11-28 03:01:39 +00:00
2019-06-21 03:28:43 +00:00
2019-10-16 17:57:09 +00:00
2018-11-17 01:48:05 +00:00
2019-02-12 20:37:45 +00:00
2018-03-13 22:10:18 +00:00
2019-01-09 05:26:50 +00:00
2016-08-31 03:12:30 +00:00
2019-07-31 12:57:09 +00:00
2018-10-06 07:25:49 +00:00
2019-09-26 18:35:06 +00:00
2019-12-13 23:02:13 +00:00
2016-09-29 21:14:13 +00:00
2019-08-06 16:38:07 +00:00
2016-07-07 10:41:29 -07:00
2017-08-05 01:03:02 +00:00
2017-08-05 01:03:02 +00:00
2018-04-28 23:41:39 +00:00
2018-02-02 01:13:08 +00:00
2016-06-02 21:33:08 -07:00
2018-10-30 22:17:49 +00:00
2018-09-26 03:54:49 +00:00
2019-09-27 01:54:39 +00:00
2018-04-28 23:41:39 +00:00
2018-04-28 23:41:39 +00:00
2018-11-19 17:27:13 +00:00
2017-08-29 23:01:10 +00:00
2019-01-22 01:20:31 +00:00
2019-01-22 01:20:31 +00:00
2018-06-05 01:14:35 +00:00
2019-10-14 15:21:16 +00:00
2017-03-03 01:43:42 +00:00
2017-03-03 01:43:42 +00:00
2018-11-01 21:27:13 +00:00
2018-11-01 21:27:13 +00:00
2019-08-22 05:03:03 +00:00
2017-12-06 00:41:11 +00:00
2019-01-22 01:20:31 +00:00
2019-04-23 13:33:57 +00:00
2018-04-11 02:21:48 +00:00
2016-02-28 21:31:37 -08:00
2019-09-27 01:54:39 +00:00
2018-11-05 21:58:45 +00:00
2019-05-21 17:01:43 +00:00
2016-11-09 03:27:12 +00:00
2019-04-23 13:33:57 +00:00
2016-11-09 03:27:12 +00:00
2018-04-11 02:21:48 +00:00
2016-12-08 04:53:53 +00:00
2016-12-08 04:53:53 +00:00
2016-11-09 03:27:12 +00:00
2018-11-20 23:49:11 +00:00
2019-05-15 22:34:28 +00:00
2018-10-09 00:45:10 +00:00
2018-05-18 03:44:46 +00:00
2018-07-03 20:34:27 +00:00
2017-07-19 22:17:13 +00:00
2018-02-02 01:13:08 +00:00
2019-05-24 15:08:19 +00:00
2019-05-24 15:08:19 +00:00
2019-05-24 15:08:19 +00:00
2019-04-23 13:33:57 +00:00
2014-06-20 13:35:10 -07:00
2014-01-08 10:54:01 -08:00
2015-10-07 14:47:40 -07:00
6ebea33a9d