jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-25 02:03:09 -05:00
Code Issues Projects Releases Wiki Activity
Files
ddd82e02b93abe5fba0a971cd993543e7c3ddea1
impala/testdata/workloads/functional-query/queries/QueryTest/kudu_describe.test
wzhou-code 40da36414f IMPALA-11809: Support non unique primary key for Kudu 
Kudu engine recently enables the auto-incrementing column feature
(KUDU-1945). The feature works by appending a system generated
auto-incrementing column to the primary key columns to guarantee the
uniqueness on primary key when the primary key columns can be non
unique. The non unique primary key columns and the auto-incrementing
column form the effective unique composite primary key.

This auto-incrementing column is named as 'auto_incrementing_id' with
big int type. The assignment to it during insertion is automatic so
insertion statements should not specify values for auto-incrementing
column. In current Kudu implementation, there is no central key provider
for auto-incrementing columns. It uses a per tablet-server global
counter to assign values for auto-incrementing columns. So the values
of auto-incrementing columns are not unique in a Kudu table, but unique
within a continuous region of the table served by a tablet-server.

This patch also upgraded Kudu version to 345fd44ca3 to pick up Kudu
changes needed for supporting non-unique primary key. It added
syntactic support for creating Kudu table with non unique primary key.
When creating a Kudu table, specifying PRIMARY KEY is optional.
If there is no primary key attribute specified, the partition key
columns will be promoted as non unique primary key if those columns
are the beginning columns of the table.
New column "key_unique" is added to the output of 'describe' table
command for Kudu table.

Examples of CREATE TABLE statement with non unique primary key:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) PARTITIONS 3
  STORED as KUDU;

  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) PARTITIONS 3
  STORED as KUDU;

  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) PARTITIONS 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY RANGE (id)
  (PARTITION VALUES <= 1000,
   PARTITION 1000 < VALUES <= 2000,
   PARTITION 2000 < VALUES <= 3000,
   PARTITION 3000 < VALUES)
  STORED as KUDU
  AS SELECT id, int_col FROM functional.alltypestiny ORDER BY id ASC
   LIMIT 4000;

  CREATE TABLE tbl (id INT, name STRING, NON UNIQUE PRIMARY KEY(id))
  STORED as KUDU;

  CREATE TABLE tbl (a INT, b STRING, c FLOAT)
  PARTITION BY HASH (a, b) PARTITIONS 3
  STORED as KUDU;

SELECT statement does not show the system generated auto-incrementing
column unless the column is explicitly specified in the select list.
Auto-incrementing column cannot be added, removed or renamed with
ALTER TABLE statements.
UPSERT operation is not supported now for Kudu tables with auto
incrementing column due to limitation in Kudu engine.

Testing:
 - Ran manual test in impala-shell with queries to create Kudu tables
   with non unique primary key, and tested insert/update/delete
   operations for these tables with non unique primary key.
 - Added front end tests, and end to end unit tests for Kudu tables
   with non unique primary key.
 - Passed exhaustive test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Reviewed-on: http://gerrit.cloudera.org:8080/19383
Reviewed-by: Riza Suminto <riza.suminto@cloudera.com>
Reviewed-by: Wenzhe Zhou <wzhou@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
2023-02-04 07:34:56 +00:00

215 lines
10 KiB
Plaintext
Raw Blame History

====
---- QUERY
describe functional_kudu.alltypes
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'bigint_col','bigint','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'bool_col','boolean','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'date_string_col','string','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'double_col','double','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'float_col','float','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'id','int','','true','true','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'int_col','int','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'month','int','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'smallint_col','smallint','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'string_col','string','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'timestamp_col','timestamp','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'tinyint_col','tinyint','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'year','int','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# Test composite primary key and column options.
create table describe_test
(pk1 int,
pk2 int,
pk3 string,
c1 string null default 'abc' comment 'testing',
c2 int not null default 100 encoding plain_encoding compression snappy,
c3 int null block_size 8388608,
primary key (pk1, pk2, pk3))
partition by hash (pk1) partitions 3
stored as kudu;
describe describe_test;
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'pk1','int','','true','true','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'pk2','int','','true','true','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'pk3','string','','true','true','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'c1','string','testing','false','','true','abc','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'c2','int','','false','','false','100','PLAIN_ENCODING','SNAPPY','0'
'c3','int','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','8388608'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# Test decimal columns and primary key
create table describe_decimal_test
(
decimal_default decimal PRIMARY KEY,
decimal_4 decimal(9, 9) not null,
decimal_8 decimal(18, 2) not null default 100.00,
decimal_16 decimal(38, 0) null)
stored as kudu;
describe describe_decimal_test;
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'decimal_default','decimal(9,0)','','true','true','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'decimal_4','decimal(9,9)','','false','','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'decimal_8','decimal(18,2)','','false','','false','100.00','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'decimal_16','decimal(38,0)','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# IMPALA-7781: Test unescaped default column values
CREATE TABLE IF NOT EXISTS unescaped_str_defaults (
id int,
s1 string default "\"",
s2 string default '\'',
s3 string default "\\\"",
s4 string default '\\\'',
primary key(id)
) STORED AS KUDU;
DESCRIBE unescaped_str_defaults;
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'id','int','','true','true','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
's1','string','','false','','true','"','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
's2','string','','false','','true','''','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
's3','string','','false','','true','\\"','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
's4','string','','false','','true','\\''','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# Test date columns and primary key
create table describe_date_test
(
date_pk date PRIMARY KEY,
date_val date not null,
date_null date null)
stored as kudu;
describe describe_date_test;
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'date_pk','date','','true','true','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'date_val','date','','false','','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'date_null','date','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# Test varchar columns and primary key
create table describe_varchar_test
(
varchar_pk varchar(1000) PRIMARY KEY,
varchar_val varchar(500) not null,
varchar_default varchar(200) not null default cast('foo' as varchar(200)),
varchar_null varchar(100) null)
stored as kudu;
describe describe_varchar_test;
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'varchar_pk','varchar(1000)','','true','true','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'varchar_val','varchar(500)','','false','','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'varchar_default','varchar(200)','','false','','false','foo','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'varchar_null','varchar(100)','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# Create Kudu table with non unique primary key.
# Verify that auto_incrementing_id column is added by Kudu.
create table describe_non_unique_key_test (key int non unique primary key, name string)
partition by hash (key) partitions 3
stored as kudu;
describe describe_non_unique_key_test;
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'key','int','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'auto_incrementing_id','bigint','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'name','string','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# Create Kudu table with non unique composite primary keys.
# Verify that auto_incrementing_id column is added by Kudu.
create table describe_non_unique_composite_key_test
(a int, b string, c float, non unique primary key(a, b))
partition by hash (a) partitions 3
stored as kudu;
describe describe_non_unique_composite_key_test;
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'a','int','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'b','string','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'auto_incrementing_id','bigint','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'c','float','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# Create Kudu table in CTAS statement with non unique primary key.
# Verify that auto_incrementing_id column is added by Kudu.
create table describe_ctas_non_unique_key_test non unique primary key (id, int_col)
partition by hash (id) partitions 3 stored as kudu
as select id, int_col, float_col, string_col from functional.alltypestiny;
describe describe_ctas_non_unique_key_test;
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'id','int','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'int_col','int','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'auto_incrementing_id','bigint','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'float_col','float','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'string_col','string','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# Create Kudu table without primary key columns.
# Verify that partition columns 'a' and 'b' are promoted as non unique key columns and
# auto_incrementing_id column is added by Kudu.
create table describe_promote_partition_keys_as_non_unique_key_test
(a int, b string, c float)
partition by hash (a, b) partitions 3
stored as kudu;
describe describe_promote_partition_keys_as_non_unique_key_test;
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'a','int','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'b','string','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'auto_incrementing_id','bigint','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'c','float','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# Create Kudu table with non unique composite primary keys, but without partitions.
# Verify that auto_incrementing_id column is added by Kudu.
create table describe_non_unique_key_no_partitions_test
(a int, b string, c float, non unique primary key(a, b))
stored as kudu;
describe describe_non_unique_key_no_partitions_test;
---- LABELS
NAME,TYPE,COMMENT,PRIMARY_KEY,KEY_UNIQUE,NULLABLE,DEFAULT_VALUE,ENCODING,COMPRESSION,BLOCK_SIZE
---- RESULTS
'a','int','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'b','string','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'auto_incrementing_id','bigint','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'c','float','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
Reference in New Issue View Git Blame Copy Permalink