jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2026-02-02 15:00:38 -05:00
Code Issues Projects Releases Wiki Activity
Files
a570ee866e78ebffd698cb05edd61cd706fb2ef9
impala/testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test
wzhou-code 40da36414f IMPALA-11809: Support non unique primary key for Kudu 
Kudu engine recently enables the auto-incrementing column feature
(KUDU-1945). The feature works by appending a system generated
auto-incrementing column to the primary key columns to guarantee the
uniqueness on primary key when the primary key columns can be non
unique. The non unique primary key columns and the auto-incrementing
column form the effective unique composite primary key.

This auto-incrementing column is named as 'auto_incrementing_id' with
big int type. The assignment to it during insertion is automatic so
insertion statements should not specify values for auto-incrementing
column. In current Kudu implementation, there is no central key provider
for auto-incrementing columns. It uses a per tablet-server global
counter to assign values for auto-incrementing columns. So the values
of auto-incrementing columns are not unique in a Kudu table, but unique
within a continuous region of the table served by a tablet-server.

This patch also upgraded Kudu version to 345fd44ca3 to pick up Kudu
changes needed for supporting non-unique primary key. It added
syntactic support for creating Kudu table with non unique primary key.
When creating a Kudu table, specifying PRIMARY KEY is optional.
If there is no primary key attribute specified, the partition key
columns will be promoted as non unique primary key if those columns
are the beginning columns of the table.
New column "key_unique" is added to the output of 'describe' table
command for Kudu table.

Examples of CREATE TABLE statement with non unique primary key:
  CREATE TABLE tbl (i INT NON UNIQUE PRIMARY KEY, s STRING)
  PARTITION BY HASH (i) PARTITIONS 3
  STORED as KUDU;

  CREATE TABLE tbl (i INT, s STRING, NON UNIQUE PRIMARY KEY(i))
  PARTITION BY HASH (i) PARTITIONS 3
  STORED as KUDU;

  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY HASH (id) PARTITIONS 3
  STORED as KUDU
  AS SELECT id, string_col FROM functional.alltypes WHERE id = 10;

  CREATE TABLE tbl NON UNIQUE PRIMARY KEY(id)
  PARTITION BY RANGE (id)
  (PARTITION VALUES <= 1000,
   PARTITION 1000 < VALUES <= 2000,
   PARTITION 2000 < VALUES <= 3000,
   PARTITION 3000 < VALUES)
  STORED as KUDU
  AS SELECT id, int_col FROM functional.alltypestiny ORDER BY id ASC
   LIMIT 4000;

  CREATE TABLE tbl (id INT, name STRING, NON UNIQUE PRIMARY KEY(id))
  STORED as KUDU;

  CREATE TABLE tbl (a INT, b STRING, c FLOAT)
  PARTITION BY HASH (a, b) PARTITIONS 3
  STORED as KUDU;

SELECT statement does not show the system generated auto-incrementing
column unless the column is explicitly specified in the select list.
Auto-incrementing column cannot be added, removed or renamed with
ALTER TABLE statements.
UPSERT operation is not supported now for Kudu tables with auto
incrementing column due to limitation in Kudu engine.

Testing:
 - Ran manual test in impala-shell with queries to create Kudu tables
   with non unique primary key, and tested insert/update/delete
   operations for these tables with non unique primary key.
 - Added front end tests, and end to end unit tests for Kudu tables
   with non unique primary key.
 - Passed exhaustive test.

Change-Id: I4d7882bf3d01a3492cc9827c072d1f3200d9eebd
Reviewed-on: http://gerrit.cloudera.org:8080/19383
Reviewed-by: Riza Suminto <riza.suminto@cloudera.com>
Reviewed-by: Wenzhe Zhou <wzhou@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
2023-02-04 07:34:56 +00:00

105 lines
3.2 KiB
Plaintext
Raw Blame History

====
---- QUERY
create table simple (id int primary key, name string, valf float, vali bigint)
partition by range (partition values < 10, partition 10 <= values < 30,
partition 30 <= values) stored as kudu tblproperties('kudu.num_tablet_replicas' = '1')
---- RESULTS
'Table has been created.'
====
---- QUERY
# Tests the SHOW TABLE STATS output without stats computed
show table stats simple;
---- RESULTS
-1,3,regex:.*,'KUDU',regex:.*
---- TYPES
BIGINT,BIGINT,STRING,STRING,STRING
---- LABELS
#Rows,#Partitions,Size,Format,Location
====
---- QUERY
# Tests the SHOW TABLE STATS output after stats computed
compute stats simple;
show table stats simple;
---- RESULTS
0,3,regex:.*,'KUDU',regex:.*
---- TYPES
BIGINT,BIGINT,STRING,STRING,STRING
---- LABELS
#Rows,#Partitions,Size,Format,Location
====
---- QUERY
# Tests the SHOW PARTITIONS output
show partitions simple;
---- RESULTS
'','8000000A',regex:.*?:\d+,1
'8000000A','8000001E',regex:.*?:\d+,1
'8000001E','',regex:.*?:\d+,1
---- TYPES
STRING,STRING,STRING,INT
---- LABELS
Start Key,Stop Key,Leader Replica,#Replicas
====
---- QUERY
# IMPALA-3373: Computing stats on a Kudu table lead to duplicate columns shown for the
# table.
compute stats simple;
describe simple;
---- RESULTS
'id','int','','true','true','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'name','string','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'valf','float','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'vali','bigint','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
---- QUERY
# Create Kudu table with non unique primary key
create table non_unique_key_stats_test (a int, b string, non unique primary key(a))
partition by range (partition values < 10, partition 10 <= values < 30,
partition 30 <= values) stored as kudu tblproperties('kudu.num_tablet_replicas' = '1');
---- RESULTS
'Table has been created.'
====
---- QUERY
# Tests the SHOW TABLE STATS output without stats computed
show table stats non_unique_key_stats_test;
---- RESULTS
-1,3,regex:.*,'KUDU',regex:.*
---- TYPES
BIGINT,BIGINT,STRING,STRING,STRING
---- LABELS
#Rows,#Partitions,Size,Format,Location
====
---- QUERY
# Tests the SHOW TABLE STATS output after stats computed
compute stats non_unique_key_stats_test;
show table stats non_unique_key_stats_test;
---- RESULTS
0,3,regex:.*,'KUDU',regex:.*
---- TYPES
BIGINT,BIGINT,STRING,STRING,STRING
---- LABELS
#Rows,#Partitions,Size,Format,Location
====
---- QUERY
# Tests the SHOW PARTITIONS output
show partitions non_unique_key_stats_test;
---- RESULTS
'','8000000A',regex:.*?:\d+,1
'8000000A','8000001E',regex:.*?:\d+,1
'8000001E','',regex:.*?:\d+,1
---- TYPES
STRING,STRING,STRING,INT
---- LABELS
Start Key,Stop Key,Leader Replica,#Replicas
====
---- QUERY
compute stats non_unique_key_stats_test;
describe non_unique_key_stats_test;
---- RESULTS
'a','int','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'auto_incrementing_id','bigint','','true','false','false','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
'b','string','','false','','true','','AUTO_ENCODING','DEFAULT_COMPRESSION','0'
---- TYPES
STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING,STRING
====
Reference in New Issue View Git Blame Copy Permalink