jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2026-01-16 18:00:39 -05:00
Code Issues Projects Releases Wiki Activity
Files
2e5d65819aaa52e1a89bc5cc212bba3b1b404339
impala/testdata/workloads/functional-query/queries/QueryTest/data-source-tables.test
Tim Armstrong d05f73f415 IMPALA-7647: Add HS2/Impyla dimension to TestQueries 
I used some ideas from Alex Leblang's abandoned patch:
https://gerrit.cloudera.org/#/c/137/ in order to run .test files through
HS2. The advantage of using Impyla is that much of the code will be
reusable for any Python client implementing the standard Python dbapi
and does not require us implementing yet another thrift client.

This gives us better coverage of non-trivial result sets from HS2,
including handling of NULLs, error logs and more interesting result
sets than the basic HS2 tests.

I added HS2 coverage to TestQueries, which has a reasonable variety of
queries and covers the data types in alltypes. I also added
TestDecimalQueries, TestStringQuery and TestCharFormats to get coverage
of DECIMAL, CHAR and VARCHAR that aren't in alltypes. Coverage of
results sets with NULLs was limited so I added a couple of queries.

Places where results differ from Beeswax:
* Impyla is a Python dbapi client so must convert timestamps into python datetime
  objects, which only have microsecond precision. Therefore result
  timestamps within nanosecond precision are truncated.
* The HS2 interface reports the NULL type as BOOLEAN as a workaround for
  IMPALA-914.
* The Beeswax interface reported VARCHAR as STRING, but HS2 reports
  VARCHAR.

I dealt with different results by adding additional result sections so
that the expected differences between the clients/protocols were
explicit.

Limitations:
* Not all of the same methods are implemented as for beeswax, so some
  tests that have more complicated interactions with the client will not
  work with HS2 yet.
* We don't have a way to get the affected row count for inserts.

I also simplified the ImpalaConnection API by removing some unnecessary
methods and moved some generic methods to the base class.

Testing:
* Confirmed that it detected IMPALA-7588 by re-applying the buggy patch.
* Ran exhaustive and CentOS6 tests.

Change-Id: I9908ccc4d3df50365be8043b883cacafca52661e
Reviewed-on: http://gerrit.cloudera.org:8080/11546
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
2018-10-09 00:45:10 +00:00

142 lines
5.1 KiB
Plaintext
Raw Blame History

====
---- QUERY
# Gets all types including a row with a NULL value. The predicate pushed to
# the data source is not actually used, but the second predicate is
# evaluated by Impala.
select *
from alltypes_datasource
where float_col != 0 and
int_col >= 1990 limit 5
---- RESULTS
1990,true,0,90,1990,19900,2189,1990,1970-01-01 00:00:01.990000000,'NULL',-999998009,-9999998009,-9999999999.9999998009,-9.9999999999999999999999999999999998009,-99999.98009
1991,false,1,91,1991,19910,2190.10009765625,1991,1970-01-01 00:00:01.991000000,'1991',999998008,9999998008,9999999999.9999998008,9.9999999999999999999999999999999998008,99999.98008
1992,true,2,92,1992,19920,2191.199951171875,1992,1970-01-01 00:00:01.992000000,'1992',-999998007,-9999998007,-9999999999.9999998007,-9.9999999999999999999999999999999998007,-99999.98007
1993,false,3,93,1993,19930,2192.300048828125,1993,1970-01-01 00:00:01.993000000,'1993',999998006,9999998006,9999999999.9999998006,9.9999999999999999999999999999999998006,99999.98006
1994,true,4,94,1994,19940,2193.39990234375,1994,1970-01-01 00:00:01.994000000,'1994',-999998005,-9999998005,-9999999999.9999998005,-9.9999999999999999999999999999999998005,-99999.98005
---- DBAPI_RESULTS
1990,true,0,90,1990,19900,2189,1990,1970-01-01 00:00:01.990000,'NULL',-999998009,-9999998009,-9999999999.9999998009,-9.9999999999999999999999999999999998009,-99999.98009
1991,false,1,91,1991,19910,2190.10009765625,1991,1970-01-01 00:00:01.991000,'1991',999998008,9999998008,9999999999.9999998008,9.9999999999999999999999999999999998008,99999.98008
1992,true,2,92,1992,19920,2191.199951171875,1992,1970-01-01 00:00:01.992000,'1992',-999998007,-9999998007,-9999999999.9999998007,-9.9999999999999999999999999999999998007,-99999.98007
1993,false,3,93,1993,19930,2192.300048828125,1993,1970-01-01 00:00:01.993000,'1993',999998006,9999998006,9999999999.9999998006,9.9999999999999999999999999999999998006,99999.98006
1994,true,4,94,1994,19940,2193.39990234375,1994,1970-01-01 00:00:01.994000,'1994',-999998005,-9999998005,-9999999999.9999998005,-9.9999999999999999999999999999999998005,-99999.98005
---- TYPES
INT, BOOLEAN, TINYINT, SMALLINT, INT, BIGINT, FLOAT, DOUBLE, TIMESTAMP, STRING, DECIMAL, DECIMAL, DECIMAL, DECIMAL, DECIMAL
====
---- QUERY
# Project a subset of the columns
select bigint_col, timestamp_col, double_col
from alltypes_datasource
where double_col != 0 and int_col >= 1990 limit 3
---- RESULTS
19900,1970-01-01 00:00:01.990000000,1990
19910,1970-01-01 00:00:01.991000000,1991
19920,1970-01-01 00:00:01.992000000,1992
---- DBAPI_RESULTS
19900,1970-01-01 00:00:01.990000,1990
19910,1970-01-01 00:00:01.991000,1991
19920,1970-01-01 00:00:01.992000,1992
---- TYPES
BIGINT, TIMESTAMP, DOUBLE
====
---- QUERY
# count(*) with a predicate evaluated by Impala
select count(*) from alltypes_datasource
where float_col = 0 and
string_col is not NULL
---- RESULTS
4000
---- TYPES
BIGINT
====
---- QUERY
# count(*) with no predicates has no materialized slots
select count(*) from alltypes_datasource
---- RESULTS
5000
---- TYPES
BIGINT
====
---- QUERY
select string_col from alltypes_datasource
where string_col = 'VALIDATE_PREDICATES##id LT 1 && id GT 1 && id LE 1 && id GE 1 && int_col EQ 1 && id NE 1'
and id < 1 and id > 1 and id <= 1 and id >= 1 and int_col = 1 and id != 1
---- RESULTS
'SUCCESS'
---- TYPES
STRING
====
---- QUERY
select string_col from alltypes_datasource
where string_col = 'VALIDATE_PREDICATES##id LT 1 && id GT 1 && id LE 1 && id GE 1 && int_col EQ 1 && id NE 1'
and 1 > id and 1 < id and 1 >= id and 1 <= id and 1 = int_col and 1 != id
---- RESULTS
'SUCCESS'
---- TYPES
STRING
====
---- QUERY
# Test that <=>, IS DISTINCT FROM, and IS NOT DISTINCT FROM all can be validated
# Note the duplicate predicate 1 IS NOT DISTINCT FROM id is removed.
select string_col from alltypes_datasource
where string_col = 'VALIDATE_PREDICATES##id NOT_DISTINCT 1 && id DISTINCT_FROM 1'
and 1 <=> id and 1 IS DISTINCT FROM id and 1 IS NOT DISTINCT FROM id
---- RESULTS
'SUCCESS'
---- TYPES
STRING
====
---- QUERY
# Test that <=>, IS DISTINCT FROM, and IS NOT DISTINCT FROM are evaluated just like their
# equality counterparts
select * from
(select count(*) from alltypes_datasource
where tinyint_col = 1 and smallint_col = 11) a
union all
(select count(*) from alltypes_datasource
where tinyint_col <=> 1 and smallint_col <=> 11)
---- RESULTS
50
50
---- TYPES
BIGINT
====
---- QUERY
select * from
(select count(*) from alltypes_datasource
where smallint_col = 11 and tinyint_col = 1) a
union all
(select count(*) from alltypes_datasource
where smallint_col <=> 11 and tinyint_col <=> 1)
---- RESULTS
500
500
---- TYPES
BIGINT
====
---- QUERY
select * from
(select count(*) from alltypes_datasource
where tinyint_col != 1 and smallint_col != 11) a
union all
(select count(*) from alltypes_datasource
where tinyint_col IS DISTINCT FROM 1 and smallint_col IS DISTINCT FROM 11)
---- RESULTS
4950
4950
---- TYPES
BIGINT
====
---- QUERY
select * from
(select count(*) from alltypes_datasource
where smallint_col != 11 and tinyint_col != 1) a
union all
(select count(*) from alltypes_datasource
where smallint_col IS DISTINCT FROM 11 and tinyint_col IS DISTINCT FROM 1)
---- RESULTS
4096
4096
---- TYPES
BIGINT
====
Reference in New Issue View Git Blame Copy Permalink