Commit Graph

179 Commits

Author SHA1 Message Date
ishaan
5138a720bb IMP-768: Enable the python test framework to check for insert results. 2014-01-08 10:48:22 -08:00
Skye Wanderman-Milne
6c08716439 IMPALA-92: Significant performance difference between LIKE = 'x' AND = 'x' 2014-01-08 10:48:21 -08:00
ishaan
f1d37646ed Fix creation of the insert_string_partitioned_table 2014-01-08 10:48:20 -08:00
Henry Robinson
222d15c6ca IMPALA-72: String partition keys should be URL encoded 2014-01-08 10:48:20 -08:00
Skye Wanderman-Milne
f4784947ce buildall.sh improvements: move 'set -e' higher, kill processes that are accessing postgres before creating metastore 2014-01-08 10:48:18 -08:00
Nong Li
27bc33d6b4 Fix snapshot cleanup. 2014-01-08 10:48:18 -08:00
ishaan
09d6d931f4 Change the way data is loaded 2014-01-08 10:48:09 -08:00
Lenni Kuff
99bb22dcac Add db name filter to compute stats, run compute stats on functional/text tables 2014-01-08 10:48:08 -08:00
Skye Wanderman-Milne
357327b5c0 Fix file offsets in DataErrorsTest 2014-01-08 10:48:06 -08:00
Alan Choi
73c8ee3d96 IMPALA-18 Ignore hidden file prefixed with . or _ 2014-01-08 10:48:00 -08:00
Lenni Kuff
d2e4776731 Support passing snapshot file to buildall, add script to run all tests, remove old tests 2014-01-08 10:47:59 -08:00
Nong Li
a0229cd12e Update tpch schema to use bigint for keys. 2014-01-08 10:47:54 -08:00
Nong Li
7001fb103e Move Impala to CDH4.2 RC2 2014-01-08 10:47:50 -08:00
Lenni Kuff
409d2ae5d7 Migrate run-test to python and add mini-stress test as part of buildall 2014-01-08 10:47:34 -08:00
Lenni Kuff
1896701399 IMPALA-44: Database names are case sensitive 2014-01-08 10:47:34 -08:00
Lenni Kuff
3fb375cdc4 Add initial set of queries for targeted perf workload
Includes a query that runs a simple "limit 0" as well as queries that perform aggregation
on columns with different numbers of GROUP BY groups.
2014-01-08 10:47:23 -08:00
Lenni Kuff
d5177c3c30 Update run-workload to support specifying table format test vectors from command line 2014-01-08 10:47:20 -08:00
Nong Li
02c329b97a Update RC files to use io mgr and remove scanner support for non-io mgr. 2014-01-08 10:47:11 -08:00
Lenni Kuff
9d981984e7 Update expected results of the 'show table/database' test to remove trevni tables 2014-01-08 10:47:10 -08:00
Lenni Kuff
deea7a86b9 Remove Trevni from mixed format table and fix data loading bug 2014-01-08 10:47:10 -08:00
Lenni Kuff
e10960b2c9 Disable test execution against Trevni and replace with seq/snap format 2014-01-08 10:47:10 -08:00
Lenni Kuff
12d18631e3 Test enhancements: dynamic table format data loading, per-workload exploration stategies 2014-01-08 10:47:07 -08:00
Lenni Kuff
c806738af2 Add scan range length tests to Python test framework 2014-01-08 10:47:06 -08:00
Nong Li
15dfd968fb Disable tpch-q21 and fix plan output for tpch-q22.
We can now generate the temp table for q22 which changes the plan output.
2014-01-08 10:47:03 -08:00
Nong Li
f46c654e01 Enable tpch-q21 and tpch-q22 in tests. 2014-01-08 10:47:03 -08:00
Lenni Kuff
1fcf094d67 Add support for comparing query test results by column type 2014-01-08 10:47:01 -08:00
Lenni Kuff
e20720f5d3 Disable loading Trevni for load/select statements that are not union compatible 2014-01-08 10:46:58 -08:00
Henry Robinson
15228f945f IMP-503: INSERTS into unpartitioned tables should be checked for union compatibility 2014-01-08 10:46:57 -08:00
Lenni Kuff
30dbf59ef2 Final changes to enable Python test infrastructure and tests
With this change the Python tests will now be called as part of buildall and
the corresponding Java tests have been disabled. The new tests can also be
invoked calling ./tests/run-tests.sh directly.

This includes a fix from Nong that caused wrong results for limit on non-io
manager formats.
2014-01-08 10:46:57 -08:00
Alan Choi
476a665763 IMP-620: print number of scanned partition and total scaned bytes 2014-01-08 10:46:57 -08:00
Nong Li
fbfef4e22e Fix crash in TopN node with null tuples. 2014-01-08 10:46:54 -08:00
Lenni Kuff
837f35eab3 Updated results for more query tests to reflect proper ordering + improved result updating 2014-01-08 10:46:53 -08:00
Lenni Kuff
bed633c1ae Extract config/metastore creation from buildall + script for loading warehouse snapshot 2014-01-08 10:46:53 -08:00
Lenni Kuff
a035cf4e73 Update results of a few TPC-H queries to reflect proper ordering
Change-Id: I41156b506155c846220cfb097f5e8120503f8da8
2014-01-08 10:46:52 -08:00
Lenni Kuff
1b248d067b Add TPC-DS dataset and workload 2014-01-08 10:46:52 -08:00
Marcel Kornacker
f6af9316d9 Fix for IMP-137: incorrect predicate placement for outer joins
Fixing predicate assignment for outer joins:
- On clause predicates for outer joins are now assigned to the join node
- the exception are On clause predicates that can be directly evaluated
  by the outer-joined tables themselves; those are "pushed down"
- Where clause predicates for outer-joined tables are assigned to the join node
  that materializes the outer join
2014-01-08 10:46:50 -08:00
Lenni Kuff
febdb112f4 Fixed bug in test file section parsing 2014-01-08 10:46:50 -08:00
Lenni Kuff
ef48f65e76 Add test framework for running Impala query tests via Python
This is the first set of changes required to start getting our functional test
infrastructure moved from JUnit to Python. After investigating a number of
option, I decided to go with a python test executor named py.test
(http://pytest.org/). It is very flexible, open source (MIT licensed), and will
enable us to do some cool things like parallel test execution.

As part of this change, we now use our "test vectors" for query test execution.
This will be very nice because it means if load the "core" dataset you know you
will be able to run the "core" query tests (specified by --exploration_strategy
when running the tests).

You will see that now each combination of table format + query exec options is
treated like an individual test case. this will make it much easier to debug
exactly where something failed.

These new tests can be run using the script at tests/run-tests.sh
2014-01-08 10:46:50 -08:00
Lenni Kuff
1e25c98fb4 Test data loading framework improvements
This change includes a number of improvements for the test data loading framework:
* Named sections for schema template definitions
* Removal of uneeded sections from schema template definitions (ex. ANALYZE TABLE)
* More granular data loading via table name filters
* Improved robustness in detecting failed data loads
* Table level constraints for specific file formats
* Re-written compute stats script
2014-01-08 10:46:49 -08:00
Henry Robinson
997df15b69 IMP-581: HBase table loading error / IMP-401: Re-enable tests for structured columns 2014-01-08 10:46:48 -08:00
Nong Li
b4dc3eeb35 Fix IMP-575 2014-01-08 10:46:45 -08:00
Nong Li
adf36b81f9 Fix data errors test. 2014-01-08 10:46:45 -08:00
Nong Li
34879a4ddc Fix IMP-297 2014-01-08 10:46:44 -08:00
Nong Li
b22b565a92 Fix codegen for min/max of bool col. 2014-01-08 10:46:43 -08:00
Alan Choi
a5a9ccf8c2 IMP-550 short-circuit queries with limit 0
Impala server would examine the plan. If the first fragment's top plan node has a "limit 0",
then the query is set to EOS immediately.
2014-01-08 10:46:41 -08:00
Alan Choi
dfe7690add IMP-522 Fix null pointer exception in HBase query
The ScanNode.keyRanges is an array list that can contain null. The existing HBase scan node
did not check for that.

A keyRanges would contain null if
1. the row-key is a string type and it is referenced in the query and,
2. there is no predicate on the row-key.
2014-01-08 10:46:36 -08:00
Michael Ubell
8a5297a526 Add HdfsLzoTextScanner 2014-01-08 10:46:35 -08:00
Marcel Kornacker
2fda5d9b99 IMP-491
Fixes bug in Planner.createHashJoinFragment(), which didn't set the left child of the
hj node to the output of the left child fragment.

Also: row descriptor was set incorrectly (too wide; included tuples that weren't materialized)
for roots of plan trees of non-root fragments if those fragments materialized an aggregate
2014-01-08 10:46:33 -08:00
Michael Ubell
0750384b41 IMP-497 Insert with limit, remove extra files from test. 2014-01-08 10:46:33 -08:00
Michael Ubell
116241f1d1 IMP-497 Insert with limit. 2014-01-08 10:46:33 -08:00