Update benchmark tests to run against generic workload, data loading with scale factor, +more

This change updates the run-benchmark script to enable it to target one or more
workloads. Now benchmarks can be run like:

./run-benchmark --workloads=hive-benchmark,tpch

We lookup the workload in the workloads directory, then read the associated
query .test files and start executing them.

To ensure the queries are not duplicated between benchmark and query tests, I
moved all existing queries (under fe/src/test/resources/* to the workloads
directory. You do NOT need to look through all the .test files, I've just moved
them. The one new file is the 'hive-benchmark.test' which contains the hive
benchmark queries.

Also added support for generating schema for different scale factors as well as
executing against these scale factors. For example, let's say we have a dataset
with a scale factor called "SF1". We would first generate the schema using:

./generate_schema_statements --workload=<workload> --scale_factor="SF3"
This will create tables with a unique names from the other scale factors.

Run the generated .sql file to load the data. Alternatively, the data can loaded
by running a new python script:
./bin/load-data.py -w <workload1>,<workload2> -e <exploration strategy> -s [scale factor]
For example: load-data.sh -w tpch -e core -s SF3

Then run against this:
./run-benchmark --workloads=<workload> --scale_factor=SF3

This changeset also includes a few other minor tweaks to some of the test
scripts.

Change-Id: Ife8a8d91567d75c9612be37bec96c1e7780f50d6
This commit is contained in:
Lenni Kuff
2012-08-02 23:38:59 -07:00
committed by Henry Robinson
parent 81d54e85e5
commit 04edc8f534
90 changed files with 1707 additions and 1566 deletions

View File

@@ -0,0 +1,18 @@
# testtbl is empty
select * from testtbl$TABLE
---- TYPES
bigint, string, int
---- RESULTS
====
# month ends at 12
select int_col from alltypessmall$TABLE where month > 100
---- TYPES
int
---- RESULTS
====
# Empty partitioned table test
select field from emptytable
---- TYPES
string
---- RESULTS
====