mirror of
https://github.com/apache/impala.git
synced 2026-02-03 09:00:39 -05:00
c12ac6c27b
Impala 4.1.0 has officially supported Iceberg tables, but we don't have
tools to generate TPC-H and TPC-DS test datasets for Iceberg tables. We
should use load-data.py to generate test datasets, e.g:
"${IMPALA_HOME}/bin/load-data.py -w tpcds -f -s 10
--table_formats=text/none,iceberg/none".
Testing:
- load-data.py -w tpch -s 1 --table_formats=text/none,parquet/none,
iceberg/none -e exhaustive -f
- load-data.py -w tpcds -s 1 --table_formats=text/none,parquet/zstd,
iceberg/zstd -e exhaustive -f
- single_node_perf_run.py --workloads=tpch --scale=30 --iterations=3 --
table_formats=parquet/lz4,iceberg/lz4 --num_impalads=1 --load 3ee3dc847
Workload: TPCH, Scale Factor: 30
Table Format: parquet/lz4/block
+----------+----------------+-----------+
| Query | Time Taken (s) | Client ID |
+----------+----------------+-----------+
| TPCH-Q1 | 7.73 | 1 |
| TPCH-Q1 | 7.50 | 1 |
| TPCH-Q1 | 7.55 | 1 |
+----------+----------------+-----------+
| TPCH-Q10 | 3.65 | 1 |
| TPCH-Q10 | 3.70 | 1 |
| TPCH-Q10 | 3.69 | 1 |
+----------+----------------+-----------+
| TPCH-Q11 | 0.68 | 1 |
| TPCH-Q11 | 0.68 | 1 |
| TPCH-Q11 | 0.67 | 1 |
+----------+----------------+-----------+
| TPCH-Q12 | 1.33 | 1 |
| TPCH-Q12 | 1.08 | 1 |
| TPCH-Q12 | 1.08 | 1 |
+----------+----------------+-----------+
| TPCH-Q13 | 10.63 | 1 |
| TPCH-Q13 | 10.36 | 1 |
| TPCH-Q13 | 10.52 | 1 |
+----------+----------------+-----------+
| TPCH-Q14 | 1.63 | 1 |
| TPCH-Q14 | 1.79 | 1 |
| TPCH-Q14 | 1.63 | 1 |
+----------+----------------+-----------+
| TPCH-Q15 | 1.44 | 1 |
| TPCH-Q15 | 1.54 | 1 |
| TPCH-Q15 | 1.45 | 1 |
+----------+----------------+-----------+
| TPCH-Q16 | 2.35 | 1 |
| TPCH-Q16 | 2.35 | 1 |
| TPCH-Q16 | 2.40 | 1 |
+----------+----------------+-----------+
| TPCH-Q17 | 1.20 | 1 |
| TPCH-Q17 | 1.13 | 1 |
| TPCH-Q17 | 1.10 | 1 |
+----------+----------------+-----------+
| TPCH-Q18 | 17.50 | 1 |
| TPCH-Q18 | 17.04 | 1 |
| TPCH-Q18 | 14.43 | 1 |
+----------+----------------+-----------+
| TPCH-Q19 | 1.61 | 1 |
| TPCH-Q19 | 1.56 | 1 |
| TPCH-Q19 | 1.56 | 1 |
+----------+----------------+-----------+
| TPCH-Q2 | 0.68 | 1 |
| TPCH-Q2 | 0.68 | 1 |
| TPCH-Q2 | 0.68 | 1 |
+----------+----------------+-----------+
| TPCH-Q20 | 1.09 | 1 |
| TPCH-Q20 | 1.09 | 1 |
| TPCH-Q20 | 1.15 | 1 |
+----------+----------------+-----------+
| TPCH-Q21 | 24.94 | 1 |
| TPCH-Q21 | 26.42 | 1 |
| TPCH-Q21 | 25.62 | 1 |
+----------+----------------+-----------+
| TPCH-Q22 | 3.69 | 1 |
| TPCH-Q22 | 3.58 | 1 |
| TPCH-Q22 | 3.24 | 1 |
+----------+----------------+-----------+
| TPCH-Q3 | 3.58 | 1 |
| TPCH-Q3 | 3.68 | 1 |
| TPCH-Q3 | 3.89 | 1 |
+----------+----------------+-----------+
| TPCH-Q4 | 1.37 | 1 |
| TPCH-Q4 | 1.37 | 1 |
| TPCH-Q4 | 1.32 | 1 |
+----------+----------------+-----------+
| TPCH-Q5 | 1.68 | 1 |
| TPCH-Q5 | 1.69 | 1 |
| TPCH-Q5 | 1.65 | 1 |
+----------+----------------+-----------+
| TPCH-Q6 | 0.46 | 1 |
| TPCH-Q6 | 0.47 | 1 |
| TPCH-Q6 | 0.47 | 1 |
+----------+----------------+-----------+
| TPCH-Q7 | 2.54 | 1 |
| TPCH-Q7 | 2.49 | 1 |
| TPCH-Q7 | 2.54 | 1 |
+----------+----------------+-----------+
| TPCH-Q8 | 1.90 | 1 |
| TPCH-Q8 | 1.90 | 1 |
| TPCH-Q8 | 1.90 | 1 |
+----------+----------------+-----------+
| TPCH-Q9 | 12.72 | 1 |
| TPCH-Q9 | 12.02 | 1 |
| TPCH-Q9 | 12.67 | 1 |
+----------+----------------+-----------+
Table Format: iceberg/lz4/block
+----------+----------------+-----------+
| Query | Time Taken (s) | Client ID |
+----------+----------------+-----------+
| TPCH-Q1 | 7.51 | 1 |
| TPCH-Q1 | 7.61 | 1 |
| TPCH-Q1 | 7.60 | 1 |
+----------+----------------+-----------+
| TPCH-Q10 | 3.79 | 1 |
| TPCH-Q10 | 3.73 | 1 |
| TPCH-Q10 | 3.67 | 1 |
+----------+----------------+-----------+
| TPCH-Q11 | 0.75 | 1 |
| TPCH-Q11 | 0.88 | 1 |
| TPCH-Q11 | 0.69 | 1 |
+----------+----------------+-----------+
| TPCH-Q12 | 1.18 | 1 |
| TPCH-Q12 | 1.13 | 1 |
| TPCH-Q12 | 1.04 | 1 |
+----------+----------------+-----------+
| TPCH-Q13 | 10.11 | 1 |
| TPCH-Q13 | 9.76 | 1 |
| TPCH-Q13 | 10.31 | 1 |
+----------+----------------+-----------+
| TPCH-Q14 | 1.64 | 1 |
| TPCH-Q14 | 1.64 | 1 |
| TPCH-Q14 | 1.53 | 1 |
+----------+----------------+-----------+
| TPCH-Q15 | 1.44 | 1 |
| TPCH-Q15 | 1.39 | 1 |
| TPCH-Q15 | 1.35 | 1 |
+----------+----------------+-----------+
| TPCH-Q16 | 2.24 | 1 |
| TPCH-Q16 | 2.15 | 1 |
| TPCH-Q16 | 2.15 | 1 |
+----------+----------------+-----------+
| TPCH-Q17 | 1.00 | 1 |
| TPCH-Q17 | 1.00 | 1 |
| TPCH-Q17 | 0.99 | 1 |
+----------+----------------+-----------+
| TPCH-Q18 | 15.14 | 1 |
| TPCH-Q18 | 14.94 | 1 |
| TPCH-Q18 | 15.04 | 1 |
+----------+----------------+-----------+
| TPCH-Q19 | 1.63 | 1 |
| TPCH-Q19 | 1.66 | 1 |
| TPCH-Q19 | 1.60 | 1 |
+----------+----------------+-----------+
| TPCH-Q2 | 0.69 | 1 |
| TPCH-Q2 | 0.75 | 1 |
| TPCH-Q2 | 0.69 | 1 |
+----------+----------------+-----------+
| TPCH-Q20 | 1.10 | 1 |
| TPCH-Q20 | 1.09 | 1 |
| TPCH-Q20 | 1.21 | 1 |
+----------+----------------+-----------+
| TPCH-Q21 | 25.38 | 1 |
| TPCH-Q21 | 25.48 | 1 |
| TPCH-Q21 | 25.18 | 1 |
+----------+----------------+-----------+
| TPCH-Q22 | 3.54 | 1 |
| TPCH-Q22 | 3.59 | 1 |
| TPCH-Q22 | 3.49 | 1 |
+----------+----------------+-----------+
| TPCH-Q3 | 3.95 | 1 |
| TPCH-Q3 | 3.96 | 1 |
| TPCH-Q3 | 3.79 | 1 |
+----------+----------------+-----------+
| TPCH-Q4 | 1.43 | 1 |
| TPCH-Q4 | 1.43 | 1 |
| TPCH-Q4 | 1.43 | 1 |
+----------+----------------+-----------+
| TPCH-Q5 | 1.84 | 1 |
| TPCH-Q5 | 1.80 | 1 |
| TPCH-Q5 | 1.71 | 1 |
+----------+----------------+-----------+
| TPCH-Q6 | 0.56 | 1 |
| TPCH-Q6 | 0.47 | 1 |
| TPCH-Q6 | 0.53 | 1 |
+----------+----------------+-----------+
| TPCH-Q7 | 2.75 | 1 |
| TPCH-Q7 | 2.59 | 1 |
| TPCH-Q7 | 2.49 | 1 |
+----------+----------------+-----------+
| TPCH-Q8 | 1.90 | 1 |
| TPCH-Q8 | 2.00 | 1 |
| TPCH-Q8 | 1.91 | 1 |
+----------+----------------+-----------+
| TPCH-Q9 | 13.34 | 1 |
| TPCH-Q9 | 12.18 | 1 |
| TPCH-Q9 | 12.71 | 1 |
+----------+----------------+-----------+
Change-Id: Ic8bbc5209e7e649f67a48144a2844b35d9f9c7f1
Reviewed-on: http://gerrit.cloudera.org:8080/19114
Reviewed-by: Zoltan Borok-Nagy <boroknagyz@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>