mirror of
https://github.com/apache/impala.git
synced 2026-01-23 03:00:12 -05:00
96143cfcf1
Add a temporary --mt_dop_auto_fallback to allow a graceful transition to using mt_dop for workloads. When this flag is set, DML queries and joins that would otherwise fail with an error when run with mt_dop > 0 fall back to running with mt_dop = 0. This means that a user can set mt_dop for their queries and it will only take effect when supported. The behaviour generally does not change when this flag is not set, with a couple of exceptions: * I made mt_dop automatic for compute stats on all file formats * mt_dop is allowed for single node plans with inserts. The quirky validatePlan() logic previously disallowed this but allowed joins in single node plans. The checks added by this patch can be removed safely once mt_dop is supported by default for all queries. This includes some cleanup: * isDmlStmt() was stale and incorrectly implemented. * Various TreeNode methods did not return instances of subclasses of the requested class, which was strange. This fix is required to make 'contains(JoinNode.class)' work correctly. I checked the callsites of the fixed functions and none of them would be affected by this change because they specified a terminal class without any subclasses. I didn't actually use this fix in the end (I had to write a custom tree traversal in hasUnsupportedMtDopJoin()), but figured I would leave the improvement in here. Testing: Add some basic functional tests ensuring that the fallback takes effect. Run basic join and insert tests with this flag enabled. Change-Id: Ie0d73d8744059874293697c8e104891a10dba04d Reviewed-on: http://gerrit.cloudera.org:8080/14344 Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
34 lines
1.1 KiB
Plaintext
34 lines
1.1 KiB
Plaintext
====
|
|
---- QUERY
|
|
select min(l_orderkey), min(p_name)
|
|
from tpch.lineitem join tpch.part on l_partkey = p_partkey;
|
|
---- RESULTS
|
|
1,'almond antique blue royal burnished'
|
|
---- TYPES
|
|
BIGINT,STRING
|
|
---- RUNTIME_PROFILE
|
|
row_regex:.*Query Options \(set by configuration and planner\):.*MT_DOP=0.*
|
|
row_regex:.*All 3 execution backends \(5 fragment instances\) started.*
|
|
row_regex:.*NumScannerThreadsStarted.*
|
|
====
|
|
---- QUERY
|
|
select min(l_orderkey), min(p_name)
|
|
from tpch_kudu.lineitem join tpch_kudu.part on l_partkey = p_partkey;
|
|
---- RESULTS
|
|
1,'almond antique blue royal burnished'
|
|
---- TYPES
|
|
BIGINT,STRING
|
|
---- RUNTIME_PROFILE
|
|
row_regex:.*Query Options \(set by configuration and planner\):.*MT_DOP=0.*
|
|
row_regex:.*All 3 execution backends \(7 fragment instances\) started.*
|
|
row_regex:.*NumScannerThreadsStarted.*
|
|
====
|
|
---- QUERY
|
|
create table tmp as
|
|
select * from functional.alltypes
|
|
---- RUNTIME_PROFILE
|
|
row_regex:.*Query Options \(set by configuration and planner\):.*MT_DOP=0.*
|
|
row_regex:.*All 3 execution backends \(3 fragment instances\) started.*
|
|
row_regex:.*NumScannerThreadsStarted.*
|
|
====
|