IMPALA-13657: Connect Calcite planner to Impala Frontend framework

This commit adds the plumbing created by IMPALA-13653. The Calcite
planner is now called from Impala's Frontend code via 4 hooks which
are:

- CalciteCompilerFactory: the factory class that creates
    the implementations of the parser, analysis, and single node
    planner hooks.
- CalciteParsedStatement: The class which holds the Calcite SqlNode
    AST.
- CalciteAnalysisDriver: The class that does the validation of the
    SqlNode AST
- CalciteSingleNodePlanner: The class that converts the AST to a
    logical plan, optimizes it, and converts it into an Impala
    PlanNode physical plan.

To run on Calcite, one needs to do two things:

1) set the USE_CALCITE_PLANNER env variable to true before starting
the cluster. This adds the jar file into the path in the
bin/setclasspath.sh file, which is not there by default at the time
of this commit.
2) set the use_calcite_planner query option to true.

This commit makes the CalciteJniFrontend class obsolete. Once the
test cases are moved out of there, that class and others can be
removed.

Change-Id: I3b30571beb797ede827ef4d794b8daefb130ccb1
Reviewed-on: http://gerrit.cloudera.org:8080/22319
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Reviewed-by: Michael Smith <michael.smith@cloudera.com>
Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com>
This commit is contained in:
Steve Carlin
2025-01-08 06:56:04 -08:00
parent a877cde76d
commit 706e1f026c
17 changed files with 675 additions and 53 deletions

View File

@@ -1022,6 +1022,9 @@ enum TImpalaQueryOptions {
// to either 1.0 (enabled) or 0.0 (disabled) and only set value in between for
// experimental purpose.
MEM_ESTIMATE_SCALE_FOR_SPILLING_OPERATOR = 190
// If True, use the Calcite planner for compilation
USE_CALCITE_PLANNER = 191
}
// The summary of a DML statement.

View File

@@ -774,6 +774,9 @@ struct TQueryOptions {
// See comment in ImpalaService.thrift
191: optional double mem_estimate_scale_for_spilling_operator = 0.0
// See comment in ImpalaService.thrift
192: optional bool use_calcite_planner = false;
}
// Impala currently has three types of sessions: Beeswax, HiveServer2 and external