Adds support to cancel a query during Frontend planning or metadata
operations. Frontend planning is handled by createExecRequest, so
registers Java Threads executing createExecRequest by their query ID and
provides cancelExecRequest to interrupt the Thread for a particular
query ID.
Cancellation is implemented by setting a boolean for the thread, and
calling Thread.interrupt to trigger InterruptedException from any wait
calls. Several ignored wait calls are updated to check the boolean and
throw an exception if the query has been cancelled, interrupting those
operations.
Adds periodic checks to the planning process to interrupt planning.
They're primarily useful when planning is waiting on catalogd/HMS. If
planning gets into an algorithmically complex operation, it will not be
interrupted.
Removes check_inflight, as we can now cancel a query before it's
inflight. In the case that cancellation doesn't happen immediately -
because we're in a busy frontend loop that can't be interrupted -
/cancel will block until the frontend reaches an interruption point and
returns to the backend to finalize the query.
When analysis returns, cancellation is finalized in the backend. The
/cancel_query request returns once the query is cancelled. Cancelling
a request can no longer fail, so additional checks for whether the
request has been cancelled before it started executing are added.
Removes setting UpdateQueryStatus when GetExecRequest returns because
that's already handled in ImpalaServer::Execute when it calls
UnregisterQuery in response to an error, and constitutes an update race
on the status with UnregisterQuery triggered by CancelQueryHandler. We
want to use the status from CancelQueryHandler in this case as it
provides more context (about who initiated the cancel); the result of
GetExecRequest is just UserCancelledException. Avoids calling
UnregisterQuery in Execute if the query is already finalized to avoid
redundant "Invalid or unknown query handle" logs.
Extends idle_query_statuses_ to save status for any query interrupted by
an external process - cancelled by a user or timeout - so they can be
handled consistently.
Testing:
- updates test_query_cancel_created to cancel a CREATED query
- added tests to cancel a query while metadata loading is delayed
- removes test_query_cancel_exception, as it no longer demonstrates
relevant behavior; cancelling a query that will encounter an exception
before the exception occurs is no different than other queries
- ran query_test/test_cancellation.py in exhaustive mode
- ran query_test/test_cancellation.py w/ DEFAULT_TEST_PROTOCOL=beeswax
- updates cancellation tests that expect INVALID_QUERY_HANDLE to accept
Cancelled, which is sometimes returned by interrupted query status.
Change-Id: I0d25d4c7fb0b8dcc7dad9510db1e8dca220eeb86
Reviewed-on: http://gerrit.cloudera.org:8080/21803
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
To support killing queries programatically, this patch adds a new
type of SQL statements, called the KILL QUERY statement, to cancel and
unregister a query on any coordinator in the cluster.
A KILL QUERY statement looks like
```
KILL QUERY '123:456';
```
where `123:456` is the query id of the query we want to kill. We follow
syntax from HIVE-17483. For backward compatibility, 'KILL' and 'QUERY'
are added as "unreserved keywords", like 'DEFAULT'. This allows the
three keywords to be used as identifiers.
A user is authorized to kill a query only if the user is an admin or is
the owner of the query. KILL QUERY statements are not affected by
admission control.
Implementation:
Since we don't know in advance which impalad is the coordinator of the
query we want to kill, we need to broadcast the kill request to all the
coordinators in the cluster. Upon receiving a kill request, each
coordinator checks whether it is the coordinator of the query:
- If yes, it cancels and unregisters the query,
- If no, it reports "Invalid or unknown query handle".
Currently, a KILL QUERY statement is not interruptible. IMPALA-13663 is
created for this.
For authorization, this patch adds a custom handler of
AuthorizationException for each statement to allow the exception to be
handled by the backend. This is because we don't know whether the user
is the owner of the query until we reach its coordinator.
To support cancelling child queries, this patch changes
ChildQuery::Cancel() to bypass the HS2 layer so that the session of the
child query will not be added to the connection used to execute the
KILL QUERY statement.
Testing:
- A new ParserTest case is added to test using "unreserved keywords" as
identifiers.
- New E2E test cases are added for the KILL QUERY statement.
- Added a new dimension in TestCancellation to use the KILL QUERY
statement.
- Added file tests/common/cluster_config.py and made
CustomClusterTestSuite.with_args() composable so that common cluster
configs can be reused in custom cluster tests.
Change-Id: If12d6e47b256b034ec444f17c7890aa3b40481c0
Reviewed-on: http://gerrit.cloudera.org:8080/21930
Reviewed-by: Riza Suminto <riza.suminto@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Reviewed-by: Michael Smith <michael.smith@cloudera.com>
