mirror of
https://github.com/apache/impala.git
synced 2026-01-24 06:00:49 -05:00
975883c470
This patch addresses Impala client hang due to AWS network load balancer
timeout which is fixed at 350s. When some long DDL operations are
executing and the timeout happens, AWS silently drops the connection and
the Impala client enters the hang state.
The fix maintains the current TCLIService protocol between the client
and Impala server and is applicable to the following Impala clients
which issue thrift RPC ExecuteStatement() followed by repeated call to
GetOperationStatus() (HS2, Impyla and HUE) or a variant of it (Beeswax)
to Impala backend.
1. HS2
2. Beeswax
3. Impyla
4. HUE
In the fix, the backend method ClientRequestState::ExecDdlRequest()
can start a new thread in 'async_exec_thread_' for ExecDdlRequestImpl()
which executes most of the DDLs asynchronously. This thread is waited
for in the wait thread 'wait_thread_'. Since the wait thread also runs
asynchronously, the execution of the DDLs will not cause a wait on the
Impala client. Thus the Impala client can keep checking its execution
status via GetOperationStatus() without long waiting, say more than
350s.
As an optimization, the above asynchronous mode is not applied to the
execution of certain DDLs that run very low risks of long execution.
1. Operations that do not access catalog service;
2. COMPUTE STATS as the stats computation queries already run
asynchronously.
External behavior change:
1. A new field with name "DDL execution mode:" is added to the
summary section in the runtime profile, next to "DDL Type". This
field takes either 'asynchronous' or 'synchronous' as value.
2. A new query option 'enable_async_ddl_execution', default to true,
is added. It can be set to false to turn off the patch.
Limitations:
This patch does not handle potential AWS NLB-type time out for LOAD
DATA (IMPALA-10967).
Testing:
1. Added new async. DDL unit tests with HS2, HS2-HTTP, Beeswax and
JDBC clients.
2. Ran core tests successfully.
Change-Id: Ib57e86926a233ef13d27a9ec8d9c36d33a88a44e
Reviewed-on: http://gerrit.cloudera.org:8080/17872
Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
17 lines
628 B
Plaintext
17 lines
628 B
Plaintext
====
|
|
---- QUERY
|
|
# Clone a table and then recover partitions for it with a delay of 15s
|
|
# for the asynchronous execution of the DDL. Expect the client to issue
|
|
# multiple GetOperationStatus() call during the delay and to receive the
|
|
# successful completaion status at the end.
|
|
drop table if exists alltypes_clone;
|
|
create external table alltypes_clone like functional_parquet.alltypes
|
|
location '$FILESYSTEM_PREFIX/test-warehouse/alltypes_parquet';
|
|
set debug_action="CRS_DELAY_BEFORE_CATALOG_OP_EXEC:SLEEP@15000";
|
|
alter table alltypes_clone recover partitions;
|
|
---- RESULTS
|
|
'Partitions have been recovered.'
|
|
---- TYPES
|
|
STRING
|
|
====
|