IMPALA-13115: Add query id to error messages

This patch adds the query id to the error messages in both

- the result of the `get_log()` RPC, and
- the error message in an RPC response

before they are returned to the client, so that the users can easily
figure out the errored queries on the client side.

To achieve this, the query id of the thread debug info is set in the
RPC handler method, and is retrieved from the thread debug info each
time the error reporting function or `get_log()` gets called.

Due to the change of the error message format, some checks in the
impala-shell.py are adapted to keep them valid.

Testing:
- Added helper function `error_msg_expected()` to check whether an
  error message is expected. It is stricter than only using the `in`
  operator.
- Added helper function `error_msg_equal()` to check if two error
  messages are equal regardless of the query ids.
- Various test cases are adapted to match the new error message format.
- `ImpalaBeeswaxException`, which is used in tests only, is simplified
  so that it has the same error message format as the exceptions for
  HS2.
- Added an assertion to the case of killing and restarting a worker
  in the custom cluster test to ensure that the query id is in
  the error message in the client log retrieved with `get_log()`.

Change-Id: I67e659681e36162cad1d9684189106f8eedbf092
Reviewed-on: http://gerrit.cloudera.org:8080/21587
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Xuebin Su
2024-07-16 15:52:41 +08:00
committed by Impala Public Jenkins
parent b1941c8f17
commit ad868b9947
26 changed files with 224 additions and 62 deletions

View File

@@ -39,10 +39,12 @@ from tests.common.skip import SkipIf
from tests.common.test_dimensions import (
create_client_protocol_dimension, create_client_protocol_strict_dimension,
create_uncompressed_text_dimension, create_single_exec_option_dimension)
from tests.common.test_result_verifier import error_msg_expected
from time import sleep, time
from tests.shell.util import (get_impalad_host_port, assert_var_substitution,
run_impala_shell_cmd, ImpalaShell, build_shell_env, wait_for_query_state,
create_impala_shell_executable_dimension, get_impala_shell_executable)
create_impala_shell_executable_dimension, get_impala_shell_executable,
stderr_get_first_error_msg)
from contextlib import closing
@@ -274,8 +276,11 @@ class TestImpalaShell(ImpalaTestSuite):
args = ['-q', 'set abort_on_error=true;'
'select id from functional_parquet.bad_column_metadata t']
result = run_impala_shell_cmd(vector, args, expect_success=False)
assert 'ERROR: Column metadata states there are 11 values, ' in result.stderr
assert 'but read 10 values from column id.' in result.stderr
assert error_msg_expected(
stderr_get_first_error_msg(result.stderr),
"Column metadata states there are 11 values, but read 10 values from column id."
)
def test_completed_query_errors_2(self, vector):
if vector.get_value('strict_hs2_protocol'):
@@ -284,9 +289,10 @@ class TestImpalaShell(ImpalaTestSuite):
'select id, cnt from functional_parquet.bad_column_metadata t, '
'(select 1 cnt) u']
result = run_impala_shell_cmd(vector, args, expect_success=False)
assert 'ERROR: Column metadata states there are 11 values, ' in result.stderr,\
result.stderr
assert 'but read 10 values from column id.' in result.stderr, result.stderr
assert error_msg_expected(
stderr_get_first_error_msg(result.stderr),
"Column metadata states there are 11 values, but read 10 values from column id."
)
def test_no_warnings_in_log_with_quiet_mode(self, vector):
if vector.get_value('strict_hs2_protocol'):