mirror of
https://github.com/apache/impala.git
synced 2025-12-19 18:12:08 -05:00
ea8f74a6ac
This patch standardizes tests against workload management tables (sys.impala_query_log and sys.impala_query_live) to use a common superclass named WorkloadManagementTestSuite. The setup_method of this superclass waits for workload management init completion (wait_for_wm_init_complete()), while the teardown_method waits until impala-server.completed-queries.queued metric reaches 0 (wait_for_wm_idle()). test_query_log.py and test_workload_mgmt_sql_details.py are refactored to extend from WorkloadManagementTestSuite. Tests to assert the query log table flush behavior are grouped together in TestQueryLogTableFlush. test_workload_mgmt_sql_details.py::TestWorkloadManagementSQLDetails now uses 1 minicluster instance for all tests. test_workload_mgmt_init.py does not extend from WorkloadManagementTestSuite because it is testing cluster start and restart scenario. This patch only adds wait_for_wm_idle() at teardown_method where it make sense to do so. test_query_live.py does not extend from WorkloadManagementTestSuite because most of its test method require long --query_log_write_interval_s so that DML queries from workload management worker does not disturb sys.impala_query_live. workload_mgmt parameter in CustomClusterTestSuite.with_args() is standardized to setup appropriate default flags in cluster_setup() rather than passing it down to _start_impala_cluster(): IMPALAD_ARGS --enable_workload_mgmt=true --query_log_write_interval_s=1 \ --shutdown_grace_period_s=0 --shutdown_deadline_s=60 and CATALOGD_ARGS --enable_workload_mgmt=true Note that IMPALAD_ARGS and CATALOGD_ARGS flags added by workload_mgmt and impalad_graceful_shutdown parameter are still overridable to different value by explicitly adding it in the impalad_args and catalogd_args parameters. Setting workload_mgmt=True now automatically enables graceful shutdown for the test. Thus, impalad_graceful_shutdown=True is now removed. With beeswax protocol deprecated, this patch also changes the protocol under test from beeswax to hs2. TestQueryLogTableBeeswax is now renamed to TestQueryLogTableBasic. Additionally, print total wait time in wait_for_metric_value(). Testing: - Run modified tests and pass. Change-Id: Iecf6452fa963304e263805ebeb017c843d17dd16 Reviewed-on: http://gerrit.cloudera.org:8080/22617 Reviewed-by: Riza Suminto <riza.suminto@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
41 lines
1.7 KiB
Python
41 lines
1.7 KiB
Python
# Licensed to the Apache Software Foundation (ASF) under one
|
|
# or more contributor license agreements. See the NOTICE file
|
|
# distributed with this work for additional information
|
|
# regarding copyright ownership. The ASF licenses this file
|
|
# to you under the Apache License, Version 2.0 (the
|
|
# "License"); you may not use this file except in compliance
|
|
# with the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing,
|
|
# software distributed under the License is distributed on an
|
|
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
# KIND, either express or implied. See the License for the
|
|
# specific language governing permissions and limitations
|
|
# under the License.
|
|
|
|
from __future__ import absolute_import, division, print_function
|
|
|
|
from tests.common.custom_cluster_test_suite import CustomClusterTestSuite
|
|
|
|
|
|
class WorkloadManagementTestSuite(CustomClusterTestSuite):
|
|
"""Base class for tests that exercise workload management behavior.
|
|
The setup_method waits for workload management init completion, while the
|
|
teardown_method waits until impala-server.completed-queries.queued metric
|
|
reaches 0."""
|
|
|
|
def setup_method(self, method):
|
|
super(WorkloadManagementTestSuite, self).setup_method(method)
|
|
self.wait_for_wm_init_complete()
|
|
|
|
def teardown_method(self, method):
|
|
self.wait_for_wm_idle()
|
|
super(WorkloadManagementTestSuite, self).teardown_method(method)
|
|
|
|
def get_client(self, protocol):
|
|
"""Retrieves the default Impala client for the specified protocol. This client is
|
|
automatically closed after the test completes."""
|
|
return self.default_impala_client(protocol=protocol)
|