mirror of
https://github.com/apache/impala.git
synced 2026-01-26 21:02:23 -05:00
d48ffc2d45
With more coordinators running in local catalog mode, the expected RPC traffic on the Catalog server is higher compared to the non-local-catalog mode. Each such RPC is handled in its own thread and consumes some non-trivial CPU for serializing and deserializing the metadata. With this change, the maximum number of threads performing the actual work are capped to a certain limit at any point and the remaining requests (if any) are blocked until the current requests are serviced or they they exceed the configured timeout and abort. Adds the following parameters for controlling this behavior. --catalog_max_parallel_partial_fetch_rpc --catalog_partial_fetch_rpc_queue_timeout_s --catalog_partial_fetch_rpc_queue_timeout_s controls the timeout for queued requests. Added some basic supportability to examine the queue length via metrics. Added a unit test to make sure the concurrent requests for this RPC method does not exceed the configured value. Change-Id: I11f77a16cfa38ada42d8b7c859850198ea7dd142 Reviewed-on: http://gerrit.cloudera.org:8080/11561 Reviewed-by: Vuk Ercegovac <vercegovac@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>