Michael Ho 13007f9634 IMPALA-561: Allow multiple callbacks in a thread resource pool. ...

Previously, thread resource manager only supports a single callback
for each resource pool. The callback is invoked when a thread token
is available. This mostly works as scan node is the only consumer
and there is usually one scan node in a plan fragment. As shown in
IMPALA-3064 and IMPALA-561, it's possible to generate a plan fragment
with more than one scan nodes. In which case, one of the scan nodes
may be running with single thread and in debug builds, a DCHECK will
be hit.

This change fixes the problem by allowing more than one callbacks in
a given resource pool. The thread resource manager will go through all
the registered callbacks in round robin fashion.

This change also adds a missing thread token release call in
HdfsScanNode::ThreadTokenAvailableCb().

Change-Id: Iddfff1feef0b59d407994ad3bc560166acbfa623
Reviewed-on: http://gerrit.cloudera.org:8080/2430
Reviewed-by: Dan Hecht <dhecht@cloudera.com>
Tested-by: Internal Jenkins

2016-03-10 23:16:29 +00:00

3.5 KiB

Raw Blame History

View Raw