Lars Volker 3bfda33487 IMPALA-6193: Track memory of incoming data streams ...

This change adds memory tracking to incoming transmit data RPCs when
using KRPC. We track memory against a global tracker called "Data Stream
Service" until it is handed over to the stream manager. There we track
it in a global tracker called "Data Stream Queued RPC Calls" until a
receiver registers and takes over the early sender RPCs. Inside the
receiver, memory for deferred RPCs is tracked against the fragment
instance's memtracker until we unpack the batches and add them to the
row batch queue.

The DCHECK in MemTracker::Close() covers that all memory consumed by a
tracker gets release eventually. In addition to that, this change adds a
custom cluster test that makes sure that queued memory gets tracked by
inspecting the peak consumption of the new memtrackers.

Change-Id: I2df1204d2483313a8a18e5e3be6cec9e402614c4
Reviewed-on: http://gerrit.cloudera.org:8080/8914
Reviewed-by: Lars Volker <lv@cloudera.com>
Tested-by: Impala Public Jenkins

2018-02-01 08:53:36 +00:00

..

__init__.py

Add insert cancellation test, dynamically build failpoint dimensions from explain output

2014-01-08 10:52:27 -08:00

mem_usage_verifier.py

IMPALA-6193: Track memory of incoming data streams

2018-02-01 08:53:36 +00:00

metric_verifier.py

IMPALA-6151: add query-level fragment/backend counters

2017-11-07 21:44:34 +00:00

test_verify_metrics.py

IMPALA-3918: Remove Cloudera copyrights and add ASF license header

2016-08-09 08:19:41 +00:00