mirror of
https://github.com/apache/impala.git
synced 2026-01-05 12:01:11 -05:00
IMPALA-3311: fix string data coming out of aggs in subplans
The problem: varlen data (e.g. strings) produced by aggregations is freed by FreeLocalAllocations() after passing up the output batch. This works for streaming operators or blocking operators that copy their input, but results in memory corruption when the output reaches non-copying blocking operators, e.g. SubplanNode and NestedLoopJoinNode. The fix: this patch makes the PartitionedAggregationNode copy out produced string data if the node is in a subplan. Otherwise it calls MarkNeedsToReturn() on the output batch. Marking the batch would work in the subplan case as well, but would likely be less efficient since it would result in many small batches coming out of the subplan. The patch includes a test case. However, this test only exposes the problem with an ASAN build and the --disable_mem_pools flag, which we don't currently have automated testing for. Change-Id: Iada891504c261ba54f4eb8c9d7e4e5223668d7b9 Reviewed-on: http://gerrit.cloudera.org:8080/2929 Reviewed-by: Dan Hecht <dhecht@cloudera.com> Tested-by: Internal Jenkins
This commit is contained in:
committed by
Tim Armstrong
parent
cb377741ec
commit
7767d300a3
@@ -450,3 +450,19 @@ inner join t2.int_array a
|
||||
---- TYPES
|
||||
bigint
|
||||
====
|
||||
---- QUERY
|
||||
# IMPALA-3311: test string data coming out of an agg in a subplan
|
||||
select id, m from complextypestbl t,
|
||||
(select min(cast(item as string)) m from t.int_array) v
|
||||
---- RESULTS
|
||||
1,'1'
|
||||
2,'1'
|
||||
3,'NULL'
|
||||
4,'NULL'
|
||||
5,'NULL'
|
||||
6,'NULL'
|
||||
7,'NULL'
|
||||
8,'-1'
|
||||
---- TYPES
|
||||
BIGINT,STRING
|
||||
====
|
||||
|
||||
Reference in New Issue
Block a user