jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2026-01-04 09:00:56 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-5725: coalesce() with outer join incorrectly rewritten

Browse Source 
A recent change, IMPALA-5016, added an expr rewrite rule to simplfy
coalesce(). This rule eliminates the coalesce() when its first
parameter (that isn't constant null) is a SlotRef pointing to a
SlotDescriptor that is non-nullable (for example because it is from
a non-nullable Kudu column or because it is from an HDFS partition
column with no null partitions), under the assumption that the SlotRef
could never have a null value.

This assumption is violated when the SlotRef is the output of an
outer join, leading to incorrect results being returned. The problem
is that the nullability of a SlotDescriptor (which determines whether
there is a null indicator bit in the tuple for that slot) is a
slightly different property than the nullability of a SlotRef pointing
to that SlotDescriptor (since the SlotRef can still be NULL if the
entire tuple is NULL).

This patch removes the portion of the rewrite rule that considers
the nullability of the SlotDescriptor. This means that we're missing
out on some optimizations opportunities and we should revisit this in
a way that works with outer joins (IMPALA-5753)

Testing:
- Updated FE tests.
- Added regression tests to exprs.test

Change-Id: I1ca6df949f9d416ab207016236dbcb5886295337
Reviewed-on: http://gerrit.cloudera.org:8080/7567
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Reviewed-by: Thomas Tauber-Marshall <tmarshall@cloudera.com>
Tested-by: Impala Public Jenkins
This commit is contained in:
Thomas Tauber-Marshall
2017-08-02 10:51:57 -07:00
committed by Impala Public Jenkins
parent dced731671
commit 2ae94e7ead
4 changed files with 29 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
testdata/workloads/functional-query/queries/QueryTest/exprs.test vendored
View File

@@ -2741,4 +2741,26 @@ select if (true, 0, sum(id)) from functional.alltypestiny
0
---- TYPES
BIGINT
====
====
---- QUERY
# IMPALA-5725: test coalesce() is not rewritten when its first parameter is a non-nullable
# Kudu column, but can have NULL values due to an outer join.
select coalesce(b.id, a.id), b.id, a.id
from functional_kudu.alltypes a left join functional_kudu.alltypestiny b on a.id = b.id
where a.id = 100
---- RESULTS
100,NULL,100
---- TYPES
INT,INT,INT
====
---- QUERY
# IMPALA-5725: test coalesce() is not rewritten when its first parameter is an HDFS
# partition col with no NULL partitions, but can have NULL values due to an outer join.
select coalesce(b.year, a.id), b.id, a.id
from functional.alltypes a left join functional.alltypestiny b on a.id = b.id
where a.id = 100
---- RESULTS
100,NULL,100
---- TYPES
INT,INT,INT
====