mirror of
https://github.com/apache/impala.git
synced 2025-12-30 03:01:44 -05:00
b00310ca89
CodegenWriteSlot() receives negative length values for the lengths of the slots passed to it if the slots contain escape characters. (This is currently only for non-string types, as we do not codegen string types with escaped characters). The DelimitedTextParser is responsible for identifying escape characters and assigning the negative lengths appropriately. CodegenWriteCompleteTuple() passes this length to CodegenWriteSlot() as it is. This differs from the behavior of WriteSlot() where the length passed to it is always positive, as all the callers of WriteSlot() make sure of that (including WriteCompleteTuple()). The IrIsNullString() and IrGenericIsNullString() functions are responsibe for checking if the given data contains a NULL pattern. They are called by CodegenWriteSlot(). A NULL pattern usually contains an escaped character which means that the length of that slot will be a negative length. However, the IrIsNullString() and IrGenericIsNullString() that take the same length argument from CodegenWriteSlot() always expect a positive length argument. So, no slots were ever marked as NULL by these NULL-checking functions when codegen was enabled. NULL slots were still detected accidentally because of some incorrect code in CodegenWriteSlot() that marked invalid slots and NULL slots as NULL. Therefore, due to this code, even invalid slots were not marked as invalid and did not return an error. Instead they were just sliently marked as NULL. This patch makes sure that only positive lengths are passed to CodegenWriteSlot() so that NULL checking is correct and it also makes sure that invalid slots are not silently marked as NULL. Testing: Re-enabled an older hdfs-scan-node-errors test. Formatted it to fit new error message format after IMPALA-3859 and IMPALA-3895. Change-Id: I858e427ad7c2b2da8c2bb657be06b7443655781f Reviewed-on: http://gerrit.cloudera.org:8080/5377 Reviewed-by: Sailesh Mukil <sailesh@cloudera.com> Tested-by: Internal Jenkins