Commit Graph

3 Commits

Author SHA1 Message Date
Lars Volker
6b566a2d35 IMPALA-3004: Fix QueryTest tests
Test files in testdata/workloads/functional-query/queries/QueryTest
are parsed by test_file_parser.py, which used to ignore everything
before the first ==== line as a file header. This change fixes all
affected files.

This change also modifies the test file parser to forbid headers
starting with what looks like a subsection title ('----'), which
should prevent the reintroduction of similar errors in the future.

Change-Id: Iaa1bc5ffd02782e24289c7843dcb35401c334519
Reviewed-on: http://gerrit.cloudera.org:8080/2220
Reviewed-by: Lars Volker <lv@cloudera.com>
Tested-by: Internal Jenkins
2016-02-19 00:03:15 -08:00
Juan Yu
d1c263402e IMPALA-1973: Fixing crash when uninitialized, empty row is added in HdfsTextScanner
This patch fixes an issue when an uninitialized, empty row is falsely
added to the rowbatch. The uninitialized data inside this row leads
later on to a crash when the null byte is checked together with the
offsets (that contains garbage).

The fix is to not only check for the number of materialized columns, but
as well for the number of materialized partition key columns. Only if both are
empty and the parser has an unfinished tuple, add the empty row.

To accommodate for the last row, check in FinishScanRange() if there is an
unfinished tuple with materialized slots or materialized partition key. Write
the fields if necessary.

Change-Id: I2808cc228e62d048d917d3a6352d869d117597ab
(cherry picked from commit c1795a8b40d10fbb32d9051a0e7de5ebffc8a6bd)
Reviewed-on: http://gerrit.cloudera.org:8080/364
Reviewed-by: Juan Yu <jyu@cloudera.com>
Tested-by: Internal Jenkins
2015-05-05 00:19:12 +00:00
Juan Yu
e121bc9b0a IMPALA-1476: Impala incorrectly handles text data missing a newline on the last line.
I did a local benchmark and there's minimal performance impact(<1%)

Change-Id: I8d84a145acad886c52587258b27d33cff96ea399
(cherry picked from commit 7e750ad5d90007cc85ebe493af4dce7a537ad7c0)
Reviewed-on: http://gerrit.cloudera.org:8080/189
Reviewed-by: Juan Yu <jyu@cloudera.com>
Tested-by: Internal Jenkins
2015-03-20 19:58:50 -07:00