Skye Wanderman-Milne de531e15bd IMPALA-694: Allow Impala to read files produced by parquet-mr version <= 1.2.8 ...

parquet-mr had a bug where it didn't include the dictionary page's
header in the total column size. We now compensate for this by
detecting these files and padding the scan range length. This required
changing how the scanner detects when it's finished: it now counts the
number of rows rather than checking eosr (since the scan range may be
longer than the column).

Change-Id: Id9933808b965003c0c3b3aa78c32fe29a0c4bcbe
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1097
Reviewed-by: Skye Wanderman-Milne <skye@cloudera.com>
Tested-by: jenkins

2014-01-08 10:54:27 -08:00

169 B

Executable File

Raw History

View Raw