Skye Wanderman-Milne 68fef6a5bf IMPALA-2213: make Parquet scanner fail query if the file size metadata is stale ...

This patch changes the Parquet scanner to check if it can't read the
full footer scan range, indicating that file has been overwritten by a
shorter file without refreshing the table metadata. Before it would
DCHECK. This patch adds a test for this case, as well as the case
where the new file is longer than the metadata states (which fails
with an existing error).

Change-Id: Ie2031ac2dc90e4f2573bd3ca8a3709db60424f07
Reviewed-on: http://gerrit.cloudera.org:8080/1084
Tested-by: Internal Jenkins
Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>

2015-10-01 13:58:39 -07:00

..

array-value-builder-test.cc

IMPALA-2377: fix error case in ArrayValueBuilder::GetFreeMemory()

2015-10-01 13:58:35 -07:00

array-value-builder.h

IMPALA-2377: fix error case in ArrayValueBuilder::GetFreeMemory()

2015-10-01 13:58:35 -07:00

array-value.h

Nested types: serialization of row batches containing collections

2015-08-13 20:59:33 +00:00

buffered-block-mgr-test.cc

Update and reenable some scratch file tests.

2015-09-23 15:15:52 -07:00

buffered-block-mgr.cc

IMPALA-2252: Crash (likely race) tearing down BufferedBlockMgr on query failure

2015-09-27 15:13:30 -07:00

buffered-block-mgr.h

IMPALA-2245: Fix for client->num_tmp_reserved_buffers_ accounting

2015-09-24 10:58:55 -07:00

buffered-tuple-stream-ir.cc

IMPALA-2295: deep copy arrays in BufferedTupleStream

2015-09-09 02:39:14 +00:00

buffered-tuple-stream-test.cc

IMPALA-1621,2241,2271,2330,2352: Lazy switch to IO buffers to reduce min mem needed for PAGG/PHJ

2015-09-23 11:07:42 -07:00

buffered-tuple-stream.cc

IMPALA-1621,2241,2271,2330,2352: Lazy switch to IO buffers to reduce min mem needed for PAGG/PHJ

2015-09-23 11:07:42 -07:00

buffered-tuple-stream.h

IMPALA-1621,2241,2271,2330,2352: Lazy switch to IO buffers to reduce min mem needed for PAGG/PHJ

2015-09-23 11:07:42 -07:00

buffered-tuple-stream.inline.h

IMPALA-1621,2241,2271,2330,2352: Lazy switch to IO buffers to reduce min mem needed for PAGG/PHJ

2015-09-23 11:07:42 -07:00

client-cache.cc

IMPALA-1916: Replace Status::OK by Status::OK()

2015-05-22 09:53:13 +00:00

client-cache.h

IMPALA-2189: [RM] Retry logic for Llama RPC may throw exception

2015-09-30 17:17:52 -07:00

CMakeLists.txt

IMPALA-2377: fix error case in ArrayValueBuilder::GetFreeMemory()

2015-10-01 13:58:35 -07:00

coordinator.cc

IMPALA-2046: Print hostname along with backend number

2015-09-27 15:13:31 -07:00

coordinator.h

IMPALA-2046: Print hostname along with backend number

2015-09-27 15:13:31 -07:00

data-stream-mgr.cc

IMPALA-1916: Replace Status::OK by Status::OK()

2015-05-22 09:53:13 +00:00

data-stream-mgr.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

data-stream-recvr.cc

IMPALA-2417: crash when updating counter in DataStreamRecvr

2015-09-30 17:17:41 -07:00

data-stream-recvr.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

data-stream-sender.cc

Nested Types: Dedup tuples when de/serializing

2015-08-31 18:45:57 +00:00

data-stream-sender.h

Nested Types: Dedup tuples when de/serializing

2015-08-31 18:45:57 +00:00

data-stream-test.cc

Nested types: send materialized path to BE

2015-09-02 10:30:24 +00:00

decimal-test.cc

IMPALA-1952: Expand parsing of decimals to include scientific notation

2015-05-06 17:22:42 +00:00

decimal-value.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

descriptors.cc

IMPALA-2213: make Parquet scanner fail query if the file size metadata is stale

2015-10-01 13:58:39 -07:00

descriptors.h

IMPALA-2213: make Parquet scanner fail query if the file size metadata is stale

2015-10-01 13:58:39 -07:00

disk-io-mgr-internal.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

disk-io-mgr-reader-context.cc

IMPALA-1916: Replace Status::OK by Status::OK()

2015-05-22 09:53:13 +00:00

disk-io-mgr-scan-range.cc

IMPALA-2366: check fread return code correctly

2015-09-22 10:58:33 -07:00

disk-io-mgr-stress-test.cc

Deprecating namespace directive declarations (std, boost)

2015-04-18 01:26:47 +00:00

disk-io-mgr-stress.cc

Deprecating namespace directive declarations (std, boost)

2015-04-18 01:26:47 +00:00

disk-io-mgr-stress.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

disk-io-mgr-test.cc

IMPALA-2366: check fread return code correctly

2015-09-22 10:58:33 -07:00

disk-io-mgr.cc

IMPALA-1621,2241,2271,2330,2352: Lazy switch to IO buffers to reduce min mem needed for PAGG/PHJ

2015-09-23 11:07:42 -07:00

disk-io-mgr.h

IMPALA-1621,2241,2271,2330,2352: Lazy switch to IO buffers to reduce min mem needed for PAGG/PHJ

2015-09-23 11:07:42 -07:00

exec-env.cc

IMPALA-2079: Part 2: handling tmp device errors

2015-09-03 01:54:32 +00:00

exec-env.h

IMPALA-2079: Part 2: handling tmp device errors

2015-09-03 01:54:32 +00:00

free-pool-test.cc

Deprecating namespace directive declarations (std, boost)

2015-04-18 01:26:47 +00:00

free-pool.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

hbase-table-factory.cc

IMPALA-1916: Replace Status::OK by Status::OK()

2015-05-22 09:53:13 +00:00

hbase-table-factory.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

hbase-table.cc

IMPALA-1916: Replace Status::OK by Status::OK()

2015-05-22 09:53:13 +00:00

hbase-table.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

hdfs-fs-cache-test.cc

Remove std namespace directive declarations.

2015-05-06 03:18:49 +00:00

hdfs-fs-cache.cc

IMPALA-2062: Replace DCHECK_NOTNULL(X) with DCHECK(X != NULL)

2015-06-25 22:36:47 +00:00

hdfs-fs-cache.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

lib-cache.cc

IMPALA-1916: Replace Status::OK by Status::OK()

2015-05-22 09:53:13 +00:00

lib-cache.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

mem-pool-test.cc

Fix MemPool allocations of INT_MAX.

2015-09-30 17:17:54 -07:00

mem-pool.cc

IMPALA-2284: Disallow long (1<<30) strings in group_concat()

2015-09-23 15:15:55 -07:00

mem-pool.h

Fix MemPool allocations of INT_MAX.

2015-09-30 17:17:54 -07:00

mem-tracker-test.cc

Nested Types: Simple BE performance improvements.

2015-09-24 10:58:58 -07:00

mem-tracker.cc

IMPALA-1735: ExpandRmReservation only check parent pools with limit

2015-06-30 02:36:30 +00:00

mem-tracker.h

Nested Types: Simple BE performance improvements.

2015-09-24 10:58:58 -07:00

multi-precision-test.cc

Fix broken build due to changed boost code

2015-04-20 23:17:32 +00:00

multi-precision.cc

Deprecating namespace directive declarations (std, boost)

2015-04-18 01:26:47 +00:00

multi-precision.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

parallel-executor-test.cc

IMPALA-1916: Replace Status::OK by Status::OK()

2015-05-22 09:53:13 +00:00

parallel-executor.cc

Deprecating namespace directive declarations (std, boost)

2015-04-18 01:26:47 +00:00

parallel-executor.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

plan-fragment-executor.cc

IMPALA-2062: Replace DCHECK_NOTNULL(X) with DCHECK(X != NULL)

2015-06-25 22:36:47 +00:00

plan-fragment-executor.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

raw-value-test.cc

IMPALA-2270: avoid FnvHash64to32 with empty inputs

2015-09-02 11:07:03 +00:00

raw-value.cc

Nested types: serialization of row batches containing collections

2015-08-13 20:59:33 +00:00

raw-value.h

IMPALA-2270: avoid FnvHash64to32 with empty inputs

2015-09-02 11:07:03 +00:00

row-batch-serialize-test.cc

IMPALA-2299: dedup zero-length tuples correctly

2015-09-22 10:58:31 -07:00

row-batch.cc

Nested Types: Simple BE performance improvements.

2015-09-24 10:58:58 -07:00

row-batch.h

Add a TODO about duplicate tuples for RowBatch::DeepCopyTo

2015-09-22 10:58:33 -07:00

runtime-state.cc

Fix reporting of custom OOM error messages.

2015-09-27 15:13:29 -07:00

runtime-state.h

Fix reporting of custom OOM error messages.

2015-09-27 15:13:29 -07:00

sorted-run-merger.cc

IMPALA-2062: Replace DCHECK_NOTNULL(X) with DCHECK(X != NULL)

2015-06-25 22:36:47 +00:00

sorted-run-merger.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

sorter.cc

IMPALA-2369, IMPALA-2435: Impala crashes when the sorter hits an OOM error

2015-09-30 17:17:43 -07:00

sorter.h

Switch std::list to std::deque in Sorter

2015-09-09 01:33:37 +00:00

string-buffer-test.cc

IMPALA-2249: Avoid allocating StringBuffer > 1GB in ScannerContext::Stream::GetBytesInternal()

2015-09-30 17:17:46 -07:00

string-buffer.h

IMPALA-2249: Avoid allocating StringBuffer > 1GB in ScannerContext::Stream::GetBytesInternal()

2015-09-30 17:17:46 -07:00

string-search.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

string-value-test.cc

Deprecating namespace directive declarations (std, boost)

2015-04-18 01:26:47 +00:00

string-value.cc

Deprecating namespace directive declarations (std, boost)

2015-04-18 01:26:47 +00:00

string-value.h

IMPALA-2284: Disallow long (1<<30) strings in group_concat()

2015-09-23 15:15:55 -07:00

string-value.inline.h

Nested types: serialization of row batches containing collections

2015-08-13 20:59:33 +00:00

thread-resource-mgr-test.cc

Deprecating namespace directive declarations (std, boost)

2015-04-18 01:26:47 +00:00

thread-resource-mgr.cc

Fix broken build due to changed boost code

2015-04-20 23:17:32 +00:00

thread-resource-mgr.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

timestamp-parse-util.cc

Deprecating namespace directive declarations (std, boost)

2015-04-18 01:26:47 +00:00

timestamp-parse-util.h

IMPALA-1963: Part2 Support timezone offset in format string.

2015-06-30 01:10:33 +00:00

timestamp-test.cc

Deprecating namespace directive declarations (std, boost)

2015-04-18 01:26:47 +00:00

timestamp-value.cc

Make UTC to local TimestampValue conversion faster

2015-06-19 22:46:13 +00:00

timestamp-value.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

tmp-file-mgr-test.cc

Update and reenable some scratch file tests.

2015-09-23 15:15:52 -07:00

tmp-file-mgr.cc

IMPALA-2305: avoid blacklisting to prevent test failures

2015-09-06 22:48:25 +00:00

tmp-file-mgr.h

IMPALA-2079: Part 2: handling tmp device errors

2015-09-03 01:54:32 +00:00

tuple-row.cc

Order by without limit for Impala

2014-06-09 16:58:08 -07:00

tuple-row.h

Rewrite header comments to use Doxygen-compatible ///

2015-05-07 23:07:57 +00:00

tuple.cc

IMPALA-2322: Set new pointer for ArrayValue in Tuple::DeepCopyVarlenData().

2015-09-22 10:58:32 -07:00

tuple.h

Nested Types: Fix projection of collection-typed slots.

2015-09-22 10:58:33 -07:00

types.cc

Nested types: serialization of row batches containing collections

2015-08-13 20:59:33 +00:00

types.h

Nested types: read and materialize nested types in Parquet scanner

2015-09-02 19:23:54 +00:00