impala

mirror of https://github.com/apache/impala.git synced 2026-01-07 09:02:19 -05:00

Files

Alex Behm 71fd1941f0 IMPALA-5955: Use totalSize tblproperty instead of rawDataSize.

Today, Impala populates the 'rawDataSize' property
during COMPUTE STATS for the purpose of extrapolating
row counts based on file sizes.

After this patch Impala will populate 'totalSize' instead of
'rawDataSize'. The 'rawDataSize' is not populated or used.

Intended meaning/use of tblproperties:
- rawDataSize' is the estimated in-memory size of a table
  (without encoding and compression)
- 'totalSize' represents the on-disk size

Using the fields correctly is important for compatibility
with other users of the HMS such as Hive and SparkSQL.
For example, SparkSQL relies on the 'totalSize' for
join ordering.

Testing:
- core/hdfs run passed

Change-Id: If7c2c4e1e99b297c849f9f0d18b2bef34ad811c6
Reviewed-on: http://gerrit.cloudera.org:8080/8110
Tested-by: Impala Public Jenkins
Reviewed-by: Alex Behm <alex.behm@cloudera.com>

2017-09-22 03:39:04 +00:00

queries/PlannerTest

IMPALA-5955: Use totalSize tblproperty instead of rawDataSize.

2017-09-22 03:39:04 +00:00

functional-planner_core.csv

Update benchmark tests to run against generic workload, data loading with scale factor, +more

2014-01-08 10:44:22 -08:00

functional-planner_dimensions.csv

Update benchmark tests to run against generic workload, data loading with scale factor, +more

2014-01-08 10:44:22 -08:00

functional-planner_exhaustive.csv

Update benchmark tests to run against generic workload, data loading with scale factor, +more