jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-19 18:12:08 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-6759: align stress test memory estimation parse pattern

Browse Source 
The stress test never expected to see memory estimates on the order of
PB. Apparently it can happen with TPC DS 10000, so update the pattern.

It's not clear how to quickly write a test to catch this, because it
involves crossing language boundaries and possibly having a
massively-scaled dataset. I think leaving a comment in both places is
good enough for now.

Change-Id: I317c271888584ed2a817ee52ad70267eae64d341
Reviewed-on: http://gerrit.cloudera.org:8080/9846
Reviewed-by: Lars Volker <lv@cloudera.com>
Tested-by: Impala Public Jenkins
This commit is contained in:
Michael Brown
2018-03-28 15:14:20 -07:00
committed by Impala Public Jenkins
parent 77efb2820e
commit 4028e9c5ec
3 changed files with 13 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
tests/util/parse_util.py
View File

@@ -56,15 +56,17 @@ def parse_mem_to_mb(mem, units):
if units.endswith("B"):
units = units[:-1]
if not units:
mem /= 10 ** 6
mem /= 2 ** 20
elif units == "K":
mem /= 10 ** 3
mem /= 2 ** 10
elif units == "M":
pass
elif units == "G":
mem *= 10 ** 3
mem *= 2 ** 10
elif units == "T":
mem *= 10 ** 6
mem *= 2 ** 20
elif units == "P":
mem *= 2 ** 30
else:
raise Exception('Unexpected memory unit "%s"' % units)
return int(mem)