mirror of
https://github.com/apache/impala.git
synced 2026-01-24 15:00:45 -05:00
9ef9dacaf7
As shown in IMPALA-7828. there is some non-determinism on whether the errors detected in FragmentInstanceState::Close() will show up in the final profile sent to the coordinator. The reason is that the current code marks a fragment instance as "done" after ExecInternal() completes but before Close() is called. There is a window between when the final status report is sent and when Close() finishes. This change fixes the problem by not sending the final report until Close() is called. This has no implication on the first row available time for normal queries. It may slightly lengthen the first row available time for DML queries. Testing done: Updated udf-no-expr-rewrite.test to exercise this test Perf run on an 8 node clusters didn't show any regression: TPCH-300 +------------+-----------------------+---------+------------+------------+----------------+ | Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +------------+-----------------------+---------+------------+------------+----------------+ | TPCH(_300) | parquet / none / none | 23.94 | -2.05% | 12.55 | -2.62% | +------------+-----------------------+---------+------------+------------+----------------+ Small concurrency +-------------------------+-----------------------+---------+------------+------------+----------------+ | Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +-------------------------+-----------------------+---------+------------+------------+----------------+ | TPCDS-UNMODIFIED(_1000) | parquet / none / none | 6.89 | -0.66% | 6.62 | +0.41% | +-------------------------+-----------------------+---------+------------+------------+----------------+ Medium concurrency +-------------------------+-----------------------+---------+------------+------------+----------------+ | Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +-------------------------+-----------------------+---------+------------+------------+----------------+ | TPCDS-UNMODIFIED(_1000) | parquet / none / none | 55.57 | -1.04% | 55.27 | -0.98% | +-------------------------+-----------------------+---------+------------+------------+----------------+ Change-Id: I61618854ae3f4e7ef20028dcb0ff5cbcfa8adb01 Reviewed-on: http://gerrit.cloudera.org:8080/11939 Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
28 lines
503 B
Plaintext
28 lines
503 B
Plaintext
====
|
|
---- QUERY
|
|
# Only run without expr rewrites (constant folding) because count_rows() is
|
|
# non-deterministic.
|
|
select count_rows() from functional.alltypestiny;
|
|
---- TYPES
|
|
BIGINT
|
|
---- RESULTS
|
|
1
|
|
2
|
|
3
|
|
4
|
|
5
|
|
6
|
|
7
|
|
8
|
|
====
|
|
---- QUERY
|
|
# Due to IMPALA-3860, the error is not propagated when expr rewrites is enabled.
|
|
select mem_test_leaks(100);
|
|
---- TYPES
|
|
bigint
|
|
---- RESULTS
|
|
100
|
|
---- ERRORS
|
|
UDF WARNING: Memory leaked via FunctionContext::Allocate(), 100 bytes leaked via FunctionContext::TrackAllocation()
|
|
====
|