Abhishek Rawat 763acffb74
IMPALA-6590: Disable expr rewrites and codegen for VALUES() statements ...
Expression rewrites for VALUES() could result in performance regression
since there is virtually no benefit of rewrite, if the expression will
only ever be evaluated once. The overhead of rewrites in some cases
could be huge, especially if there are several constant expressions.
The regression also seems to non-linearly increase as number of columns
increases. Similarly, there is no value in doing codegen for such const
expressions.
The rewriteExprs() for ValuesStmt class was overridden with an empty
function body. As a result rewrites for VALUES() is a no-op.
Codegen was disabled for const expressions within a UNION node, if
the UNION node is not within a subplan. This applies to all UNION nodes
with const expressions (and not just limited to UNION nodes associated
with a VALUES clause).
The decision for whether or not to enable codegen for const expressions
in a UNION is made in the planner when a UnionNode is initialized. A new
member 'is_codegen_disabled' was added to the thrift struct TExprNode
for communicating this decision to backend. The Optimizer should take
decisions it can and so it seemed like the right place to disable/enable
codegen. The infrastructure is generic and could be extended in future
to selectively disable codegen for any given expression, if needed.
Testing:
- Added a new e2e test case in tests/query_test/test_codegen.py, which
tests the different scenarios involving UNION with const expressions.
- Passed exhaustive unit-tests.
- Ran manual tests to validate that the non-linear regression in VALUES
clause when involving increasing number of columns is no longer seen.
Results below.
for i in 256 512 1024 2048 4096 8192 16384 32768;
do (echo 'VALUES ('; for x in $(seq $i);
do echo "cast($x as string),"; done;
echo "NULL); profile;") |
time impala-shell.sh -f /dev/stdin |& grep Analysis; done
Base:
- Analysis finished: 20.137ms (19.215ms)
- Analysis finished: 46.275ms (44.597ms)
- Analysis finished: 119.642ms (116.663ms)
- Analysis finished: 361.195ms (355.856ms)
- Analysis finished: 1s277ms (1s266ms)
- Analysis finished: 5s664ms (5s640ms)
- Analysis finished: 29s689ms (29s646ms)
- Analysis finished: 2m (2m)
Test:
- Analysis finished: 1.868ms (986.520us)
- Analysis finished: 3.195ms (1.856ms)
- Analysis finished: 7.332ms (3.484ms)
- Analysis finished: 13.896ms (8.071ms)
- Analysis finished: 31.015ms (18.963ms)
- Analysis finished: 60.157ms (38.125ms)
- Analysis finished: 113.694ms (67.642ms)
- Analysis finished: 253.044ms (163.180ms)
Change-Id: I229d67b821968321abd8f97f7c89cf2617000d8d
Reviewed-on: http://gerrit.cloudera.org:8080/13645
Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com >
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com >
2021-12-16 05:51:27 +00:00
2019-08-13 18:55:09 +00:00
2020-09-01 22:27:27 +00:00
2020-05-12 23:29:04 +00:00
2019-07-27 13:45:51 +00:00
2021-01-05 21:31:05 +00:00
2020-08-12 17:45:50 +00:00
2020-05-18 16:51:42 +00:00
2019-07-27 13:45:51 +00:00
2019-11-07 14:30:47 +00:00
2019-07-03 20:46:45 +00:00
2020-05-20 21:00:44 +00:00
2020-05-20 21:00:44 +00:00
2020-05-20 21:00:44 +00:00
2020-09-02 13:29:25 +00:00
2021-11-23 03:24:08 +00:00
2021-03-12 14:23:04 +00:00
2021-03-12 14:23:04 +00:00
2021-03-12 14:23:04 +00:00
2016-09-13 21:57:36 +00:00
2019-11-26 19:14:24 +00:00
2019-10-16 02:04:12 +00:00
2020-07-08 22:59:57 +00:00
2020-08-13 02:22:31 +00:00
2018-09-29 11:59:03 +00:00
2017-02-14 05:56:33 +00:00
2020-05-12 23:29:04 +00:00
2021-06-10 15:46:56 +00:00
2021-02-10 23:52:28 +00:00
2018-08-08 20:51:45 +00:00
2021-04-20 22:21:32 +00:00
2021-10-23 00:08:14 +00:00
2020-03-31 19:11:20 +00:00
2019-09-27 17:18:35 +00:00
2019-09-27 17:18:35 +00:00
2018-05-15 22:23:14 +00:00
2021-02-18 14:31:51 +00:00
2021-04-30 23:36:31 +00:00
2018-10-09 00:45:10 +00:00
2018-04-11 02:21:48 +00:00
2018-11-17 01:48:05 +00:00
2019-08-13 18:55:09 +00:00
2021-03-02 04:58:51 +00:00
2020-05-12 23:29:04 +00:00
2020-05-12 23:29:04 +00:00
2021-04-02 21:50:17 +00:00
2019-08-06 16:38:07 +00:00
2020-05-12 23:29:04 +00:00
2020-05-12 23:29:04 +00:00
2021-01-21 19:35:26 +00:00
2018-12-08 00:21:55 +00:00
2015-04-21 19:27:50 +00:00
2021-09-14 21:21:47 +00:00
2020-12-11 19:51:28 +00:00
2020-08-13 03:10:16 +00:00
2021-07-06 18:35:30 +00:00
2020-06-24 03:47:18 +00:00
2021-05-07 14:11:08 +00:00
2020-06-24 03:47:18 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2019-05-05 02:11:55 +00:00
2021-04-20 22:21:32 +00:00
2021-09-15 13:58:23 +00:00
2021-09-15 13:58:23 +00:00
2021-09-15 13:58:23 +00:00
2021-09-15 13:58:23 +00:00
2020-06-24 16:51:49 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2020-09-28 14:02:35 +00:00
2020-07-08 19:32:15 +00:00
2020-08-13 18:58:23 +00:00
2018-01-25 04:33:11 +00:00
2019-01-14 09:07:03 +00:00
2019-04-23 13:33:57 +00:00
2021-03-23 22:52:38 +00:00
2018-10-09 00:45:10 +00:00
2021-03-12 14:23:04 +00:00
2018-04-11 02:21:48 +00:00
2018-04-11 02:21:48 +00:00
2021-07-06 18:35:30 +00:00
2020-04-02 12:01:41 +00:00
2020-06-05 17:43:32 +00:00
2019-05-15 22:34:28 +00:00
2018-11-01 07:30:58 +00:00
2015-07-16 19:38:17 +00:00
2018-04-12 22:01:35 +00:00
2018-02-02 01:13:08 +00:00
2020-10-21 05:20:33 +00:00
2020-07-31 17:23:45 +00:00
2018-02-15 10:55:55 +00:00
2018-02-15 10:55:55 +00:00
2021-06-14 17:13:25 +00:00
2018-05-12 01:43:37 +00:00
2019-01-12 04:03:26 +00:00
2021-03-12 14:23:04 +00:00
2021-03-02 04:58:51 +00:00
2021-08-25 05:31:42 +00:00
2020-02-22 02:02:56 +00:00
2020-08-12 17:45:50 +00:00
2020-09-30 21:01:54 +00:00
2021-03-18 19:35:58 +00:00
2020-07-14 12:53:51 +00:00
2018-12-11 02:47:16 +00:00
2021-03-12 00:00:03 +00:00
2020-07-14 03:13:18 +00:00
2019-02-07 23:05:30 +00:00
2020-05-12 23:29:04 +00:00
2020-05-12 23:29:04 +00:00
2015-02-23 23:32:41 +00:00
2019-04-23 13:33:57 +00:00
2015-02-23 23:32:41 +00:00
2015-02-23 23:32:41 +00:00
2019-04-23 13:33:57 +00:00
2020-05-12 23:29:04 +00:00
2015-02-23 23:32:41 +00:00
2017-10-10 01:30:33 +00:00
2017-09-21 17:38:08 +00:00
2015-04-21 19:27:50 +00:00
2020-12-11 19:51:28 +00:00
2018-07-03 23:49:44 +00:00
2019-06-05 14:23:23 +00:00
2021-03-02 04:58:51 +00:00
2017-05-09 01:36:46 +00:00
2016-02-19 00:03:15 -08:00
2016-05-12 14:17:59 -07:00
2020-01-17 18:54:33 +00:00
2019-09-27 17:18:35 +00:00
2020-02-14 04:26:16 +00:00
2020-02-11 15:08:39 +00:00
2021-08-03 16:27:07 +00:00
2021-10-04 17:34:56 +00:00
2021-07-20 23:58:06 +00:00
2021-07-15 15:15:07 +00:00
2021-07-20 23:58:06 +00:00
2021-11-04 17:55:21 +00:00
2021-08-30 23:09:42 +00:00
2021-10-18 19:32:29 +00:00
2021-05-20 19:19:50 +00:00
2021-07-15 15:15:07 +00:00
2021-09-02 21:34:41 +00:00
2021-09-02 21:34:41 +00:00
2020-09-09 13:26:42 +00:00
2021-10-18 19:32:29 +00:00
2020-11-23 21:47:55 +00:00
2021-07-15 15:15:07 +00:00
2021-09-02 21:34:41 +00:00
2021-09-08 18:38:37 +00:00
2021-09-08 18:38:37 +00:00
2018-08-09 19:18:08 +00:00
2021-04-20 22:21:32 +00:00
2021-04-20 22:21:32 +00:00
2020-05-11 19:32:08 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2015-05-27 22:25:12 +00:00
2019-06-05 11:15:04 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2020-05-11 19:32:08 +00:00
2018-11-06 00:00:12 +00:00
2020-12-11 19:51:28 +00:00
2020-07-31 17:23:45 +00:00
2016-02-19 00:03:15 -08:00
2020-09-04 00:55:02 +00:00
2019-10-22 04:21:51 +00:00
2015-02-23 23:32:41 +00:00
2020-02-05 22:00:48 +00:00
2021-06-10 15:46:56 +00:00
2020-04-01 15:48:36 +00:00
2020-04-01 15:48:36 +00:00
2020-04-01 15:48:36 +00:00
2020-11-03 20:05:20 +00:00
2018-05-11 22:41:49 +00:00
2020-06-05 00:40:33 +00:00
2018-04-27 21:55:11 +00:00
2020-03-18 18:05:34 +00:00
2020-03-18 18:05:34 +00:00
2020-04-01 15:48:36 +00:00
2020-04-01 15:48:36 +00:00
2017-05-11 20:55:51 +00:00
2017-05-11 20:55:51 +00:00
2018-09-26 06:51:56 +00:00
2020-04-01 15:48:36 +00:00
2019-10-21 20:23:56 +00:00
2017-07-21 21:49:04 +00:00
2020-05-13 22:01:34 +00:00
2018-04-11 02:21:48 +00:00
2021-04-20 22:21:32 +00:00
2020-10-08 22:49:59 +00:00
2018-12-27 21:02:57 +00:00
2020-12-11 19:51:28 +00:00
2018-04-11 02:21:48 +00:00
2020-02-22 02:02:56 +00:00
2020-06-09 16:26:23 +00:00
2019-03-08 04:39:08 +00:00
2021-11-03 03:26:27 +00:00
2021-02-15 22:25:41 +00:00
2019-02-12 20:37:45 +00:00
2016-11-03 11:59:07 +00:00
2017-03-17 19:33:31 +00:00
2016-12-02 01:46:55 +00:00
2019-11-26 07:28:23 +00:00
2016-12-02 01:46:55 +00:00
2021-11-30 20:13:18 +00:00
2019-09-25 16:58:14 +00:00
2020-12-11 19:51:28 +00:00
2021-09-14 21:21:47 +00:00
2018-01-05 20:44:21 +00:00
2020-10-21 20:30:05 +00:00
2017-11-22 22:00:16 +00:00
2020-07-15 17:10:50 +00:00
2015-09-02 19:23:54 +00:00
2021-05-03 20:42:30 +00:00
2016-04-01 05:06:38 +00:00
2015-09-02 19:23:54 +00:00
2018-11-17 01:48:05 +00:00
2019-01-22 01:20:31 +00:00
2019-03-08 04:39:08 +00:00
2020-04-16 15:45:49 +00:00
2019-03-08 04:39:08 +00:00
2019-03-08 04:39:08 +00:00
2019-03-08 04:39:08 +00:00
2015-10-07 14:47:40 -07:00
2015-08-22 01:46:26 +00:00
2021-05-07 14:11:08 +00:00
2021-10-21 15:45:39 +00:00
2020-01-17 18:54:33 +00:00
2019-10-04 18:36:22 +00:00
2018-10-01 13:20:40 +00:00
2018-10-01 13:20:40 +00:00
2021-10-28 20:09:46 +00:00
2017-08-19 20:56:10 +00:00
2021-01-27 17:30:37 +00:00
2017-03-03 01:43:42 +00:00
2021-08-21 14:46:51 +00:00
2021-08-21 14:46:51 +00:00
2021-08-21 14:46:51 +00:00
2021-08-05 07:55:26 +00:00
2017-03-09 05:07:44 +00:00
2019-03-14 22:43:50 +00:00
2018-03-08 04:48:36 +00:00
2018-01-17 04:18:24 +00:00
2021-06-03 06:32:45 +00:00
2021-06-03 06:32:45 +00:00
2021-08-05 07:55:26 +00:00
2018-09-10 16:09:41 +00:00
2018-09-10 16:09:41 +00:00
2016-06-20 15:37:18 -07:00
2016-06-20 15:37:18 -07:00
2020-06-18 21:00:27 +00:00
2021-07-14 12:51:09 +00:00
2021-07-14 12:51:09 +00:00
2018-08-22 18:08:20 +00:00
2017-05-09 15:47:21 +00:00
2019-08-24 00:32:28 +00:00
2019-05-07 00:36:56 +00:00
2021-01-14 19:34:38 +00:00
2018-02-22 00:57:46 +00:00
2021-10-28 20:09:46 +00:00
2018-11-27 02:01:12 +00:00
2019-05-10 11:46:38 +00:00
2019-05-10 11:46:38 +00:00
2019-05-10 11:46:38 +00:00
2020-11-24 18:35:07 +00:00
2020-12-10 19:01:08 +00:00
2021-01-05 23:30:35 +00:00
2018-03-22 02:47:33 +00:00
2017-07-22 05:53:06 +00:00
2020-05-21 18:40:20 +00:00
2018-08-23 15:55:53 +00:00
2016-10-14 05:41:22 +00:00
2020-08-25 15:42:01 +00:00
2020-12-11 19:51:28 +00:00
2018-04-11 02:21:48 +00:00
2019-05-10 12:06:01 +00:00
2020-05-20 23:03:23 +00:00
2020-05-20 23:03:23 +00:00
2020-05-20 23:03:23 +00:00
2021-02-10 23:52:28 +00:00
2018-08-15 04:15:46 +00:00
2018-08-15 04:15:46 +00:00
2020-12-22 06:10:39 +00:00
2017-12-13 10:04:40 +00:00
2021-02-13 23:02:54 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-03-31 05:38:50 +00:00
2020-04-07 15:10:27 +00:00
2021-09-14 21:21:47 +00:00
2021-03-31 05:38:50 +00:00
2021-03-31 05:38:50 +00:00
2019-08-06 02:35:54 +00:00
2021-07-09 13:40:40 +00:00
2017-11-17 21:33:51 +00:00
2021-07-09 13:40:40 +00:00
2021-03-02 04:58:51 +00:00
2020-04-14 15:42:13 +00:00
2020-05-15 21:46:38 +00:00
2018-11-17 01:48:05 +00:00
2019-02-12 20:37:45 +00:00
2021-03-14 03:35:40 +00:00
2020-07-08 19:32:15 +00:00
2019-01-09 05:26:50 +00:00
2016-08-31 03:12:30 +00:00
2021-08-13 16:04:19 +00:00
2020-04-02 12:01:41 +00:00
2019-09-26 18:35:06 +00:00
2021-10-04 17:34:56 +00:00
2016-09-29 21:14:13 +00:00
2020-05-12 23:29:04 +00:00
2020-04-14 12:18:21 +00:00
2017-08-05 01:03:02 +00:00
2017-08-05 01:03:02 +00:00
2018-04-28 23:41:39 +00:00
2018-02-02 01:13:08 +00:00
2020-10-21 05:20:33 +00:00
2021-01-06 04:39:56 +00:00
2021-03-02 04:58:51 +00:00
2021-08-25 20:05:47 +00:00
2020-08-25 16:11:20 +00:00
2020-04-24 20:56:58 +00:00
2018-04-28 23:41:39 +00:00
2020-08-25 16:11:20 +00:00
2017-08-29 23:01:10 +00:00
2021-03-02 04:58:51 +00:00
2019-01-22 01:20:31 +00:00
2021-09-22 20:42:10 +00:00
2021-05-15 22:20:58 +00:00
2017-03-03 01:43:42 +00:00
2017-03-03 01:43:42 +00:00
2018-11-01 21:27:13 +00:00
2018-11-01 21:27:13 +00:00
2019-08-22 05:03:03 +00:00
2021-11-10 21:11:30 +00:00
2020-08-25 23:08:40 +00:00
2020-10-21 05:20:33 +00:00
2021-04-20 22:21:32 +00:00
2018-04-11 02:21:48 +00:00
2016-02-28 21:31:37 -08:00
2019-09-27 01:54:39 +00:00
2021-04-20 22:21:32 +00:00
2021-05-28 04:02:10 +00:00
2016-11-09 03:27:12 +00:00
2019-04-23 13:33:57 +00:00
2016-11-09 03:27:12 +00:00
2018-04-11 02:21:48 +00:00
2016-12-08 04:53:53 +00:00
2016-12-08 04:53:53 +00:00
2016-11-09 03:27:12 +00:00
2018-11-20 23:49:11 +00:00
2019-05-15 22:34:28 +00:00
2021-12-16 05:51:27 +00:00
2021-12-09 23:17:20 +00:00
2020-06-15 23:42:12 +00:00
2018-07-03 20:34:27 +00:00
2021-05-14 13:58:04 +00:00
2021-09-15 05:04:07 +00:00
2021-08-15 20:40:39 +00:00
2019-05-24 15:08:19 +00:00
2019-05-24 15:08:19 +00:00
2020-07-14 03:13:18 +00:00
2019-04-23 13:33:57 +00:00
2020-10-21 05:20:33 +00:00
2021-11-23 07:03:10 +00:00
2021-11-23 07:03:10 +00:00
2021-11-23 07:03:10 +00:00
763acffb74