Taras Bobrovytsky 575b5a20e6
IMPALA-5017: Error on decimal overflow ...
Before this patch, decimal operations would either silently overflow (in
the case of sum() and avg()), or produce a warning.
In this patch, the behaviour is changed so that an error is produced in
the case of overflow when DECIMAL_v2 is enabled. Decimal v1 behaviour is
unchanged.
We introduce overflow checks when computing sum() and avg(). This
results in a ~30% performance regression when we are in decimal v2 mode
compared to decimal v1.
Benchmarks:
Query:
select sum(dec_38_19) from decimal_tbl
Decimal v1: 11.57s
Decimal v2: 16.58s
Query:
select avg(dec_38_19) from decimal_tbl
Decimal v1: 12.08s
Decimal v2: 17.08s
The performance regression is not as bad if we are computing the sum or
average of decimal column with a lower precision:
Query:
select sum(dec_9_5) from decimal_tbl
Decimal v1: 11.06s
Decimal v2: 13.08s
Query:
select avg(dec_9_5) from decimal_tbl
Decimal v1: 11.56s
Decimal v2: 13.57s
Testing:
- Added several end to end tests.
- Updated Expr tests to check for error in case of overflow.
Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019
Reviewed-on: http://gerrit.cloudera.org:8080/8404
Reviewed-by: Taras Bobrovytsky <tbobrovytsky@cloudera.com >
Tested-by: Impala Public Jenkins
2017-12-01 23:23:01 +00:00
2017-08-30 21:20:47 +00:00
2016-09-13 21:57:36 +00:00
2017-08-30 21:20:47 +00:00
2017-10-06 00:01:08 +00:00
2017-04-29 02:23:51 +00:00
2017-02-14 05:56:33 +00:00
2017-11-29 22:37:01 +00:00
2017-09-06 03:05:38 +00:00
2017-05-27 05:19:53 +00:00
2017-08-22 08:27:12 +00:00
2016-12-07 09:45:11 +00:00
2016-07-14 19:04:43 +00:00
2017-11-17 21:33:51 +00:00
2017-11-17 21:33:51 +00:00
2016-11-24 08:03:39 +00:00
2017-02-22 23:10:34 +00:00
2017-11-29 22:37:01 +00:00
2016-05-12 14:17:50 -07:00
2017-11-29 22:37:01 +00:00
2017-11-29 22:37:01 +00:00
2016-08-31 00:58:03 +00:00
2016-09-02 02:47:02 +00:00
2017-05-12 15:43:30 +00:00
2017-06-09 23:29:17 +00:00
2017-05-12 15:43:30 +00:00
2017-05-12 15:43:30 +00:00
2017-05-02 01:12:14 +00:00
2016-07-14 19:04:44 +00:00
2017-12-01 23:23:01 +00:00
2017-12-01 23:23:01 +00:00
2016-06-07 09:34:30 -07:00
2016-06-07 09:34:30 -07:00
2016-06-07 09:34:30 -07:00
2016-06-07 09:34:30 -07:00
2017-06-29 21:14:59 +00:00
2017-08-07 00:57:46 +00:00
2015-07-16 19:38:17 +00:00
2016-02-19 00:03:15 -08:00
2016-08-19 06:04:18 +00:00
2016-01-27 20:41:45 +00:00
2017-09-22 05:08:22 +00:00
2017-09-22 05:08:22 +00:00
2017-08-12 08:10:07 +00:00
2017-08-12 08:10:07 +00:00
2017-11-29 22:37:01 +00:00
2017-11-29 22:37:01 +00:00
2017-11-29 05:22:32 +00:00
2017-08-17 03:12:48 +00:00
2017-07-26 05:43:01 +00:00
2017-10-04 03:14:20 +00:00
2017-11-29 22:37:01 +00:00
2017-11-29 22:37:01 +00:00
2015-08-22 01:46:26 +00:00
2016-10-13 00:40:41 +00:00
2017-02-22 23:10:34 +00:00
2017-10-10 01:30:33 +00:00
2017-09-21 17:38:08 +00:00
2016-11-15 03:27:36 +00:00
2015-10-12 14:41:00 -07:00
2017-05-09 01:36:46 +00:00
2016-02-19 00:03:15 -08:00
2016-05-12 14:17:59 -07:00
2017-10-28 06:29:38 +00:00
2017-09-28 18:36:17 +00:00
2015-08-22 01:46:26 +00:00
2015-05-27 22:25:12 +00:00
2016-05-12 14:17:49 -07:00
2017-05-22 00:40:04 +00:00
2016-02-19 00:03:15 -08:00
2017-07-07 01:30:59 +00:00
2017-08-30 20:22:25 +00:00
2017-06-22 19:08:59 +00:00
2017-08-07 19:53:59 +00:00
2016-12-07 07:31:16 +00:00
2017-05-11 20:55:51 +00:00
2017-09-27 18:55:26 +00:00
2016-12-10 00:05:50 +00:00
2016-12-07 05:11:13 +00:00
2016-12-06 10:41:53 +00:00
2016-12-07 07:31:16 +00:00
2017-05-11 20:55:51 +00:00
2017-05-11 20:55:51 +00:00
2017-11-29 05:53:15 +00:00
2017-07-21 21:49:04 +00:00
2017-02-15 01:33:23 +00:00
2016-11-09 03:27:12 +00:00
2016-11-09 03:27:12 +00:00
2016-07-19 23:30:02 -07:00
2016-06-08 16:30:32 -07:00
2016-09-03 00:39:07 +00:00
2017-11-17 21:33:51 +00:00
2016-11-17 05:31:34 +00:00
2016-11-03 11:59:07 +00:00
2017-03-17 19:33:31 +00:00
2016-12-02 01:46:55 +00:00
2016-12-02 01:46:55 +00:00
2016-12-02 01:46:55 +00:00
2016-05-31 23:32:11 -07:00
2017-11-22 22:00:16 +00:00
2017-11-08 07:39:02 +00:00
2015-09-02 19:23:54 +00:00
2015-09-02 19:23:54 +00:00
2016-04-01 05:06:38 +00:00
2015-09-02 19:23:54 +00:00
2015-09-02 19:23:54 +00:00
2017-08-18 02:53:23 +00:00
2017-08-05 01:03:02 +00:00
2015-10-07 14:47:40 -07:00
2015-08-22 01:46:26 +00:00
2016-12-08 02:20:50 +00:00
2016-12-08 02:20:50 +00:00
2017-08-19 20:56:10 +00:00
2017-03-03 01:43:42 +00:00
2016-05-12 14:17:48 -07:00
2017-03-09 05:07:44 +00:00
2017-03-09 05:07:44 +00:00
2016-08-25 10:20:36 +00:00
2016-06-20 15:37:18 -07:00
2016-06-20 15:37:18 -07:00
2017-11-07 04:34:26 +00:00
2017-11-16 21:23:09 +00:00
2017-05-09 15:47:21 +00:00
2017-05-31 06:49:43 +00:00
2017-03-03 10:20:07 +00:00
2017-07-22 05:53:06 +00:00
2017-07-06 01:26:44 +00:00
2016-10-14 05:41:22 +00:00
2017-05-25 02:55:36 +00:00
2016-09-08 04:31:27 +00:00
2017-07-25 03:31:19 +00:00
2016-11-22 00:42:57 +00:00
2017-11-17 21:33:51 +00:00
2017-11-28 03:01:39 +00:00
2017-10-25 18:56:00 +00:00
2017-11-29 05:53:15 +00:00
2015-09-27 15:13:32 -07:00
2016-08-31 03:12:30 +00:00
2017-07-19 06:48:06 +00:00
2017-11-28 00:31:15 +00:00
2017-05-12 15:43:30 +00:00
2016-09-29 21:14:13 +00:00
2017-11-30 07:33:16 +00:00
2016-07-07 10:41:29 -07:00
2017-08-05 01:03:02 +00:00
2017-08-05 01:03:02 +00:00
2016-01-26 03:13:05 +00:00
2016-09-30 01:21:05 +00:00
2016-06-02 21:33:08 -07:00
2017-08-25 00:48:28 +00:00
2017-08-23 03:27:26 +00:00
2017-08-24 04:24:10 +00:00
2017-08-24 04:24:10 +00:00
2017-08-29 23:01:10 +00:00
2017-08-24 06:27:05 +00:00
2017-08-25 01:17:46 +00:00
2017-11-29 22:37:01 +00:00
2017-03-03 01:43:42 +00:00
2017-03-03 01:43:42 +00:00
2016-05-12 23:06:36 -07:00
2016-07-19 23:30:02 -07:00
2016-02-28 21:31:37 -08:00
2016-05-12 14:17:45 -07:00
2017-11-29 22:37:01 +00:00
2016-11-09 03:27:12 +00:00
2017-08-17 03:12:48 +00:00
2016-11-09 03:27:12 +00:00
2017-11-17 02:24:48 +00:00
2016-12-08 04:53:53 +00:00
2016-12-08 04:53:53 +00:00
2016-11-09 03:27:12 +00:00
2016-12-08 04:53:53 +00:00
2017-02-03 23:35:25 +00:00
2017-05-04 02:04:03 +00:00
2017-07-19 22:17:13 +00:00
2016-01-23 23:59:27 +00:00
2016-09-02 02:47:02 +00:00
2015-10-07 14:47:40 -07:00
575b5a20e6