Thomas Tauber-Marshall
4e17839033
IMPALA-5611: KuduPartitionExpr holds onto memory unnecessarily
...
IMPALA-3742 introduced KuduPartitionExpr, which takes a row and passes
it to the Kudu client to determine what partitionit belongs to.
The DataStreamSender never frees the local allocations for the Kudu
partition exprs causing it to hang on to memory longer than it needs to.
This patch also fixes two other related issues:
- DataStreamSender was dropping the Status from AddRow in the Kudu
branch. Adds 'RETURN_IF_ERROR' and 'WARN_UNUSED_RESULT'
- Changes the HASH case in DataStreamSender to call FreeLocalAllocations
on a per-batch basis, instead of a per-row basis.
Testing:
- Added an e2e test that runs a large insert with a mem limit that
failed with oom previously.
Change-Id: Ia661eb8bed114070728a1497ccf7ed6893237e5e
Reviewed-on: http://gerrit.cloudera.org:8080/7346
Reviewed-by: Dan Hecht <dhecht@cloudera.com >
Reviewed-by: Michael Ho <kwho@cloudera.com >
Tested-by: Impala Public Jenkins
2017-07-06 21:45:39 +00:00
..
2016-09-13 21:57:36 +00:00
2017-07-06 01:26:44 +00:00
2017-03-16 05:59:40 +00:00
2017-04-29 02:23:51 +00:00
2017-02-14 05:56:33 +00:00
2017-02-22 23:10:34 +00:00
2017-06-22 19:08:59 +00:00
2017-05-27 05:19:53 +00:00
2016-11-24 00:23:58 +00:00
2016-12-07 09:45:11 +00:00
2016-07-14 19:04:43 +00:00
2014-09-26 12:28:03 -07:00
2014-09-26 12:24:07 -07:00
2015-03-07 09:51:27 +00:00
2016-11-24 08:03:39 +00:00
2017-02-22 23:10:34 +00:00
2017-02-22 23:10:34 +00:00
2016-05-12 14:17:50 -07:00
2015-04-21 19:27:50 +00:00
2017-02-22 23:10:34 +00:00
2016-08-31 00:58:03 +00:00
2016-09-02 02:47:02 +00:00
2017-05-12 15:43:30 +00:00
2017-06-09 23:29:17 +00:00
2017-05-12 15:43:30 +00:00
2017-05-12 15:43:30 +00:00
2017-05-02 01:12:14 +00:00
2016-07-14 19:04:44 +00:00
2017-03-02 20:12:05 +00:00
2017-02-22 06:31:14 +00:00
2016-06-07 09:34:30 -07:00
2016-06-07 09:34:30 -07:00
2016-06-07 09:34:30 -07:00
2016-06-07 09:34:30 -07:00
2017-06-29 21:14:59 +00:00
2015-07-16 19:38:17 +00:00
2016-02-19 00:03:15 -08:00
2016-08-19 06:04:18 +00:00
2016-01-27 20:41:45 +00:00
2016-03-02 23:23:04 -08:00
2017-04-18 20:36:08 +00:00
2017-04-18 20:36:08 +00:00
2017-07-03 00:04:54 +00:00
2017-07-03 00:04:54 +00:00
2017-04-24 21:41:11 +00:00
2016-09-02 02:47:02 +00:00
2017-02-04 01:47:23 +00:00
2017-02-22 23:10:34 +00:00
2017-02-22 23:10:34 +00:00
2014-05-08 13:59:00 -07:00
2015-02-23 23:32:41 +00:00
2015-08-22 01:46:26 +00:00
2015-02-23 23:32:41 +00:00
2015-02-23 23:32:41 +00:00
2016-10-13 00:40:41 +00:00
2015-02-23 23:32:41 +00:00
2017-02-22 23:10:34 +00:00
2014-01-08 10:52:36 -08:00
2015-02-23 23:32:41 +00:00
2017-04-26 01:12:01 +00:00
2015-04-21 19:27:50 +00:00
2016-11-15 03:27:36 +00:00
2015-10-12 14:41:00 -07:00
2014-06-24 02:14:27 -07:00
2017-05-09 01:36:46 +00:00
2016-02-19 00:03:15 -08:00
2014-01-08 10:52:14 -08:00
2016-05-12 14:17:59 -07:00
2014-01-08 10:46:49 -08:00
2016-02-19 00:03:15 -08:00
2015-10-07 14:47:40 -07:00
2015-08-22 01:46:26 +00:00
2014-01-08 10:53:51 -08:00
2015-05-27 22:25:12 +00:00
2014-12-02 18:08:09 -08:00
2016-05-12 14:17:49 -07:00
2017-05-22 00:40:04 +00:00
2016-02-19 00:03:15 -08:00
2016-11-09 03:27:12 +00:00
2015-02-23 23:32:41 +00:00
2015-10-12 14:41:05 -07:00
2016-10-25 05:52:33 +00:00
2017-06-22 19:08:59 +00:00
2017-06-02 01:47:48 +00:00
2016-12-07 07:31:16 +00:00
2017-05-11 20:55:51 +00:00
2017-07-06 21:45:39 +00:00
2016-12-10 00:05:50 +00:00
2016-12-07 05:11:13 +00:00
2016-12-06 10:41:53 +00:00
2016-12-07 07:31:16 +00:00
2017-05-11 20:55:51 +00:00
2017-05-11 20:55:51 +00:00
2017-05-18 21:09:51 +00:00
2016-12-07 07:31:16 +00:00
2016-11-05 06:43:45 +00:00
2017-02-15 01:33:23 +00:00
2015-09-10 04:50:31 +00:00
2016-11-09 03:27:12 +00:00
2015-02-23 23:32:41 +00:00
2016-11-09 03:27:12 +00:00
2016-07-19 23:30:02 -07:00
2016-06-08 16:30:32 -07:00
2016-09-03 00:39:07 +00:00
2016-11-17 05:31:34 +00:00
2014-01-08 10:48:09 -08:00
2016-11-03 11:59:07 +00:00
2017-03-17 19:33:31 +00:00
2016-12-02 01:46:55 +00:00
2016-12-02 01:46:55 +00:00
2016-12-02 01:46:55 +00:00
2016-05-31 23:32:11 -07:00
2017-03-01 02:00:19 +00:00
2016-05-12 23:06:36 -07:00
2015-09-02 19:23:54 +00:00
2015-09-02 19:23:54 +00:00
2016-04-01 05:06:38 +00:00
2015-09-02 19:23:54 +00:00
2015-09-02 19:23:54 +00:00
2017-06-22 21:56:25 +00:00
2017-06-13 05:13:44 +00:00
2015-10-07 14:47:40 -07:00
2015-08-22 01:46:26 +00:00
2016-12-08 02:20:50 +00:00
2016-12-08 02:20:50 +00:00
2016-02-10 07:16:58 +00:00
2017-03-03 01:43:42 +00:00
2016-05-12 14:17:48 -07:00
2017-03-09 05:07:44 +00:00
2017-03-09 05:07:44 +00:00
2016-08-25 10:20:36 +00:00
2016-06-20 15:37:18 -07:00
2016-06-20 15:37:18 -07:00
2017-05-09 15:47:21 +00:00
2017-05-31 06:49:43 +00:00
2017-03-03 10:20:07 +00:00
2017-07-06 01:26:44 +00:00
2017-07-06 01:26:44 +00:00
2016-10-14 05:41:22 +00:00
2017-05-25 02:55:36 +00:00
2016-09-08 04:31:27 +00:00
2016-12-22 10:45:39 +00:00
2016-11-22 00:42:57 +00:00
2016-05-12 23:06:35 -07:00
2017-06-08 21:31:08 +00:00
2016-09-01 02:35:41 +00:00
2017-06-08 21:31:08 +00:00
2016-11-03 23:08:56 +00:00
2015-09-27 15:13:32 -07:00
2016-08-31 03:12:30 +00:00
2017-05-27 05:19:53 +00:00
2017-06-29 05:26:56 +00:00
2017-05-12 15:43:30 +00:00
2016-09-29 21:14:13 +00:00
2017-02-22 23:10:34 +00:00
2016-07-07 10:41:29 -07:00
2017-06-22 21:56:25 +00:00
2017-06-27 05:36:11 +00:00
2016-01-26 03:13:05 +00:00
2016-09-30 01:21:05 +00:00
2016-06-02 21:33:08 -07:00
2017-06-30 22:49:40 +00:00
2017-06-29 21:14:59 +00:00
2017-03-03 01:43:42 +00:00
2017-03-03 01:43:42 +00:00
2016-05-12 23:06:36 -07:00
2016-07-19 23:30:02 -07:00
2016-02-28 21:31:37 -08:00
2014-09-26 12:28:03 -07:00
2016-05-12 14:17:45 -07:00
2017-02-22 23:10:34 +00:00
2016-11-09 03:27:12 +00:00
2017-04-26 21:40:32 +00:00
2016-11-09 03:27:12 +00:00
2016-11-09 03:27:12 +00:00
2016-12-08 04:53:53 +00:00
2016-12-08 04:53:53 +00:00
2016-11-09 03:27:12 +00:00
2016-12-08 04:53:53 +00:00
2017-02-03 23:35:25 +00:00
2017-05-04 02:04:03 +00:00
2014-01-08 10:48:09 -08:00
2016-01-23 23:59:27 +00:00
2014-09-18 00:36:03 -07:00
2016-09-02 02:47:02 +00:00
2014-06-20 13:35:10 -07:00
2014-01-08 10:54:01 -08:00
2015-10-07 14:47:40 -07:00