Tim Armstrong
acaf8b9f0c
IMPALA-5570: fix spilling null-aware anti join
...
IMPALA-4672: Part 2 regressed NAAJ by tightening up the spilling
invariants (e.g. can't unpin with spilling disabled) but we
didn't have tests for spilling NAAJs that could detect the
regression. This patch adds those tests, fixes the regressions,
and improves NAAJ by reliably spilling the probe side and not
trying to bring the whole probe side into memory.
The changes are:
* All null-aware streams start off in memory and are only unpinned if
spilling is enabled.
* The null-aware build partition can be spilled in the same way as hash
partitions.
* Probe streams are unpinned whenever there is memory pressure - if
spilling is enabled and either a build partition is spilled or
appending to the probe stream fails.
* Spilled probe streams are not re-pinned in EvaluateNullProbe().
Instead we just iterate over the rows of the stream.
Testing:
Add query tests where the three different buckets of rows are large
enough to spill: the build and probe of the null-aware partition and the
null probe rows.
Test both spilling and in-memory (with spilling disabled) cases.
Change-Id: Ie2e60eb4dd32bd287a31479a6232400df65964c1
Reviewed-on: http://gerrit.cloudera.org:8080/7367
Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com >
Tested-by: Impala Public Jenkins
2017-08-24 04:24:10 +00:00
..
2016-09-13 21:57:36 +00:00
2017-07-06 01:26:44 +00:00
2017-08-17 03:12:48 +00:00
2017-04-29 02:23:51 +00:00
2017-02-14 05:56:33 +00:00
2017-02-22 23:10:34 +00:00
2017-06-22 19:08:59 +00:00
2017-05-27 05:19:53 +00:00
2017-08-22 08:27:12 +00:00
2016-12-07 09:45:11 +00:00
2016-07-14 19:04:43 +00:00
2014-09-26 12:28:03 -07:00
2014-09-26 12:24:07 -07:00
2015-03-07 09:51:27 +00:00
2016-11-24 08:03:39 +00:00
2017-02-22 23:10:34 +00:00
2017-07-25 03:31:19 +00:00
2016-05-12 14:17:50 -07:00
2015-04-21 19:27:50 +00:00
2017-02-22 23:10:34 +00:00
2016-08-31 00:58:03 +00:00
2016-09-02 02:47:02 +00:00
2017-05-12 15:43:30 +00:00
2017-06-09 23:29:17 +00:00
2017-05-12 15:43:30 +00:00
2017-05-12 15:43:30 +00:00
2017-05-02 01:12:14 +00:00
2016-07-14 19:04:44 +00:00
2017-03-02 20:12:05 +00:00
2017-02-22 06:31:14 +00:00
2016-06-07 09:34:30 -07:00
2016-06-07 09:34:30 -07:00
2016-06-07 09:34:30 -07:00
2016-06-07 09:34:30 -07:00
2017-06-29 21:14:59 +00:00
2017-08-07 00:57:46 +00:00
2015-07-16 19:38:17 +00:00
2016-02-19 00:03:15 -08:00
2016-08-19 06:04:18 +00:00
2016-01-27 20:41:45 +00:00
2016-03-02 23:23:04 -08:00
2017-08-12 08:10:07 +00:00
2017-08-12 08:10:07 +00:00
2017-08-12 08:10:07 +00:00
2017-08-12 08:10:07 +00:00
2017-08-04 21:51:19 +00:00
2017-08-17 03:12:48 +00:00
2017-07-26 05:43:01 +00:00
2017-07-21 19:16:57 +00:00
2017-02-22 23:10:34 +00:00
2017-02-22 23:10:34 +00:00
2014-05-08 13:59:00 -07:00
2015-02-23 23:32:41 +00:00
2015-08-22 01:46:26 +00:00
2015-02-23 23:32:41 +00:00
2015-02-23 23:32:41 +00:00
2016-10-13 00:40:41 +00:00
2015-02-23 23:32:41 +00:00
2017-02-22 23:10:34 +00:00
2014-01-08 10:52:36 -08:00
2015-02-23 23:32:41 +00:00
2017-04-26 01:12:01 +00:00
2015-04-21 19:27:50 +00:00
2016-11-15 03:27:36 +00:00
2015-10-12 14:41:00 -07:00
2014-06-24 02:14:27 -07:00
2017-05-09 01:36:46 +00:00
2016-02-19 00:03:15 -08:00
2014-01-08 10:52:14 -08:00
2016-05-12 14:17:59 -07:00
2014-01-08 10:46:49 -08:00
2016-02-19 00:03:15 -08:00
2015-10-07 14:47:40 -07:00
2015-08-22 01:46:26 +00:00
2014-01-08 10:53:51 -08:00
2015-05-27 22:25:12 +00:00
2014-12-02 18:08:09 -08:00
2016-05-12 14:17:49 -07:00
2017-05-22 00:40:04 +00:00
2016-02-19 00:03:15 -08:00
2017-07-07 01:30:59 +00:00
2015-02-23 23:32:41 +00:00
2015-10-12 14:41:05 -07:00
2016-10-25 05:52:33 +00:00
2017-06-22 19:08:59 +00:00
2017-08-07 19:53:59 +00:00
2016-12-07 07:31:16 +00:00
2017-05-11 20:55:51 +00:00
2017-07-22 00:28:36 +00:00
2016-12-10 00:05:50 +00:00
2016-12-07 05:11:13 +00:00
2016-12-06 10:41:53 +00:00
2016-12-07 07:31:16 +00:00
2017-05-11 20:55:51 +00:00
2017-05-11 20:55:51 +00:00
2017-05-18 21:09:51 +00:00
2017-07-21 21:49:04 +00:00
2017-07-21 21:49:04 +00:00
2017-02-15 01:33:23 +00:00
2016-11-09 03:27:12 +00:00
2015-02-23 23:32:41 +00:00
2016-11-09 03:27:12 +00:00
2016-07-19 23:30:02 -07:00
2016-06-08 16:30:32 -07:00
2016-09-03 00:39:07 +00:00
2016-11-17 05:31:34 +00:00
2014-01-08 10:48:09 -08:00
2016-11-03 11:59:07 +00:00
2017-03-17 19:33:31 +00:00
2016-12-02 01:46:55 +00:00
2016-12-02 01:46:55 +00:00
2016-12-02 01:46:55 +00:00
2016-05-31 23:32:11 -07:00
2017-03-01 02:00:19 +00:00
2016-05-12 23:06:36 -07:00
2015-09-02 19:23:54 +00:00
2015-09-02 19:23:54 +00:00
2016-04-01 05:06:38 +00:00
2015-09-02 19:23:54 +00:00
2015-09-02 19:23:54 +00:00
2017-08-18 02:53:23 +00:00
2017-08-05 01:03:02 +00:00
2015-10-07 14:47:40 -07:00
2015-08-22 01:46:26 +00:00
2016-12-08 02:20:50 +00:00
2016-12-08 02:20:50 +00:00
2017-08-19 20:56:10 +00:00
2017-03-03 01:43:42 +00:00
2016-05-12 14:17:48 -07:00
2017-03-09 05:07:44 +00:00
2017-03-09 05:07:44 +00:00
2016-08-25 10:20:36 +00:00
2016-06-20 15:37:18 -07:00
2016-06-20 15:37:18 -07:00
2017-05-09 15:47:21 +00:00
2017-05-31 06:49:43 +00:00
2017-03-03 10:20:07 +00:00
2017-07-22 05:53:06 +00:00
2017-07-06 01:26:44 +00:00
2016-10-14 05:41:22 +00:00
2017-05-25 02:55:36 +00:00
2016-09-08 04:31:27 +00:00
2017-07-25 03:31:19 +00:00
2016-11-22 00:42:57 +00:00
2016-05-12 23:06:35 -07:00
2017-06-08 21:31:08 +00:00
2017-08-05 01:03:02 +00:00
2017-06-08 21:31:08 +00:00
2016-11-03 23:08:56 +00:00
2015-09-27 15:13:32 -07:00
2016-08-31 03:12:30 +00:00
2017-07-19 06:48:06 +00:00
2017-08-23 03:27:26 +00:00
2017-05-12 15:43:30 +00:00
2016-09-29 21:14:13 +00:00
2017-02-22 23:10:34 +00:00
2016-07-07 10:41:29 -07:00
2017-08-05 01:03:02 +00:00
2017-08-05 01:03:02 +00:00
2016-01-26 03:13:05 +00:00
2016-09-30 01:21:05 +00:00
2016-06-02 21:33:08 -07:00
2017-08-23 03:27:26 +00:00
2017-08-23 03:27:26 +00:00
2017-08-24 04:24:10 +00:00
2017-08-24 04:24:10 +00:00
2017-08-07 00:57:46 +00:00
2017-08-23 03:59:02 +00:00
2017-08-12 08:10:07 +00:00
2017-03-03 01:43:42 +00:00
2017-03-03 01:43:42 +00:00
2016-05-12 23:06:36 -07:00
2016-07-19 23:30:02 -07:00
2016-02-28 21:31:37 -08:00
2014-09-26 12:28:03 -07:00
2016-05-12 14:17:45 -07:00
2017-02-22 23:10:34 +00:00
2016-11-09 03:27:12 +00:00
2017-08-17 03:12:48 +00:00
2016-11-09 03:27:12 +00:00
2016-11-09 03:27:12 +00:00
2016-12-08 04:53:53 +00:00
2016-12-08 04:53:53 +00:00
2016-11-09 03:27:12 +00:00
2016-12-08 04:53:53 +00:00
2017-02-03 23:35:25 +00:00
2017-05-04 02:04:03 +00:00
2014-01-08 10:48:09 -08:00
2017-07-19 22:17:13 +00:00
2016-01-23 23:59:27 +00:00
2014-09-18 00:36:03 -07:00
2016-09-02 02:47:02 +00:00
2014-06-20 13:35:10 -07:00
2014-01-08 10:54:01 -08:00
2015-10-07 14:47:40 -07:00