Minghui Zhu 88275f327f
IMPALA-12051: Propagate analytic tuple predicates of outer-joined InlineView ...
In some cases, direct pushing down predicates that reference analytic
tuple into inline view leads to incorrect query results. While pushing
down analytic predicates (e.g. row_number() < 10), we should also divide
them into two groups. Some of them can be migrated into the view so are
removed in the current scope. Some of them can be copied into the view
but still need to be evaluated in the current scope as demonstrated with
the following query. The bug is due to we migrate all of them into the
view.
WITH detail_measure AS (
SELECT
*
FROM
(
VALUES
(
1 AS `isqbiuar`,
1 AS `bgsfrbun`,
1 AS `result_type`,
1 AS `bjuzzevg`
),
(2, 2, 2, 2)
) a
),
order_measure_sql0 AS (
SELECT
row_number() OVER (
ORDER BY
row_number_0 DESC NULLS LAST,
isqbiuar ASC NULLS LAST
) AS `row_number_0`,
`isqbiuar`
FROM
(
VALUES
(1 AS `row_number_0`, 1 AS `isqbiuar`),
(2, 2)
) b
)
SELECT
detail_measure.`isqbiuar` AS `isqbiuar`,
detail_measure.`bgsfrbun` AS `bgsfrbun`,
detail_measure.`result_type` AS `result_type`,
detail_measure.`bjuzzevg` AS `bjuzzevg`,
`row_number_0` AS `row_number_0`
FROM
detail_measure
LEFT JOIN order_measure_sql0
ON order_measure_sql0.isqbiuar = detail_measure.isqbiuar
WHERE
row_number_0 BETWEEN 1
AND 1
ORDER BY
`row_number_0` ASC NULLS LAST,
`bgsfrbun` ASC NULLS LAST
The current incorrect result is:
+----------+----------+-------------+----------+--------------+
| isqbiuar | bgsfrbun | result_type | bjuzzevg | row_number_0 |
+----------+----------+-------------+----------+--------------+
| 2 | 2 | 2 | 2 | 1 |
| 1 | 1 | 1 | 1 | NULL |
+----------+----------+-------------+----------+--------------+
The correct result is:
+----------+----------+-------------+----------+--------------+
| isqbiuar | bgsfrbun | result_type | bjuzzevg | row_number_0 |
+----------+----------+-------------+----------+--------------+
| 2 | 2 | 2 | 2 | 1 |
+----------+----------+-------------+----------+--------------+
In the plan, the analysis predicate is pushed down to the TOP-N node,
but not in the HASH JOIN node, which leads to incorrect results.
...
05:HASH JOIN [RIGHT OUTER JOIN]
| hash predicates: isqbiuar = isqbiuar
| row-size=14B cardinality=2
...
02:TOP-N [LIMIT=1]
| order by: row_number_0 DESC NULLS LAST, isqbiuar ASC NULLS LAST
| source expr: row_number() <= CAST(1 AS BIGINT)
| row-size=2B cardinality=1
...
The HASH JOIN node shoud be:
05:HASH JOIN [RIGHT OUTER JOIN]
| hash predicates: isqbiuar = isqbiuar
| other predicates: row_number() <= 1, row_number() >= 1
| row-size=14B cardinality=2
Tests:
* Add plan tests in analytic-rank-pushdown.test
* Add e2e tests in analytic-fns.test
Change-Id: If6c209b2a64bad37d893ba8b520342bf1f9a7513
Reviewed-on: http://gerrit.cloudera.org:8080/19768
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com >
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com >
2023-06-06 19:06:24 +00:00
2019-08-13 18:55:09 +00:00
2022-12-14 22:37:14 +00:00
2022-12-14 22:37:14 +00:00
2019-07-27 13:45:51 +00:00
2022-12-14 22:37:14 +00:00
2020-08-12 17:45:50 +00:00
2020-05-18 16:51:42 +00:00
2019-07-27 13:45:51 +00:00
2019-11-07 14:30:47 +00:00
2019-07-03 20:46:45 +00:00
2020-05-20 21:00:44 +00:00
2020-05-20 21:00:44 +00:00
2020-05-20 21:00:44 +00:00
2022-12-14 22:37:14 +00:00
2022-12-14 22:37:14 +00:00
2022-09-28 05:46:03 +00:00
2021-03-12 14:23:04 +00:00
2021-03-12 14:23:04 +00:00
2016-09-13 21:57:36 +00:00
2022-06-19 04:35:04 +00:00
2019-10-16 02:04:12 +00:00
2020-07-08 22:59:57 +00:00
2021-12-16 11:49:15 +00:00
2018-09-29 11:59:03 +00:00
2022-12-14 22:37:14 +00:00
2020-05-12 23:29:04 +00:00
2022-12-14 22:37:14 +00:00
2022-04-07 16:07:56 +00:00
2018-08-08 20:51:45 +00:00
2023-06-06 19:06:24 +00:00
2021-10-23 00:08:14 +00:00
2022-02-23 20:09:10 +00:00
2019-09-27 17:18:35 +00:00
2019-09-27 17:18:35 +00:00
2018-05-15 22:23:14 +00:00
2022-08-25 15:12:38 +00:00
2021-02-18 14:31:51 +00:00
2021-04-30 23:36:31 +00:00
2018-10-09 00:45:10 +00:00
2018-04-11 02:21:48 +00:00
2018-11-17 01:48:05 +00:00
2019-08-13 18:55:09 +00:00
2022-12-07 21:57:46 +00:00
2021-03-02 04:58:51 +00:00
2022-12-14 22:37:14 +00:00
2022-12-14 22:37:14 +00:00
2021-04-02 21:50:17 +00:00
2019-08-06 16:38:07 +00:00
2022-12-14 22:37:14 +00:00
2022-12-14 22:37:14 +00:00
2022-12-14 22:37:14 +00:00
2018-12-08 00:21:55 +00:00
2022-12-14 22:37:14 +00:00
2021-09-14 21:21:47 +00:00
2022-12-14 22:37:14 +00:00
2022-12-14 22:37:14 +00:00
2021-07-06 18:35:30 +00:00
2020-06-24 03:47:18 +00:00
2021-05-07 14:11:08 +00:00
2020-06-24 03:47:18 +00:00
2022-12-14 22:37:14 +00:00
2022-11-25 05:30:50 +00:00
2022-11-11 17:53:58 +00:00
2021-04-20 22:21:32 +00:00
2021-09-15 13:58:23 +00:00
2021-09-15 13:58:23 +00:00
2021-09-15 13:58:23 +00:00
2021-09-15 13:58:23 +00:00
2020-06-24 16:51:49 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2020-09-28 14:02:35 +00:00
2020-07-08 19:32:15 +00:00
2020-08-13 18:58:23 +00:00
2018-01-25 04:33:11 +00:00
2019-01-14 09:07:03 +00:00
2019-04-23 13:33:57 +00:00
2021-03-23 22:52:38 +00:00
2018-10-09 00:45:10 +00:00
2022-10-17 21:59:57 +00:00
2022-03-04 16:49:22 +00:00
2018-04-11 02:21:48 +00:00
2018-04-11 02:21:48 +00:00
2021-07-06 18:35:30 +00:00
2023-01-02 01:12:53 +00:00
2020-04-02 12:01:41 +00:00
2020-06-05 17:43:32 +00:00
2019-05-15 22:34:28 +00:00
2018-04-12 22:01:35 +00:00
2018-02-02 01:13:08 +00:00
2020-10-21 05:20:33 +00:00
2020-07-31 17:23:45 +00:00
2018-02-15 10:55:55 +00:00
2018-02-15 10:55:55 +00:00
2021-06-14 17:13:25 +00:00
2022-12-06 21:18:33 +00:00
2022-12-06 21:18:33 +00:00
2022-12-06 21:18:33 +00:00
2022-12-06 21:18:33 +00:00
2023-03-21 20:01:14 +00:00
2020-02-22 02:02:56 +00:00
2020-08-12 17:45:50 +00:00
2020-09-30 21:01:54 +00:00
2021-03-18 19:35:58 +00:00
2022-12-14 22:37:14 +00:00
2018-12-11 02:47:16 +00:00
2023-03-06 13:45:56 +00:00
2023-03-08 19:54:38 +00:00
2023-02-07 20:18:47 +00:00
2022-12-14 22:37:14 +00:00
2020-07-14 03:13:18 +00:00
2019-02-07 23:05:30 +00:00
2020-05-12 23:29:04 +00:00
2020-05-12 23:29:04 +00:00
2022-07-18 19:02:34 +00:00
2022-08-19 13:55:42 +00:00
2019-04-23 13:33:57 +00:00
2020-05-12 23:29:04 +00:00
2017-10-10 01:30:33 +00:00
2017-09-21 17:38:08 +00:00
2022-12-14 22:37:14 +00:00
2022-12-14 22:37:14 +00:00
2018-07-03 23:49:44 +00:00
2019-06-05 14:23:23 +00:00
2021-03-02 04:58:51 +00:00
2017-05-09 01:36:46 +00:00
2016-02-19 00:03:15 -08:00
2022-12-14 22:37:14 +00:00
2020-01-17 18:54:33 +00:00
2019-09-27 17:18:35 +00:00
2020-02-14 04:26:16 +00:00
2020-02-11 15:08:39 +00:00
2023-03-02 16:51:12 +00:00
2022-12-16 17:37:35 +00:00
2022-12-14 22:37:14 +00:00
2023-05-09 00:05:36 +00:00
2022-12-14 22:37:14 +00:00
2022-09-28 22:35:48 +00:00
2023-02-03 17:00:44 +00:00
2022-12-14 22:37:14 +00:00
2022-01-23 20:55:00 +00:00
2023-05-09 00:05:36 +00:00
2022-12-14 22:37:14 +00:00
2023-05-09 00:05:36 +00:00
2023-01-27 12:35:52 +00:00
2023-03-29 20:53:18 +00:00
2022-07-14 13:06:04 +00:00
2022-06-10 13:01:28 +00:00
2021-11-04 17:55:21 +00:00
2022-12-16 17:37:35 +00:00
2022-12-14 22:37:14 +00:00
2023-03-27 21:42:49 +00:00
2022-09-28 22:35:48 +00:00
2021-05-20 19:19:50 +00:00
2022-08-01 13:36:51 +00:00
2022-11-02 22:14:47 +00:00
2022-12-14 22:37:14 +00:00
2023-05-09 00:05:36 +00:00
2023-05-09 00:05:36 +00:00
2020-09-09 13:26:42 +00:00
2023-03-22 16:44:05 +00:00
2023-03-09 22:00:08 +00:00
2020-11-23 21:47:55 +00:00
2022-09-26 15:49:22 +00:00
2022-09-26 15:49:22 +00:00
2021-07-15 15:15:07 +00:00
2023-05-09 00:05:36 +00:00
2023-05-09 00:05:36 +00:00
2023-05-09 00:05:36 +00:00
2023-05-09 00:05:36 +00:00
2022-11-10 00:51:50 +00:00
2022-12-14 22:37:14 +00:00
2022-12-14 22:37:14 +00:00
2022-11-22 15:21:11 +00:00
2018-08-09 19:18:08 +00:00
2021-04-20 22:21:32 +00:00
2022-06-04 03:36:11 +00:00
2020-05-11 19:32:08 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2022-04-04 20:28:07 +00:00
2019-06-05 11:15:04 +00:00
2020-12-11 19:51:28 +00:00
2020-12-11 19:51:28 +00:00
2020-05-11 19:32:08 +00:00
2018-11-06 00:00:12 +00:00
2020-12-11 19:51:28 +00:00
2020-07-31 17:23:45 +00:00
2016-02-19 00:03:15 -08:00
2023-03-06 13:45:56 +00:00
2023-03-08 19:54:38 +00:00
2019-10-22 04:21:51 +00:00
2022-02-04 17:55:47 +00:00
2023-03-02 16:51:12 +00:00
2023-02-20 16:38:16 +00:00
2023-05-02 02:47:46 +00:00
2023-02-04 07:34:56 +00:00
2023-02-04 07:34:56 +00:00
2023-02-04 07:34:56 +00:00
2018-05-11 22:41:49 +00:00
2023-02-04 07:34:56 +00:00
2018-04-27 21:55:11 +00:00
2023-02-04 07:34:56 +00:00
2023-02-04 07:34:56 +00:00
2023-02-04 07:34:56 +00:00
2023-02-04 07:34:56 +00:00
2017-05-11 20:55:51 +00:00
2017-05-11 20:55:51 +00:00
2018-09-26 06:51:56 +00:00
2023-02-04 07:34:56 +00:00
2019-10-21 20:23:56 +00:00
2017-07-21 21:49:04 +00:00
2020-05-13 22:01:34 +00:00
2018-04-11 02:21:48 +00:00
2021-04-20 22:21:32 +00:00
2020-10-08 22:49:59 +00:00
2023-03-08 19:54:38 +00:00
2022-02-03 09:13:21 +00:00
2023-03-08 19:54:38 +00:00
2020-12-11 19:51:28 +00:00
2018-04-11 02:21:48 +00:00
2020-02-22 02:02:56 +00:00
2023-03-07 13:22:33 +00:00
2020-06-09 16:26:23 +00:00
2019-03-08 04:39:08 +00:00
2021-11-03 03:26:27 +00:00
2022-08-25 16:24:41 +00:00
2023-05-18 09:56:55 +00:00
2023-05-06 22:55:05 +00:00
2022-08-12 19:21:55 +00:00
2016-11-03 11:59:07 +00:00
2017-03-17 19:33:31 +00:00
2016-12-02 01:46:55 +00:00
2023-05-03 03:16:29 +00:00
2016-12-02 01:46:55 +00:00
2021-11-30 20:13:18 +00:00
2019-09-25 16:58:14 +00:00
2022-12-14 22:37:14 +00:00
2023-05-19 22:11:03 +00:00
2023-05-19 22:11:03 +00:00
2023-04-14 13:46:59 +00:00
2018-01-05 20:44:21 +00:00
2020-10-21 20:30:05 +00:00
2017-11-22 22:00:16 +00:00
2023-01-18 17:46:54 +00:00
2022-05-18 23:59:58 +00:00
2022-08-19 13:55:42 +00:00
2016-04-01 05:06:38 +00:00
2018-11-17 01:48:05 +00:00
2022-11-15 09:24:03 +00:00
2022-04-13 12:31:36 +00:00
2019-03-08 04:39:08 +00:00
2020-04-16 15:45:49 +00:00
2019-03-08 04:39:08 +00:00
2019-03-08 04:39:08 +00:00
2022-05-18 23:59:58 +00:00
2021-05-07 14:11:08 +00:00
2022-08-09 06:45:52 +00:00
2020-01-17 18:54:33 +00:00
2019-10-04 18:36:22 +00:00
2018-10-01 13:20:40 +00:00
2018-10-01 13:20:40 +00:00
2021-10-28 20:09:46 +00:00
2022-04-04 05:54:58 +00:00
2023-04-22 23:24:10 +00:00
2017-03-03 01:43:42 +00:00
2021-08-21 14:46:51 +00:00
2021-08-21 14:46:51 +00:00
2022-02-04 21:23:55 +00:00
2022-03-04 16:49:22 +00:00
2022-06-22 17:55:07 +00:00
2019-03-14 22:43:50 +00:00
2018-03-08 04:48:36 +00:00
2018-01-17 04:18:24 +00:00
2021-06-03 06:32:45 +00:00
2021-06-03 06:32:45 +00:00
2022-03-04 16:49:22 +00:00
2018-09-10 16:09:41 +00:00
2018-09-10 16:09:41 +00:00
2016-06-20 15:37:18 -07:00
2016-06-20 15:37:18 -07:00
2020-06-18 21:00:27 +00:00
2021-07-14 12:51:09 +00:00
2021-07-14 12:51:09 +00:00
2018-08-22 18:08:20 +00:00
2017-05-09 15:47:21 +00:00
2022-01-23 20:55:00 +00:00
2019-08-24 00:32:28 +00:00
2019-05-07 00:36:56 +00:00
2021-01-14 19:34:38 +00:00
2018-02-22 00:57:46 +00:00
2022-07-11 19:25:02 +00:00
2021-10-28 20:09:46 +00:00
2018-11-27 02:01:12 +00:00
2019-05-10 11:46:38 +00:00
2019-05-10 11:46:38 +00:00
2022-08-19 19:11:58 +00:00
2019-05-10 11:46:38 +00:00
2022-04-02 03:26:28 +00:00
2020-12-10 19:01:08 +00:00
2021-01-05 23:30:35 +00:00
2018-03-22 02:47:33 +00:00
2023-05-06 22:55:05 +00:00
2020-05-21 18:40:20 +00:00
2018-08-23 15:55:53 +00:00
2023-05-12 18:31:03 +00:00
2016-10-14 05:41:22 +00:00
2020-08-25 15:42:01 +00:00
2022-09-28 22:35:48 +00:00
2022-12-14 22:37:14 +00:00
2019-05-10 12:06:01 +00:00
2020-05-20 23:03:23 +00:00
2020-05-20 23:03:23 +00:00
2023-05-06 22:55:05 +00:00
2022-07-22 04:23:30 +00:00
2023-05-18 09:56:55 +00:00
2023-01-17 02:38:27 +00:00
2018-08-15 04:15:46 +00:00
2022-12-03 00:00:37 +00:00
2020-12-22 06:10:39 +00:00
2017-12-13 10:04:40 +00:00
2023-03-05 22:41:11 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2021-01-15 13:01:53 +00:00
2022-12-06 03:55:42 +00:00
2023-04-14 13:46:59 +00:00
2023-04-14 13:46:59 +00:00
2023-02-01 04:24:00 +00:00
2022-12-06 03:55:42 +00:00
2019-08-06 02:35:54 +00:00
2023-05-06 22:55:05 +00:00
2017-11-17 21:33:51 +00:00
2023-05-06 22:55:05 +00:00
2021-03-02 04:58:51 +00:00
2020-04-14 15:42:13 +00:00
2022-02-09 21:20:23 +00:00
2018-11-17 01:48:05 +00:00
2023-05-06 22:55:05 +00:00
2021-03-14 03:35:40 +00:00
2020-07-08 19:32:15 +00:00
2019-01-09 05:26:50 +00:00
2016-08-31 03:12:30 +00:00
2022-11-11 22:02:16 +00:00
2022-11-21 18:51:30 +00:00
2020-04-02 12:01:41 +00:00
2019-09-26 18:35:06 +00:00
2022-11-25 05:30:50 +00:00
2016-09-29 21:14:13 +00:00
2022-12-14 22:37:14 +00:00
2022-12-14 22:37:14 +00:00
2017-08-05 01:03:02 +00:00
2017-08-05 01:03:02 +00:00
2018-04-28 23:41:39 +00:00
2018-02-02 01:13:08 +00:00
2023-05-18 09:56:55 +00:00
2020-10-21 05:20:33 +00:00
2021-01-06 04:39:56 +00:00
2021-03-02 04:58:51 +00:00
2021-08-25 20:05:47 +00:00
2022-10-17 21:59:57 +00:00
2020-04-24 20:56:58 +00:00
2018-04-28 23:41:39 +00:00
2020-08-25 16:11:20 +00:00
2017-08-29 23:01:10 +00:00
2021-03-02 04:58:51 +00:00
2019-01-22 01:20:31 +00:00
2022-02-24 07:06:50 +00:00
2022-12-14 22:37:14 +00:00
2017-03-03 01:43:42 +00:00
2017-03-03 01:43:42 +00:00
2018-11-01 21:27:13 +00:00
2018-11-01 21:27:13 +00:00
2019-08-22 05:03:03 +00:00
2023-03-07 13:22:33 +00:00
2023-05-24 01:18:56 +00:00
2020-08-25 23:08:40 +00:00
2020-10-21 05:20:33 +00:00
2021-04-20 22:21:32 +00:00
2018-04-11 02:21:48 +00:00
2016-02-28 21:31:37 -08:00
2019-09-27 01:54:39 +00:00
2023-05-18 09:56:55 +00:00
2021-04-20 22:21:32 +00:00
2022-12-14 22:37:14 +00:00
2016-11-09 03:27:12 +00:00
2019-04-23 13:33:57 +00:00
2016-11-09 03:27:12 +00:00
2018-04-11 02:21:48 +00:00
2016-12-08 04:53:53 +00:00
2016-12-08 04:53:53 +00:00
2016-11-09 03:27:12 +00:00
2018-11-20 23:49:11 +00:00
2022-08-19 13:55:42 +00:00
2022-04-13 12:31:36 +00:00
2021-12-09 23:17:20 +00:00
2020-06-15 23:42:12 +00:00
2018-07-03 20:34:27 +00:00
2021-05-14 13:58:04 +00:00
2022-10-11 20:30:50 +00:00
2021-08-15 20:40:39 +00:00
2019-05-24 15:08:19 +00:00
2019-05-24 15:08:19 +00:00
2020-07-14 03:13:18 +00:00
2022-09-27 13:27:41 +00:00
2022-04-14 11:56:20 +00:00
2022-11-15 09:24:03 +00:00
2022-12-08 23:07:08 +00:00
2022-06-08 13:02:52 +00:00
2022-06-08 13:02:52 +00:00
2022-06-08 13:02:52 +00:00
2020-10-21 05:20:33 +00:00
2022-04-05 07:56:39 +00:00
2022-02-17 18:51:06 +00:00
2022-11-15 09:24:03 +00:00
88275f327f