jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-19 18:12:08 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-13250: [DOCS] Document ENABLED_RUNTIME_FILTER_TYPES query option

Browse Source 
This patch documents the ENABLED_RUNTIME_FILTER_TYPES query option based
on the respective code comments in ImpalaService.thrift and
query-options.cc.

Change-Id: Ib7a34782bed6f812fedf717d8a076e2706f0bba9
Reviewed-on: http://gerrit.cloudera.org:8080/21645
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Reviewed-by: Quanlong Huang <huangquanlong@gmail.com>
This commit is contained in:
Fang-Yu Rao
2024-08-05 16:44:16 -07:00
committed by Quanlong Huang
parent ad868b9947
commit 13a3d19a2c
4 changed files with 109 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/impala.ditamap
View File

@@ -241,6 +241,7 @@ under the License.
<topicref href="topics/impala_request_pool.xml"/>
<topicref href="topics/impala_resource_trace_ratio.xml"/>
<topicref rev="4.0.0" href="topics/impala_retry_failed_queries.xml"/>
<topicref rev="4.0.0" href="topics/impala_enabled_runtime_filter_types.xml"/>
<topicref rev="2.5.0" href="topics/impala_runtime_bloom_filter_size.xml"/>
<topicref rev="2.6.0" href="topics/impala_runtime_filter_max_size.xml"/>
<topicref rev="2.6.0" href="topics/impala_runtime_filter_min_size.xml"/>

1
docs/shared/ImpalaVariables.xml
View File

@@ -42,6 +42,7 @@ under the License.
The docs included with a distro can refer to the distro release number by
editing the values here.
<ul>
<li><ph id="impala40">Impala 4.0</ph></li>
<li><ph id="impala34">Impala 3.4</ph></li>
<li><ph id="impala33">Impala 3.3</ph></li>
<li><ph id="impala32">Impala 3.2</ph></li>

19
docs/shared/impala_common.xml
View File

@@ -1524,7 +1524,20 @@ alter table partitioned_data set tblproperties ('numRows'='1030000', 'STATS_GENE
<xref href="https://asciinema.org/a/1rv7qippo0fe7h5k1b6k4nexk" scope="external" format="html">this
animated demo</xref>.
</p>
<p id="comma_separated_values_blurb">
Impala backend expects comma separated values to be in quotes when executing the
<codeph>SET</codeph> statement.
This is usually the case when running SET statement like
<codeph>SET ENABLED_RUNTIME_FILTER_TYPES="value1,value2"</codeph> using a JDBC
driver. When using Impala-shell client, the <codeph>SET</codeph> statement is not
executed immediately but query options are updated in the client and applied as
part of the following statement, so no quotes are required for Impala-shell. That
is, we use SET statement like
<codeph>SET ENABLED_RUNTIME_FILTER_TYPES=value1,value2</codeph> when
submitting the query to Impala backend via Impala-shell client.
</p>
<p rev="2.5.0" id="runtime_filter_mode_blurb">
Because the runtime filtering feature is
enabled by default only for local processing, the other filtering-related query options have
@@ -3419,6 +3432,10 @@ flight_num: INT32 SNAPPY DO:83456393 FPO:83488603 SZ:10216514/11474301
needed to represent each value.
</p>
<p rev="4.0.0" id="added_in_400">
<b>Added in:</b> <keyword keyref="impala40"/>
</p>
<p rev="3.0" id="added_in_30">
<b>Added in:</b> <keyword keyref="impala30_full"/>
</p>

89
docs/topics/impala_enabled_runtime_filter_types.xml Normal file
View File

@@ -0,0 +1,89 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!DOCTYPE concept PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">
<concept id="enabled_runtime_filter_types" rev="4.0.0">
<title>ENABLED_RUNTIME_FILTER_TYPES Query Option (<keyword keyref="impala40"/> or higher only)</title>
<titlealts audience="PDF"><navtitle>ENABLED RUNTIME FILTER TYPES</navtitle></titlealts>
<prolog>
<metadata>
<data name="Category" value="Impala"/>
<data name="Category" value="Impala Query Options"/>
<data name="Category" value="Performance"/>
<data name="Category" value="Developers"/>
<data name="Category" value="Data Analysts"/>
</metadata>
</prolog>
<conbody>
<p rev="4.0.0">
<indexterm audience="hidden">ENABLED_RUNTIME_FILTER_TYPES query option</indexterm>
The <codeph>ENABLED_RUNTIME_FILTER_TYPES</codeph> query option
sets enabled runtime filter types to be applied to scanners.
This option only applies to HDFS scan nodes and Kudu scan nodes.
The following types are supported.
Specify the enabled types by a comma-separated list of the following values
or enable all types by "<codeph>ALL</codeph>".
<ul>
<li>
<codeph>BLOOM</codeph>
</li>
<li>
<codeph>MIN_MAX</codeph>
</li>
<li>
<codeph>IN_LIST</codeph>
</li>
</ul>
</p>
<p>
<b>Default:</b> <codeph>"BLOOM,MIN_MAX"</codeph>
</p>
<p conref="../shared/impala_common.xml#common/type_string"/>
<p conref="../shared/impala_common.xml#common/added_in_400"/>
<p conref="../shared/impala_common.xml#common/usage_notes_blurb"/>
<p conref="../shared/impala_common.xml#common/comma_separated_values_blurb"/>
<p>
Depending on the scan node type, Planner can schedule compatible runtime filter
types as follows.
<ul>
<li>Kudu scan: <codeph>BLOOM</codeph>, <codeph>MIN_MAX</codeph></li>
<li>
HDFS scan on Parquet files: <codeph>BLOOM</codeph>, <codeph>MIN_MAX</codeph>
</li>
<li>HDFS scan on ORC files: <codeph>BLOOM</codeph>, <codeph>IN_LIST</codeph></li>
<li>HDFS scan on other kinds of files: <codeph>BLOOM</codeph></li>
</ul>
</p>
<p conref="../shared/impala_common.xml#common/related_info"/>
<p>
<xref href="impala_runtime_filtering.xml"/>,
<xref href="impala_runtime_filter_mode.xml#runtime_filter_mode"/>
</p>
</conbody>
</concept>