jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-19 09:58:28 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-3401: [DOCS] Physically remove Cloudera Manager info

Browse Source 
Followup from Laurel's code reviews, to physically
remove references to Cloudera Manager that were hidden.

Remove a few stray instances of Cloudera Manager that I found
still remaining in the source.

Fix up trailing spaces introduced during earlier
Cloudera Manager-related edits.

Also remove stray 'Cloudera' references, or stale/commented
Cloudera-specific info, noticed near other spots being edited.

Change-Id: Ifc4a84527ae42c39b3717190b6cf669e17fff04b
Reviewed-on: http://gerrit.cloudera.org:8080/6325
Reviewed-by: Ambreen Kazi <ambreen.kazi@cloudera.com>
Reviewed-by: John Russell <jrussell@cloudera.com>
Tested-by: Impala Public Jenkins
This commit is contained in:
John Russell
2017-03-08 16:31:07 -08:00
committed by Impala Public Jenkins
parent 1bda28c485
commit 0467b0d54a
30 changed files with 123 additions and 839 deletions
Download Patch File Download Diff File Expand all files Collapse all files

102
docs/shared/impala_common.xml
View File

@@ -574,14 +574,6 @@ under the License.
in the <cmdname>impalad</cmdname> and <cmdname>catalogd</cmdname> configuration settings.
</p>
</li>
<li audience="hidden">
<p>
For clusters managed by Cloudera Manager, select the
<uicontrol>Use HDFS Rules to Map Kerberos Principals to Short Names</uicontrol>
checkbox to enable the service-wide <codeph>load_auth_to_local_rules</codeph> configuration setting.
Then restart the Impala service.
</p>
</li>
</ul>
</p>
@@ -616,46 +608,22 @@ under the License.
indicating <q>Access Denied</q> .
<!--
[1]
Impala: Impala Daemon -> Advanced -> Impala Daemon Logging Safety Valve
Hive: Hive Server 2 -> Advanced -> HiveServer2 Logging Safety Valve
Impala: Impala Daemon -> Advanced -> Impala Daemon Logging Safety Valve
Hive: Hive Server 2 -> Advanced -> HiveServer2 Logging Safety Valve
Search: Solr Server -> Advanced -> HiveServer2 Logging Safety Valve
-->
</p>
</section>
<section id="cm">
<section id="restrictions">
<title>Cloudera Manager Terminology</title>
<title>Restrictions and Limitations</title>
<p>
Especially during the transition from CM 4 to CM 5, we'll use some stock phraseology to talk about fields
and such. Also there are some task steps etc. to conref under the Impala Service page that are easier
to keep track of here instead of in cm_common_elements.xml. (Although as part of Apache work, anything
CM might naturally move out of this file.)
</p>
<p>
<ph id="safety_valve"> In Cloudera Manager 4, these fields are labelled <uicontrol>Safety
Valve</uicontrol>; in Cloudera Manager 5, they are called <uicontrol>Advanced Configuration
Snippet</uicontrol>. </ph>
</p>
<ul>
<li id="go_impala_service">Go to the Impala service.</li>
<li id="restart_impala_service">Restart the Impala service.</li>
</ul>
</section>
<section id="citi">
<title>Items from the Citibank Escalation Spreadsheet</title>
<p>
Paragraphs with IDs are intended to be reused both in the FAQ and the User's Guide. They refer to feature
requests or misunderstandings encountered by Citibank, captured in the escalation spreadsheet here:
<xref href="https://docs.google.com/a/cloudera.com/spreadsheet/ccc?key=0AplfwQJKyyTWdFdhY0E5WHVwNXZSTG9sMEZwQy1QZ1E&amp;usp=drive_web#gid=0" scope="external" format="html"/>.
Potential misunderstandings for people familiar with other
database systems. Currently not referenced anywhere, because
they were only conref'ed from the FAQ page.
</p>
<p id="string_concatenation">
@@ -835,7 +803,7 @@ set PARQUET_FALLBACK_SCHEMA_RESOLUTION=position;
PARQUET_FALLBACK_SCHEMA_RESOLUTION set to position
select * from t2;
WARNINGS:
WARNINGS:
File 'schema_evolution.db/t2/45331705_data.0.parq'
has an incompatible Parquet schema for column 'schema_evolution.t2.c4'.
Column type: TIMESTAMP, Parquet schema: optional int32 c1 [i:0 d:1 r:0]
@@ -868,7 +836,7 @@ select * from t2;
</note>
<p rev="IMPALA-3732" id="avro_2gb_strings">
The Avro specification allows string values up to 2**64 bytes in length.
The Avro specification allows string values up to 2**64 bytes in length.
Impala queries for Avro tables use 32-bit integers to hold string lengths.
In <keyword keyref="impala25_full"/> and higher, Impala truncates <codeph>CHAR</codeph>
and <codeph>VARCHAR</codeph> values in Avro tables to (2**31)-1 bytes.
@@ -971,7 +939,7 @@ alter table partitioned_data set tblproperties ('numRows'='1030000', 'STATS_GENE
<p id="impala_shell_progress_reports_shell_only_caveat">
Because the <codeph>LIVE_PROGRESS</codeph> and <codeph>LIVE_SUMMARY</codeph> query options
are available only within the <cmdname>impala-shell</cmdname> interpreter:
are available only within the <cmdname>impala-shell</cmdname> interpreter:
<ul>
<li>
<p>
@@ -995,7 +963,7 @@ alter table partitioned_data set tblproperties ('numRows'='1030000', 'STATS_GENE
Likewise, the <cmdname>impala-shell</cmdname> command relies on
some information only available in <keyword keyref="impala23_full"/> and higher
to prepare live progress reports and query summaries. The
<codeph>LIVE_PROGRESS</codeph> and <codeph>LIVE_SUMMARY</codeph>
<codeph>LIVE_PROGRESS</codeph> and <codeph>LIVE_SUMMARY</codeph>
query options have no effect when <cmdname>impala-shell</cmdname> connects
to a cluster running an older version of Impala.
</p>
@@ -1035,7 +1003,7 @@ use default;
-- Before dropping a database, first drop all the tables inside it,
<ph rev="2.3.0">-- or in <keyword keyref="impala23_full"/> and higher use the CASCADE clause.</ph>
drop database temp;
ERROR: ImpalaRuntimeException: Error making 'dropDatabase' RPC to Hive Metastore:
ERROR: ImpalaRuntimeException: Error making 'dropDatabase' RPC to Hive Metastore:
CAUSED BY: InvalidOperationException: Database temp is not empty
show tables in temp;
+------+
@@ -1462,14 +1430,14 @@ explain select s from yy2 where year in (select year from yy where year between
operation finishes. (Multiple concurrent queries can perform operations that use the <q>spill to disk</q>
technique, without any name conflicts for these temporary files.) You can specify a different location by
starting the <cmdname>impalad</cmdname> daemon with the
<codeph>--scratch_dirs="<varname>path_to_directory</varname>"</codeph> configuration option.
You can specify a single directory, or a comma-separated list of directories. The scratch directories must
be on the local filesystem, not in HDFS. You might specify different directory paths for different hosts,
<codeph>--scratch_dirs="<varname>path_to_directory</varname>"</codeph> configuration option.
You can specify a single directory, or a comma-separated list of directories. The scratch directories must
be on the local filesystem, not in HDFS. You might specify different directory paths for different hosts,
depending on the capacity and speed
of the available storage devices. In <keyword keyref="impala23_full"/> or higher, Impala successfully starts (with a warning
Impala successfully starts (with a warning written to the log) if it cannot create or read and write files
in one of the scratch directories. If there is less than 1 GB free on the filesystem where that directory resides,
Impala still runs, but writes a warning message to its log. If Impala encounters an error reading or writing
Impala successfully starts (with a warning written to the log) if it cannot create or read and write files
in one of the scratch directories. If there is less than 1 GB free on the filesystem where that directory resides,
Impala still runs, but writes a warning message to its log. If Impala encounters an error reading or writing
files in a scratch directory during a query, Impala logs the error and the query fails.
</p>
@@ -1642,7 +1610,7 @@ show functions in _impala_builtins like '*<varname>substring</varname>*';
</p>
<p rev="" id="hive_column_stats_caveat">
If you run the Hive statement <codeph>ANALYZE TABLE COMPUTE STATISTICS FOR COLUMNS</codeph>,
If you run the Hive statement <codeph>ANALYZE TABLE COMPUTE STATISTICS FOR COLUMNS</codeph>,
Impala can only use the resulting column statistics if the table is unpartitioned.
Impala cannot use Hive-generated column statistics for a partitioned table.
</p>
@@ -1727,7 +1695,7 @@ show functions in _impala_builtins like '*<varname>substring</varname>*';
following technique for queries involving a single table:
</p>
<codeblock xml:space="preserve">select v1.c1 result1, v2.c1 result2 from
(select count(distinct col1) as c1 from t1) v1
(select count(distinct col1) as c1 from t1) v1
cross join
(select count(distinct col2) as c1 from t1) v2;
</codeblock>
@@ -2211,7 +2179,7 @@ message schema {
$ parquet-tools meta sample.parq
creator: impala version 2.2.0-...
file schema: schema
file schema: schema
-------------------------------------------------------------------
year: OPTIONAL INT32 R:0 D:1
month: OPTIONAL INT32 R:0 D:1
@@ -2225,7 +2193,7 @@ carrier: OPTIONAL BINARY R:0 D:1
flight_num: OPTIONAL INT32 R:0 D:1
...
row group 1: RC:20636601 TS:265103674
row group 1: RC:20636601 TS:265103674
-------------------------------------------------------------------
year: INT32 SNAPPY DO:4 FPO:35 SZ:10103/49723/4.92 VC:20636601 ENC:PLAIN_DICTIONARY,RLE,PLAIN
month: INT32 SNAPPY DO:10147 FPO:10210 SZ:11380/35732/3.14 VC:20636601 ENC:PLAIN_DICTIONARY,RLE,PLAIN
@@ -2265,7 +2233,7 @@ flight_num: INT32 SNAPPY DO:83456393 FPO:83488603 SZ:10216514/11474301
<p id="impala_parquet_encodings_caveat">
Impala can query Parquet files that use the <codeph>PLAIN</codeph>, <codeph>PLAIN_DICTIONARY</codeph>,
<codeph>BIT_PACKED</codeph>, and <codeph>RLE</codeph> encodings.
<codeph>BIT_PACKED</codeph>, and <codeph>RLE</codeph> encodings.
Currently, Impala does not support <codeph>RLE_DICTIONARY</codeph> encoding.
When creating files outside of Impala for use by Impala, make sure to use one of the supported encodings.
In particular, for MapReduce jobs, <codeph>parquet.writer.version</codeph> must not be defined
@@ -3124,15 +3092,6 @@ select * from header_line limit 10;
worked on SSSE3-enabled processors.
</p>
<p id="logbuflevel_caveat">
Due to a change to the implementation of logging in Impala 1.1.1 and higher, currently you should change
the default setting for the <codeph>logbuflevel</codeph> property for the Impala service after installing
through Cloudera Manager. In Cloudera Manager, on the log settings page for the Impala service, change the
setting <uicontrol>Impala Daemon Log Buffer Level (logbuflevel)</uicontrol> from -1 to 0. You might change
this setting to a value higher than 0, if you prefer to reduce the I/O overhead for logging, at the expense
of possibly losing some lower-priority log messages in the event of a crash.
</p>
<p id="rhel5_kerberos">
On version 5 of Red Hat Enterprise Linux and comparable distributions, some additional setup is needed for
the <cmdname>impala-shell</cmdname> interpreter to connect to a Kerberos-enabled Impala cluster:
@@ -3198,7 +3157,7 @@ sudo pip-python install ssl</codeblock>
<section id="admin_conrefs">
<title>Administration</title>
<p id="statestored_catalogd_ha_blurb" rev="">
Most considerations for load balancing and high availability apply to the <cmdname>impalad</cmdname> daemon.
The <cmdname>statestored</cmdname> and <cmdname>catalogd</cmdname> daemons do not have special
@@ -3206,7 +3165,7 @@ sudo pip-python install ssl</codeblock>
If those daemons become unavailable due to an outage on a particular
host, you can stop the Impala service, delete the <uicontrol>Impala StateStore</uicontrol> and
<uicontrol>Impala Catalog Server</uicontrol> roles, add the roles on a different host, and restart the
Impala service.
Impala service.
</p>
<p id="hdfs_caching_encryption_caveat" rev="IMPALA-3679">
@@ -3412,15 +3371,6 @@ sudo pip-python install ssl</codeblock>
JAVA_TOOL_OPTIONS="-Xmx8g"
</codeblock>
</li>
<li audience="hidden">
<p rev="OPSAPS-26483">
On systems managed by Cloudera Manager, include this value in the configuration field
<uicontrol>Java Heap Size of Catalog Server in Bytes</uicontrol> (Cloudera Manager 5.7 and higher), or
<uicontrol>Impala Catalog Server Environment Advanced Configuration Snippet (Safety Valve)</uicontrol>
(prior to Cloudera Manager 5.7).
Then restart the Impala service.
</p>
</li>
<li>
<p>
On systems not using cluster management software, put this environment variable setting into the
@@ -3500,7 +3450,7 @@ sudo pip-python install ssl</codeblock>
such as adding or dropping a column, by a mechanism other than
Impala.
</p>
<p id="kudu_internal_external_tables">
The distinction between internal and external tables has some special
details for Kudu tables. Tables created entirely through Impala are

69
docs/topics/impala_admission.xml
View File

@@ -213,19 +213,13 @@ under the License.
The Impala admission control feature uses the same configuration mechanism as the YARN resource manager to map users to
pools and authenticate them.
</p>
<p>
Although the Impala admission control feature uses a <codeph>fair-scheduler.xml</codeph> configuration file
behind the scenes, this file does not depend on which scheduler is used for YARN. You still use this file
even when YARN is using the capacity scheduler.
</p>
<p rev="DOCS-648" audience="hidden">
Although the Impala admission control feature uses a <codeph>fair-scheduler.xml</codeph> configuration file
behind the scenes, this file does not depend on which scheduler is used for YARN. You still use this file,
and Cloudera Manager can generate it for you, even when YARN is using the capacity scheduler.
</p>
</conbody>
</concept>
@@ -400,31 +394,6 @@ under the License.
set of options) to the complex (multiple resource pools with different options, each pool handling queries
for a different set of users and groups).
</p>
<p audience="hidden">
The configuration options for admission control range from the simple (a single resource pool with a single
set of options) to the complex (multiple resource pools with different options, each pool handling queries
for a different set of users and groups). <ph rev="upstream">Cloudera</ph> recommends configuring the settings through the Cloudera Manager user
interface.
<!--
, or on a system without Cloudera Manager by editing configuration files or through startup
options to the <cmdname>impalad</cmdname> daemon.
-->
</p>
<!-- To do: reconcile the similar notes in impala_admission.xml and admin_impala_admission_control.xml
and make into a conref in both places. -->
<note type="important" audience="hidden">
Although the following options are still present in the Cloudera Manager interface under the
<uicontrol>Admission Control</uicontrol> configuration settings dialog,
<ph rev="upstream">Cloudera</ph> recommends you not use them in <keyword keyref="impala25_full"/> and higher.
These settings only apply if you enable admission control but leave dynamic resource pools disabled.
In <keyword keyref="impala25_full"/> and higher, prefer to set up dynamic resource pools and
customize the settings for each pool, as described in
<ph audience="integrated"><xref href="cm_mc_resource_pools.xml#concept_xkk_l1d_wr/section_p15_mhn_2v"/> and <xref href="cm_mc_resource_pools.xml#concept_xkk_l1d_wr/section_gph_tnk_lm"/></ph>
<xref audience="standalone" href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_resource_pools.html" scope="external" format="html"/>.
</note>
<section id="admission_flags">
@@ -689,7 +658,7 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph
<title>Example of Admission Control Configuration</title>
<conbody>
<p> Here are sample <filepath>fair-scheduler.xml</filepath> and
<filepath>llama-site.xml</filepath> files that define resource pools
<codeph>root.default</codeph>, <codeph>root.development</codeph>, and
@@ -697,28 +666,28 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph
deployment they might contain other settings for use with various aspects of the YARN
component. The settings shown here are the significant ones for the Impala admission
control feature. </p>
<p>
<b>fair-scheduler.xml:</b>
</p>
<p>
Although Impala does not use the <codeph>vcores</codeph> value, you must still specify it to satisfy
YARN requirements for the file contents.
</p>
<p>
Each <codeph>&lt;aclSubmitApps&gt;</codeph> tag (other than the one for <codeph>root</codeph>) contains
a comma-separated list of users, then a space, then a comma-separated list of groups; these are the
users and groups allowed to submit Impala statements to the corresponding resource pool.
</p>
<p>
If you leave the <codeph>&lt;aclSubmitApps&gt;</codeph> element empty for a pool, nobody can submit
directly to that pool; child pools can specify their own <codeph>&lt;aclSubmitApps&gt;</codeph> values
to authorize users and groups to submit to those pools.
</p>
<codeblock><![CDATA[<allocations>
<queue name="root">
@@ -743,11 +712,11 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph
</allocations>
]]>
</codeblock>
<p>
<b>llama-site.xml:</b>
</p>
<codeblock rev="2.5.0 IMPALA-2538"><![CDATA[
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
@@ -821,21 +790,6 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph
</configuration>
]]>
</codeblock>
<section id="section_fqn_qgb_rq" audience="hidden">
<title>Example Admission Control Configurations Using Cloudera Manager</title>
<p>
For full instructions about configuring dynamic resource pools through Cloudera Manager, see
<xref audience="integrated" href="cm_mc_resource_pools.xml#xd_583c10bfdbd326ba--43d5fd93-1410993f8c2--7ff2"/><xref audience="standalone" href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_resource_pools.html" scope="external" format="html"/>.
</p>
</section>
</conbody>
</concept>
@@ -865,11 +819,6 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph
usage for the query, so you can fine-tune the configuration for the memory limits of the resource pools.
</p>
<p audience="hidden">
Where practical, use Cloudera Manager to configure the admission control parameters. The Cloudera Manager
GUI is much simpler than editing the configuration files directly.
</p>
<p>
Remember that the limits imposed by admission control are <q>soft</q> limits.
The decentralized nature of this mechanism means that each Impala node makes its own decisions about whether

7
docs/topics/impala_auditing.xml
View File

@@ -53,11 +53,10 @@ under the License.
<li>
Decide how many queries will be represented in each log file. By default,
Impala starts a new log file every 5000 queries. To specify a different number, <ph
audience="standalone">include
Impala starts a new log file every 5000 queries. To specify a different number,
<ph audience="standalone">include
the option <codeph>-max_audit_event_log_file_size=<varname>number_of_queries</varname></codeph>
in the <cmdname>impalad</cmdname> startup options</ph>
<xref href="cn_iu_audit_log.xml#xd_583c10bfdbd326ba--6eed2fb8-14349d04bee--7d6f/section_v25_lmy_bn" audience="integrated">configure Impala Daemon logging in Cloudera Manager</xref>.
in the <cmdname>impalad</cmdname> startup options</ph>.
</li>
<li>

3
docs/topics/impala_authorization.xml
View File

@@ -826,8 +826,7 @@ sales = hdfs://ha-nn-uri/etc/access/sales.ini
<p>
To enable URIs in per-DB policy files, the Java configuration option <codeph>sentry.allow.uri.db.policyfile</codeph>
must be set to <codeph>true</codeph>.
For example: <!-- in the Cloudera Manager field <uicontrol>Impala Service Environment
Advanced Configuration Snippet (Safety Valve)</uicontrol>: -->
For example:
</p>
<codeblock>JAVA_TOOL_OPTIONS="-Dsentry.allow.uri.db.policyfile=true"

57
docs/topics/impala_breakpad.xml
View File

@@ -134,43 +134,14 @@ under the License.
<concept id="breakpad_minidump_logging">
<title>Detecting Crash Events</title>
<conbody>
<p>
You can see in the Impala log files when crash events occur that generate
minidump files. Because each restart begins a new log file, the <q>crashed</q> message
is always at or near the bottom of the log file. There might be another later message
You can see in the Impala log files when crash events occur that generate
minidump files. Because each restart begins a new log file, the <q>crashed</q> message
is always at or near the bottom of the log file. There might be another later message
if core dumps are also enabled.
</p>
<p audience="hidden">
You can see in the Impala log files or in the Cloudera Manager charts for Impala
when crash events occur that generate minidump files. Because each restart begins
a new log file, the <q>crashed</q> message is always at or near the bottom of the
log file. (There might be another later message if core dumps are also enabled.)
</p>
</conbody>
</concept>
<concept id="breakpad_support_process" rev="CDH-39818" audience="hidden">
<title>Using the Minidump Files for Problem Resolution</title>
<conbody>
<p>
Typically, you provide minidump files to <keyword keyref="support_org"/> as part of problem resolution,
in the same way that you might provide a core dump. The <uicontrol>Send Diagnostic Data</uicontrol>
under the <uicontrol>Support</uicontrol> menu in Cloudera Manager guides you through the
process of selecting a time period and volume of diagnostic data, then collects the data
from all hosts and transmits the relevant information for you.
</p>
<fig id="fig_pqw_gvx_pr">
<title>Send Diagnostic Data choice under Support menu</title>
<image href="../images/support_send_diagnostic_data.png" scalefit="yes" placement="break"/>
</fig>
<p>
You might get additional instructions from <keyword keyref="support_org"/> about collecting minidumps to better isolate a specific problem.
Because the information in the minidump files is limited to stack traces and register contents,
the possibility of including sensitive information is much lower than with core dump files.
If any sensitive information is included in the minidump, <keyword keyref="support_org"/> preserves the confidentiality of that information.
</p>
</conbody>
</concept>
@@ -182,22 +153,17 @@ under the License.
simulate a <codeph>SIGSEGV</codeph> crash for an <cmdname>impalad</cmdname>
process on a single DataNode, then examines the relevant log files and minidump file.
</p>
<p>
First, as root on a worker node, kill the <cmdname>impalad</cmdname> process with a
<codeph>SIGSEGV</codeph> error. The original process ID was 23114.
</p>
<p audience="hidden">
First, as root on a worker node, we kill the <cmdname>impalad</cmdname> process with a
<codeph>SIGSEGV</codeph> error. The original process ID was 23114. (Cloudera Manager
restarts the process with a new pid, as shown by the second <cmdname>ps</cmdname> command.)
</p>
<codeblock><![CDATA[
# ps ax | grep impalad
23114 ? Sl 0:18 /opt/cloudera/parcels/<parcel_version>/lib/impala/sbin-retail/impalad --flagfile=/var/run/cloudera-scm-agent/process/114-impala-IMPALAD/impala-conf/impalad_flags
31259 pts/0 S+ 0:00 grep impalad
#
#
# kill -11 23114
#
# ps ax | grep impalad
@@ -262,17 +228,12 @@ E0623 14:03:43.911002 23114 logging.cc:118] stderr will be logged to this file.
Wrote minidump to /var/log/impala-minidumps/impalad/0980da2d-a905-01e1-25ff883a-04ee027a.dmp
]]>
</codeblock>
<p>
The resulting minidump file is much smaller than the corresponding core file,
making it much easier to supply diagnostic information to <keyword keyref="support_org"/>.
</p>
<p audience="hidden">
The resulting minidump file is much smaller than the corresponding core file,
making it much easier to supply diagnostic information to <keyword keyref="support_org"/>.
The transmission process for the minidump files is automated through Cloudera Manager.
</p>
<codeblock><![CDATA[
# pwd
/var/log/impalad

8
docs/topics/impala_config_options.xml
View File

@@ -153,8 +153,8 @@ Starting Impala Catalog Server: [ OK ]</codeblock>
<li rev="1.2">
<p>
Catalog server address (including both the hostname and the port number). Update the
value of the <codeph>IMPALA_CATALOG_SERVICE_HOST</codeph> variable. Cloudera
recommends the catalog server be on the same host as the statestore. In that
value of the <codeph>IMPALA_CATALOG_SERVICE_HOST</codeph> variable. Where
practical, run the catalog server on the same host as the statestore. In that
recommended configuration, the <cmdname>impalad</cmdname> daemon cannot refer to the
catalog server using the loopback address. If the catalog service is hosted on a
machine with an IP address of 192.168.0.27, add the following line:
@@ -333,8 +333,8 @@ Starting Impala Catalog Server: [ OK ]</codeblock>
<li rev="1.2">
<p>
Catalog server address. Update the <codeph>IMPALA_CATALOG_SERVICE_HOST</codeph>
variable, including both the hostname and the port number in the value. Cloudera
recommends the catalog server be on the same host as the statestore. In that
variable, including both the hostname and the port number in the value. Where
practical, run the catalog server on the same host as the statestore. In that
recommended configuration, the <cmdname>impalad</cmdname> daemon cannot refer to
the catalog server using the loopback address. If the catalog service is hosted on
a machine with an IP address of 192.168.0.27, add the following line:

43
docs/topics/impala_config_performance.xml
View File

@@ -37,41 +37,16 @@ under the License.
installed using cluster management software, some of these configurations might be completed automatically; you must still
configure short-circuit reads manually. If you want to customize your environment, consider making the changes described in this topic.
</p>
<p id="p_twenty-four" audience="hidden">
This section describes the mandatory and recommended configuration settings for Impala. If Impala is
installed using Cloudera Manager, some of these configurations are completed automatically; you must still
configure short-circuit reads manually. If you installed Impala without Cloudera Manager, or if you want to
customize your environment, consider making the changes described in this topic.
</p>
<p audience="hidden">
<!-- Could conref this paragraph from ciiu_install.xml. -->
In some cases, depending on the level of Impala, CDH, and Cloudera Manager, you might need to add particular
component configuration details in one of the free-form fields on the Impala configuration pages within
Cloudera Manager. <ph conref="../shared/impala_common.xml#common/safety_valve"/>
</p>
<ul>
<li>
You must enable short-circuit reads, whether or not Impala was installed with cluster
management software. This setting goes in the Impala configuration settings, not the Hadoop-wide settings.
</li>
<li>
You must enable block location tracking, and you can optionally enable native checksumming for optimal performance.
</li>
<li audience="hidden">
If you installed Impala in an environment that is not managed by Cloudera Manager, you must enable block
location tracking, and you can optionally enable native checksumming for optimal performance.
</li>
<li audience="hidden">
If you deployed Impala using Cloudera Manager see
<xref href="impala_perf_testing.xml#performance_testing"/> to confirm proper configuration.
</li>
</ul>
<section id="section_fhq_wyv_ls">
@@ -81,19 +56,11 @@ under the License.
DataNodes, improving performance. This setting also minimizes the number
of additional copies of data. Short-circuit reads requires
<codeph>libhadoop.so</codeph>
<!-- This link went stale. Not obvious how to keep it in sync with whatever Hadoop CDH is using behind the scenes. So hide the link for now. -->
<!-- (the <xref href="http://hadoop.apache.org/docs/r0.19.1/native_libraries.html" scope="external" format="html">Hadoop Native Library</xref>) -->
(the Hadoop Native Library) to be accessible to both the server and the
client. <codeph>libhadoop.so</codeph> is not available if you have
installed from a tarball. You must install from an
<codeph>.rpm</codeph>, <codeph>.deb</codeph>, or parcel to use
short-circuit local reads.
<note audience="hidden">
If you use Cloudera Manager, you can
enable short-circuit reads through a checkbox in the user interface
and that setting takes effect for Impala as well.
</note>
short-circuit local reads.
</p>
<p>
<b>To configure DataNodes for short-circuit reads:</b>
@@ -105,11 +72,9 @@ under the License.
Impala configuration directory. The default Impala configuration
location is <codeph>/etc/impala/conf</codeph>. </li>
<li>
<indexterm audience="hidden"
>dfs.client.read.shortcircuit</indexterm>
<indexterm audience="hidden">dfs.client.read.shortcircuit</indexterm>
<indexterm audience="hidden">dfs.domain.socket.path</indexterm>
<indexterm audience="hidden"
>dfs.client.file-block-storage-locations.timeout.millis</indexterm>
<indexterm audience="hidden">dfs.client.file-block-storage-locations.timeout.millis</indexterm>
On all Impala nodes, configure the following properties in <!-- Exact timing is unclear, since we say farther down to copy /etc/hadoop/conf/hdfs-site.xml to /etc/impala/conf.
Which wouldn't work if we already modified the Impala version of the file here. Not to mention that this
doesn't take the CM interface into account, where these /etc files might not exist in those locations. -->

4
docs/topics/impala_faq.xml
View File

@@ -39,13 +39,11 @@ under the License.
This section lists frequently asked questions for Apache Impala (incubating),
the interactive SQL engine for Hadoop.
</p>
<p>
This section is under construction.
</p>
</conbody>
</concept>

93
docs/topics/impala_fixed_issues.xml
View File

@@ -3784,7 +3784,7 @@ db3362f IMPALA-1801: external-data-source-executor leaking global jni refs
<b>Bug:</b>
<xref href="https://issues.cloudera.org/browse/IMPALA-1019" scope="external" format="html">IMPALA-1019</xref>
</p>
<p>
<b>Resolution:</b> This issue is fixed in Impala 1.3.2. The addition of HDFS caching
support in Impala 1.4 means that this issue does not apply to any new level of Impala.
@@ -4018,7 +4018,7 @@ db3362f IMPALA-1801: external-data-source-executor leaking global jni refs
<b>Bug:</b>
<xref href="https://issues.cloudera.org/browse/IMPALA-1019" scope="external" format="html">IMPALA-1019</xref>
</p>
<p>
<b>Resolution:</b> This issue is fixed in Impala 1.3.2. The addition of HDFS caching
support in Impala 1.4 means that this issue does not apply to any new level of Impala.
@@ -4770,11 +4770,6 @@ Bad stats:
key.
</p>
<p audience="hidden">
<b>Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-1188" scope="external" format="html">IMP-1188</xref>
</p>
<p>
<b>Resolution:</b> Queries now return appropriate results when function calls are used in the row key
comparison. For queries involving non-existent row keys, such as <codeph>WHERE <varname>row_key</varname>
@@ -5614,12 +5609,6 @@ hive&gt; NULL</codeblock>
Insert INTO TABLE SELECT &lt;constant&gt; will not insert any data and may return an error.
</p>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-231" scope="external" format="html"/> ; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Anticipated Resolution</b>: Fixed
</p>
@@ -6076,12 +6065,6 @@ hive&gt; NULL</codeblock>
<conbody>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-474" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.7
</p>
@@ -6191,12 +6174,6 @@ hive&gt; NULL</codeblock>
does not propagate to Impala.
</p>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-56" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Severity:</b> Low
</p>
@@ -6435,12 +6412,6 @@ hive&gt; NULL</codeblock>
impala-shell will incorrectly report that the failed metadata refresh completed successfully.
</p>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-611" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Anticipated Resolution:</b> To be fixed in a future release
</p>
@@ -6511,12 +6482,6 @@ hive&gt; NULL</codeblock>
<conbody>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-601" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.4
</p>
@@ -6541,12 +6506,6 @@ hive&gt; NULL</codeblock>
table will return an <codeph>unknown table</codeph> error message, even if the table is known.
</p>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-298" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.3
</p>
@@ -6565,12 +6524,6 @@ hive&gt; NULL</codeblock>
even if the metadata for that table is fixed.
</p>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-298" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.3
</p>
@@ -6587,12 +6540,6 @@ hive&gt; NULL</codeblock>
Attempting to select from these tables fails.
</p>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-581" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.3
</p>
@@ -6610,12 +6557,6 @@ hive&gt; NULL</codeblock>
any of the joined tables in the WHERE clause.
</p>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-137" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.3.
</p>
@@ -6642,12 +6583,6 @@ hive&gt; NULL</codeblock>
<codeblock>SELECT * FROM (SELECT sum(col1) FROM some_table GROUP BY col1) t1 JOIN other_table ON (...);</codeblock>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-491" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.2
</p>
@@ -6666,12 +6601,6 @@ hive&gt; NULL</codeblock>
<codeblock>INSERT OVERWRITE TABLE test SELECT * FROM test2 LIMIT 1;</codeblock>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-497" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.2
</p>
@@ -6690,12 +6619,6 @@ hive&gt; NULL</codeblock>
<codeblock>SELECT * FROM test2 LIMIT 1;</codeblock>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-535" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.2
</p>
@@ -6712,12 +6635,6 @@ hive&gt; NULL</codeblock>
Attempting to read such files does not generate a diagnostic.
</p>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-270" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.2
</p>
@@ -6735,12 +6652,6 @@ hive&gt; NULL</codeblock>
exception.
</p>
<p audience="hidden">
<b>Cloudera Bug:</b>
<xref href="https://jira.cloudera.com/browse/IMP-522" scope="external" format="html"/>; KI added 0.1
<i>Cloudera internal only</i>
</p>
<p>
<b>Resolution:</b> Fixed in 0.2
</p>

15
docs/topics/impala_impala_shell.xml
View File

@@ -93,20 +93,9 @@ under the License.
and edit previous commands.
</li>
</ul>
<p>
For information on installing the Impala shell, see <xref href="impala_install.xml#install"/>.
</p>
<p audience="hidden">
For information on installing the Impala shell, see <xref href="impala_install.xml#install"/>. In Cloudera
Manager 4.1 and higher, Cloudera Manager installs <cmdname>impala-shell</cmdname> automatically. You might
install <cmdname>impala-shell</cmdname> manually on other systems not managed by Cloudera Manager, so that
you can issue queries from client systems that are not also running the Impala daemon or other Apache Hadoop
components.
<p>
For information on installing the Impala shell, see <xref href="impala_install.xml#install"/>.
</p>
<p>

17
docs/topics/impala_incompatible_changes.xml
View File

@@ -1229,12 +1229,6 @@ select * from `cross`;</codeblock>
</li>
</ul>
<p audience="hidden">
Impala 1.2.1 goes along with CDH 4.5 and Cloudera Manager 4.8. If you used the beta version Impala 1.2.0
that came with the beta of CDH 5, Impala 1.2.1 includes all the features of Impala 1.2.0 except for
resource management, which relies on the YARN framework from CDH 5.
</p>
<p>
The new <cmdname>catalogd</cmdname> service might require changes to any user-written scripts that stop,
start, or restart Impala services, install or upgrade Impala packages, or issue <codeph>REFRESH</codeph> or
@@ -1267,7 +1261,7 @@ select * from `cross`;</codeblock>
<ul conref="../shared/impala_common.xml#common/catalogd_xrefs">
<li/>
</ul>
<p>
The new resource management feature interacts with both YARN and Llama services.
<ph audience="PDF">See
@@ -1366,17 +1360,8 @@ ALTER TABLE <varname>table_name</varname> SET FILEFORMAT
<codeph>impala-lzo-cdh4</codeph> to the latest level. See <xref href="impala_txtfile.xml#lzo"/> for
details.
</li>
<li audience="hidden">
Cloudera Manager 4.5.2 and higher only supports Impala 1.0 and higher, and vice versa. If you upgrade to
Impala 1.0 or higher managed by Cloudera Manager, you must also upgrade Cloudera Manager to version 4.5.2
or higher. If you upgrade from an earlier version of Cloudera Manager, and were using Impala, you must
also upgrade Impala to version 1.0 or higher. The beta versions of Impala are no longer supported as of
the release of Impala 1.0.
</li>
</ul>
</conbody>
</concept>
</concept>

2
docs/topics/impala_isilon.xml
View File

@@ -87,7 +87,7 @@ CREATE TABLE d1.t2 (a TINYINT, b BOOLEAN);
the temporary data for the spill-to-disk feature, that configuration is
not recommended due to the need to transfer the data both ways using remote I/O.
</p>
<p>
When tuning Impala queries on HDFS, you typically try to avoid any remote reads.
When the data resides on Isilon storage, all the I/O consists of remote reads.

5
docs/topics/impala_kerberos.xml
View File

@@ -148,9 +148,8 @@ under the License.
</li>
<li>
Editing <codeph>/etc/default/impala</codeph> <ph audience="hidden">(in cluster not managed by Cloudera Manager), or editing the
<uicontrol>Security</uicontrol> settings in the Cloudera Manager interface,</ph>to accommodate Kerberos
authentication.
Editing <codeph>/etc/default/impala</codeph>
to accommodate Kerberos authentication.
</li>
</ul>
</conbody>

12
docs/topics/impala_logging.xml
View File

@@ -178,7 +178,6 @@ under the License.
<prolog>
<metadata>
<data name="Category" value="Administrators"/>
<data name="Category" value="Cloudera Manager"/>
</metadata>
</prolog>
@@ -358,19 +357,18 @@ I0107 08:42:12.292706 14876 logging.cc:76] Flags (see also /varz are on debug we
<title>Setting Logging Levels</title>
<conbody>
<p>
Impala uses the GLOG system, which supports three logging levels. You can adjust logging levels
by exporting variable settings. To change logging settings manually, use a command
similar to the following on each node before starting <codeph>impalad</codeph>:
</p>
<codeblock>export GLOG_v=1</codeblock>
<note>
For performance reasons, Cloudera highly recommends not enabling the most verbose logging level of 3.
For performance reasons, do not enable the most verbose logging level of 3 unless there is
no other alternative for troubleshooting.
</note>
<p>
@@ -390,7 +388,7 @@ I0107 08:42:12.292706 14876 logging.cc:76] Flags (see also /varz are on debug we
</p>
<p>
Increasing logging levels imposes performance overhead and increases log size. <ph rev="upstream">Cloudera</ph> recommends using
Increasing logging levels imposes performance overhead and increases log size. Where practical, use
GLOG_v=1 for most cases: this level has minimal performance impact but still captures useful
troubleshooting information.
</p>
@@ -443,7 +441,7 @@ I0107 08:42:12.292706 14876 logging.cc:76] Flags (see also /varz are on debug we
system, such as credit card numbers or tax IDs, and literals matching these patterns are obfuscated wherever
they would normally be recorded in log files or displayed in administration or debugging user interfaces.
</p>
<p>
In a security context, the log redaction feature is complementary to the Sentry authorization framework.
Sentry prevents unauthorized users from being able to directly access table data. Redaction prevents

52
docs/topics/impala_new_features.xml
View File

@@ -1044,7 +1044,7 @@ under the License.
now use an optimized code path.
</p>
</li>
<li>
<p rev="IMPALA-3044 IMPALA-2538 IMPALA-1168">
Improvements to the memory reservation mechanism for the Impala
@@ -1055,20 +1055,7 @@ under the License.
setting) is now unlimited instead of 200.
</p>
</li>
<li audience="hidden">
<p rev="IMPALA-3044 IMPALA-2538 IMPALA-1168 CDH-33289 CDH-34603">
Improvements to the memory reservation mechanism for the Impala
admission control feature. You can specify more settings, such
as the timeout period and maximum aggregate memory used, for each
resource pool instead of globally for the Impala instance. The
default limit for concurrent queries (the <uicontrol>max requests</uicontrol>
setting) is now unlimited instead of 200.
The Cloudera Manager user interface for admission control has been
reworked, with the settings available under the
<uicontrol>Dynamic Resource Pools</uicontrol> window.
</p>
</li>
<li>
<p rev="IMPALA-1755">
Performance improvements related to code generation.
@@ -3395,14 +3382,6 @@ under the License.
information about Impala queries that succeed or are blocked due to insufficient privileges. For details,
see <xref href="impala_security.xml#security"/>.
</li>
<li audience="hidden">
Additional security feature: auditing. New startup options for <cmdname>impalad</cmdname> let you capture
information about Impala queries that succeed or are blocked due to insufficient privileges. To take full
advantage of this feature with Cloudera Manager, upgrade to Cloudera Manager 4.7 or higher. For details,
see <xref href="impala_security.xml#security"/>.
</li>
<li>
Parquet data files generated by Impala 1.1.1 are now compatible with the Parquet support in Hive. See
@@ -3445,13 +3424,6 @@ under the License.
groups. By assigning privileges for views, you can control access to table data at the column level. For
details, see <xref href="impala_security.xml#security"/>.
</li>
<li audience="hidden">
Impala 1.1 works with Cloudera Manager 4.6 or higher. To use Cloudera Manager to manage authorization for
the Impala web UI (the web pages served from port 25000 by default), use Cloudera Manager 4.6.2 or
higher.
</li>
<li>
Impala can now create, alter, drop, and query views. Views provide a flexible way to set up simple
@@ -3570,12 +3542,6 @@ under the License.
<conbody>
<p audience="hidden">
The primary enhancements in Impala 1.0.1 are internal, for compatibility with the new Cloudera Manager 4.6
release. Try out the new <uicontrol>Impala Query Monitoring</uicontrol> feature in Cloudera Manager 4.6,
which requires Impala 1.0.1.
</p>
<p>
New user-visible features include:
</p>
@@ -3694,12 +3660,6 @@ under the License.
</li>
</ul>
<p audience="hidden">
In this version, both CDH 4.1 and 4.2 are supported, but due to performance improvements added, we highly
recommend you use CDH 4.2 or higher to see the full benefit. If you are using Cloudera Manager, version 4.5
is required.
</p>
</conbody>
</concept>
@@ -3732,10 +3692,6 @@ under the License.
</ul>
</li>
<li audience="hidden">
Cloudera Manager 4.5 and CDH 4.2 support Impala 0.6.
</li>
<li>
Support for the RCFile file format. For more information on file formats, see
<xref href="impala_file_formats.xml#file_formats">Understanding File Formats</xref>.
@@ -3784,10 +3740,6 @@ under the License.
Added support for Impala on RHEL5.7/Centos5.7. Impala is now supported on RHEL5.7/6.2 and Centos5.7/6.2.
</li>
<li audience="hidden">
Cloudera Manager 4.1.3 supports Impala 0.4.
</li>
<li>
The Impala debug webserver now has the ability to serve static files from
<codeph>${IMPALA_HOME}/www</codeph>. This can be disabled by setting

11
docs/topics/impala_noncm_installation.xml
View File

@@ -173,16 +173,7 @@ $ sudo apt-get install impala-catalog # Service start/stop script
<xref href="impala_config_performance.xml#config_performance"/>. Some of these configuration changes are
mandatory.
</li>
<li audience="hidden">
Complete any required or recommended configuration, as described in
<xref href="impala_config_performance.xml#config_performance"/>. Some of these configuration changes are
mandatory. (They are applied automatically when you install using Cloudera Manager.)
</li>
</ol>
<p>

11
docs/topics/impala_perf_resources.xml
View File

@@ -64,22 +64,13 @@ under the License.
hold cached metadata.
</p>
</li>
<li>
<p>
For production deployments, implement resource isolation using your cluster management
tool.
</p>
</li>
<li audience="hidden">
<p>
For production deployment, <ph rev="upstream">Cloudera</ph> recommends that you implement resource isolation using mechanisms
such as cgroups, which you can configure using Cloudera Manager. For details, see the
<xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_service_pools.html" scope="external" format="html">Static
Resource Pools</xref> in the Cloudera Manager documentation.
</p>
</li>
</ul>
</conbody>
</concept>

11
docs/topics/impala_perf_skew.xml
View File

@@ -67,22 +67,13 @@ under the License.
by a detailed performance analysis.
</p>
</li>
<li>
<p>
In the Impala debug web UI, click on the <uicontrol>Profile</uicontrol> link associated with the query after it is
complete. The executive summary information is displayed early in the profile output.
</p>
</li>
<li audience="hidden">
<p>
In the Cloudera Manager interface or the Impala debug web UI, click on the <uicontrol>Profile</uicontrol>
link associated with the query after it is complete. The executive summary information is displayed early
in the profile output.
</p>
</li>
</ul>
<p>

10
docs/topics/impala_perf_testing.xml
View File

@@ -39,18 +39,10 @@ under the License.
<conbody>
<p>
Test to ensure that Impala is configured for optimal performance. If you have installed Impala with cluster
Test to ensure that Impala is configured for optimal performance. If you have installed Impala with cluster
management software, complete the processes described in this topic to help ensure a proper
configuration. These procedures can be used to verify that Impala is set up correctly.
</p>
<p audience="hidden">
Test to ensure that Impala is configured for optimal performance. If you have installed Impala without
Cloudera Manager, complete the processes described in this topic to help ensure a proper configuration. Even
if you installed Impala with Cloudera Manager, which automatically applies appropriate configurations, these
procedures can be used to verify that Impala is set up correctly.
</p>
<section id="checking_config_performance">

15
docs/topics/impala_prereqs.xml
View File

@@ -106,23 +106,14 @@ under the License.
the metastore database. For the process of installing and configuring the metastore, see
<xref href="impala_install.xml#install"/>.
</p>
<p audience="hidden">
Always configure a <b>Hive metastore service</b> rather than connecting directly to the metastore
database. The Hive metastore service is required to interoperate between possibly different levels of
metastore APIs used by CDH and Impala, and avoids known issues with connecting directly to the
metastore database. The Hive metastore service is set up for you by default if you install through
Cloudera Manager 4.5 or higher.
</p>
<p>
Always configure a <b>Hive metastore service</b> rather than connecting directly to the metastore
database. The Hive metastore service is required to interoperate between different levels of
metastore APIs if this is necessary for your environment, and using it avoids known issues with
metastore APIs if this is necessary for your environment, and using it avoids known issues with
connecting directly to the metastore database.
</p>
<p>
A summary of the metastore installation process is as follows:
</p>

233
docs/topics/impala_proxy.xml
View File

@@ -161,7 +161,7 @@ under the License.
</concept>
<concept id="proxy_balancing" rev="CDH-33836 DOCS-349 CDH-39925 CDH-36812" audience="hidden">
<concept id="proxy_balancing" rev="" audience="hidden">
<title>Choosing the Load-Balancing Algorithm</title>
<conbody>
<p>
@@ -260,27 +260,19 @@ under the License.
running the <cmdname>impalad</cmdname> daemon.
</li>
<li rev="CDH-40363" audience="hidden">
For a cluster managed by Cloudera Manager (5.4.2 or higher), fill in the Impala configuration setting
<uicontrol>Impala Daemons Load Balancer</uicontrol> with the appropriate host:port combination.
Then restart the Impala service.
For systems using a recent level of Cloudera Manager, this is all the configuration you need; you can skip the remaining steps in this procedure.
</li>
<li>
Copy the keytab file from the proxy host to all other hosts in the cluster that run the
<cmdname>impalad</cmdname> daemon. (For optimal performance, <cmdname>impalad</cmdname> should be running
on all DataNodes in the cluster.) Put the keytab file in a secure location on each of these other hosts.
</li>
<li>
Add an entry <codeph>impala/<varname>actual_hostname</varname>@<varname>realm</varname></codeph> to the keytab on each
host running the <cmdname>impalad</cmdname> daemon.
</li>
<li>
For each impalad node, merge the existing keytab with the proxy’s keytab using
<cmdname>ktutil</cmdname>, producing a new keytab file. For example:
<codeblock>$ ktutil
@@ -288,11 +280,11 @@ under the License.
ktutil: read_kt impala.keytab
ktutil: write_kt proxy_impala.keytab
ktutil: quit</codeblock>
</li>
<li>
To verify that the keytabs are merged, run the command:
<codeblock>
klist -k <varname>keytabfile</varname>
@@ -300,14 +292,14 @@ klist -k <varname>keytabfile</varname>
which lists the credentials for both <codeph>principal</codeph> and <codeph>be_principal</codeph> on
all nodes.
</li>
<li>
Make sure that the <codeph>impala</codeph> user has permission to read this merged keytab file.
Make sure that the <codeph>impala</codeph> user has permission to read this merged keytab file.
</li>
<li>
Change the following configuration settings for each host in the cluster that participates
in the load balancing:
@@ -322,98 +314,27 @@ klist -k <varname>keytabfile</varname>
<note>
Every host has different <codeph>--be_principal</codeph> because the actual hostname
is different on each host.
Specify the fully qualified domain name (FQDN) for the proxy host, not the IP
address. Use the exact FQDN as returned by a reverse DNS lookup for the associated
IP address.
</note>
</li>
<li>
Modify the startup options. See <xref href="impala_config_options.xml#config_options"/> for the procedure to modify the startup
options.
</li>
</ul>
</li>
<li>
Restart Impala to make the changes take effect. Restart the <cmdname>impalad</cmdname> daemons on all
hosts in the cluster, as well as the <cmdname>statestored</cmdname> and <cmdname>catalogd</cmdname>
daemons.
</li>
<li audience="hidden">
On systems not managed by Cloudera Manager, or systems using Cloudera Manager earlier than 5.4.2:
<ol>
<li>
Copy the keytab file from the proxy host to all other hosts in the cluster that run the
<cmdname>impalad</cmdname> daemon. (For optimal performance, <cmdname>impalad</cmdname> should be running
on all DataNodes in the cluster.) Put the keytab file in a secure location on each of these other hosts.
</li>
<li>
Add an entry <codeph>impala/<varname>actual_hostname</varname>@<varname>realm</varname></codeph> to the keytab on each
host running the <cmdname>impalad</cmdname> daemon.
</li>
<li>
For each impalad node, merge the existing keytab with the proxy’s keytab using
<cmdname>ktutil</cmdname>, producing a new keytab file. For example:
<codeblock>$ ktutil
ktutil: read_kt proxy.keytab
ktutil: read_kt impala.keytab
ktutil: write_kt proxy_impala.keytab
ktutil: quit</codeblock>
<note>
On systems managed by Cloudera Manager 5.1.0 and later, the keytab merging happens automatically. To
verify that Cloudera Manager has merged the keytabs, run the command:
<codeblock>klist -k <varname>keytabfile</varname></codeblock>
which lists the credentials for both <codeph>principal</codeph> and <codeph>be_principal</codeph> on
all nodes.
</note>
</li>
<li>
Make sure that the <codeph>impala</codeph> user has permission to read this merged keytab file.
</li>
<li>
Change some configuration settings for each host in the cluster that participates in the load balancing.
Follow the appropriate steps depending on whether you use Cloudera Manager or not:
<ul>
<li> In the <cmdname>impalad</cmdname> option definition, or the advanced
configuration snippet, add: <codeblock>--principal=impala/<varname>proxy_host</varname>@<varname>realm</varname>
--be_principal=impala/<varname>actual_host</varname>@<varname>realm</varname>
--keytab_file=<varname>path_to_merged_keytab</varname></codeblock>
</li>
<li>
On a cluster not managed by Cloudera Manager, see
<xref href="impala_config_options.xml#config_options"/> for the procedure to modify the startup
options.
</li>
</ul>
</li>
<li>
Restart Impala to make the changes take effect. Follow the appropriate steps depending on whether you use
Cloudera Manager or not:
<ul>
<li>
On a cluster not managed by Cloudera Manager, restart the <cmdname>impalad</cmdname> daemons on all
hosts in the cluster, as well as the <cmdname>statestored</cmdname> and <cmdname>catalogd</cmdname>
daemons.
</li>
</ul>
</li>
</ol>
</li>
</ol>
<!--
@@ -571,128 +492,6 @@ listen impalajdbc :21051
<note conref="../shared/impala_common.xml#common/proxy_jdbc_caveat"/>
<p audience="hidden">
The following example shows extra steps needed for a cluster using Kerberos authentication:
</p>
<codeblock audience="hidden">$ klist
$ impala-shell -k
$ kinit -r 1d -kt /systest/keytabs/hdfs.keytab hdfs
$ impala-shell -i c2104.hal.cloudera.com:21000
$ impala-shell -i c2104.hal.cloudera.com:25003
[root@c2104 alan]# ps -ef |grep impalad
root 6442 6428 0 12:21 pts/0 00:00:00 grep impalad
impala 30577 22192 99 Nov14 ? 3-16:42:32 /usr/lib/impala/sbin-debug/impalad --flagfile=/var/run/cloudera-scm-agent/process/10342-impala-IMPALAD/impala-conf/impalad_flags
[root@c2104 alan]# vi /var/run/cloudera-scm-agent/process/10342-impala-IMPALAD/impala-conf/impalad_flags
$ klist -k /var/run/cloudera-scm-agent/process/10342-impala-IMPALAD/impala.keytab
Keytab name: FILE:/var/run/cloudera-scm-agent/process/10342-impala-IMPALAD/impala.keytab
KVNO Principal
---- --------------------------------------------------------------------------
2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
$ klist
Ticket cache: FILE:/tmp/krb5cc_4028
Default principal: hdfs@HAL17.CLOUDERA.COM
Valid starting Expires Service principal
11/15/13 12:17:17 11/15/13 12:32:17 krbtgt/HAL17.CLOUDERA.COM@HAL17.CLOUDERA.COM
renew until 11/16/13 12:17:17
11/15/13 12:17:21 11/15/13 12:32:17 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
renew until 11/16/13 12:17:17
$ kinit -r 1d -kt /systest/keytabs/hdfs.keytab hdfs
$ kinit -R
$ impala-shell -k -i c2106.hal.cloudera.com:21000
Starting Impala Shell using Kerberos authentication
Using service name 'impala'
Connected to c2106.hal.cloudera.com:21000
$ impala-shell -i c2104.hal.cloudera.com:25003
$ impala-shell -k -i c2104.hal.cloudera.com:25003
Starting Impala Shell using Kerberos authentication
Using service name 'impala'
Connected to c2104.hal.cloudera.com:25003
[c2104.hal.cloudera.com:25003] &gt; create table alan_tmp(a int);
Query: create table alan_tmp(a int)
ERROR: InternalException: Got exception: org.apache.hadoop.ipc.RemoteException User: hive/c2102.hal.cloudera.com@HAL17.CLOUDERA.COM is not allowed to impersonate impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
$ kdestroy
$ kinit -r 1d -kt /systest/keytabs/hdfs.keytab hdfs
$ impala-shell -k -i c2104.hal.cloudera.com:25003
# klist -k c2104.keytab
Keytab name: FILE:c2104.keytab
KVNO Principal
---- --------------------------------------------------------------------------
2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
$ klist -k -t c2106.keytab
Keytab name: FILE:c2106.keytab
KVNO Timestamp Principal
---- ----------------- --------------------------------------------------------
2 02/14/13 12:12:22 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 02/14/13 12:12:22 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 02/14/13 12:12:22 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 02/14/13 12:12:22 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 02/14/13 12:12:22 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 02/14/13 12:12:22 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 02/14/13 12:12:22 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 02/14/13 12:12:22 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
$ ktutil
ktutil: rkt c2104.keytab
ktutil: rkt c2106.keytab
ktutil: wkt my_test.keytab
ktutil: q
$ klist -k -t my_test.keytab
Keytab name: FILE:my_test.keytab
KVNO Timestamp Principal
---- ----------------- --------------------------------------------------------
2 11/21/13 16:22:40 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:40 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:40 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:40 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:40 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:40 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:40 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:40 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:40 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:41 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:41 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:41 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:41 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:41 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:41 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
2 11/21/13 16:22:41 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
$ kdestroy
$ kinit -r 1d -kt /systest/keytabs/hdfs.keytab hdfs
$ vi README
$ kinit -R
$ impala-shell -k -i c2104.hal.cloudera.com:25003
Starting Impala Shell using Kerberos authentication
Using service name 'impala'
Connected to c2104.hal.cloudera.com:25003
<ph conref="../shared/ImpalaVariables.xml#impala_vars/ImpaladBanner"/>
Welcome to the Impala shell. Press TAB twice to see a list of available commands.
...
<ph conref="../shared/ImpalaVariables.xml#impala_vars/ShellBanner"/>
[c2104.hal.cloudera.com:25003] &gt; show tables;
Query: show tables
ERROR: AnalysisException: This Impala daemon is not ready to accept user requests. Status: Waiting for catalog update from the StateStore.
[c2104.hal.cloudera.com:25003] &gt; quit;</codeblock>
<!--
At that point in the walkthrough with Alan Choi, we could never get Impala to accept any requests through the catalog server.
So I have not seen a 100% successful proxy setup process to verify all the details.
-->
</conbody>
</concept>

21
docs/topics/impala_resource_management.xml
View File

@@ -61,21 +61,7 @@ under the License.
can be safely run at the same time. Then the Impala daemon enforces the limit by activating the
spill-to-disk mechanism when necessary, or cancelling a query altogether if the limit is exceeded at runtime.
</p>
<ul audience="hidden">
<li>
If Cloudera Manager Static Partitioning is used, it creates a cgroup in which Impala runs.
This cgroup limits CPU, network, and IO according to the static partitioning policy.
</li>
<li>
Limits on memory usage are enforced by Impala's process memory limit (the <codeph>MEM_LIMIT</codeph>
query option setting). The admission control feature checks this setting to decide how many queries
can be safely run at the same time. Then the Impala daemon enforces the limit by activating the
spill-to-disk mechanism when necessary, or cancelling a query altogether if the limit is exceeded at runtime.
</li>
</ul>
</conbody>
</concept>
@@ -110,11 +96,6 @@ under the License.
cluster.
</p>
<p>
For information about setting up the YARN service, see the instructions for
<xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_yarn_service.html" scope="external" format="html">Cloudera
Manager</xref>.
</p>
</conbody>
</concept>
@@ -144,7 +125,7 @@ under the License.
<codeph>-rm_always_use_defaults</codeph>: If this Boolean option is enabled, Impala ignores computed
estimates and always obtains the default memory and CPU allocation settings at the start of the
query. These default estimates are approximately 2 CPUs and 4 GB of memory, possibly varying slightly
depending on cluster size, workload, and so on. <ph rev="upstream">Cloudera</ph> recommends enabling
depending on cluster size, workload, and so on. Where practical, enable
<codeph>-rm_always_use_defaults</codeph> whenever resource management is used, and relying on these
default values (that is, leaving out the two following options).
</li>

4
docs/topics/impala_s3.xml
View File

@@ -108,7 +108,7 @@ under the License.
form of the <codeph>CREATE TABLE</codeph> statement, can copy data from an HDFS table or another S3
table into an S3 table. The <xref href="impala_s3_skip_insert_staging.xml#s3_skip_insert_staging"/>
query option chooses whether or not to use a fast code path for these write operations to S3,
with the tradeoff of potential inconsistency in the case of a failure during the statement.
with the tradeoff of potential inconsistency in the case of a failure during the statement.
</p>
</li>
</ul>
@@ -152,7 +152,7 @@ under the License.
Hive services. (Restarting Hive is required because Impala queries, CREATE TABLE statements, and so on go
through the Hive metastore.)
</p>
<note type="important">
<!--
<ul>

48
docs/topics/impala_scalability.xml
View File

@@ -294,11 +294,6 @@ Memory Usage: Additional Notes
</dlentry>
</dl>
<p audience="hidden">
As of CDH 5.3, not all of these flags are present in the Cloudera Manager user interface. Some must be set
using the <uicontrol>Advanced Configuration Snippet</uicontrol> fields for the statestore component.
</p>
<p>
If it takes a very long time for a cluster to start up, and <cmdname>impala-shell</cmdname> consistently
displays <codeph>This Impala daemon is not ready to accept user requests</codeph>, the statestore might be
@@ -682,32 +677,6 @@ these tables, hint the plan or disable this behavior via query options to enable
case, you know that any queries that spill will not go overboard with their memory consumption.
</p>
<!--
<p>
<b>Turning off the spill-to-disk feature: (<keyword keyref="impala24_full"/> and lower only)</b>
</p>
<p>
Prior to <keyword keyref="impala25_full"/> certain conditions...
</p>
<p>
You might turn off the spill-to-disk feature if you are in an environment with constraints on disk space,
or if you prefer for queries that exceed the memory capacity in your cluster to <q>fail fast</q> so that
you can tune and retry them.
</p>
<p>
To turn off this feature, set the following configuration options for each <cmdname>impalad</cmdname>
daemon, either through the <cmdname>impalad</cmdname> advanced configuration snippet in Cloudera Manager,
or during <cmdname>impalad</cmdname> startup on each DataNode on systems not managed by Cloudera Manager:
</p>
<codeblock>&minus;&minus;enable_partitioned_aggregation=false
&minus;&minus;enable_partitioned_hash_join=false
</codeblock>
-->
</conbody>
</concept>
@@ -795,21 +764,14 @@ While these authentication requests are being processed, any submitted Impala qu
During this period, the KDC and DNS may be slow to respond to requests from components other than Impala,
so other secure services might be affected temporarily.
</p>
<p>
To reduce the frequency of the <codeph>kinit</codeph> renewal that initiates
a new set of authentication requests, increase the <codeph>kerberos_reinit_interval</codeph>
configuration setting for the <cmdname>impalad</cmdname> daemons. Currently, the default is 60 minutes.
Consider using a higher value such as 360 (6 hours).
</p>
<p audience="hidden">
To reduce the frequency of the <codeph>kinit</codeph> renewal that initiates a new set of
authentication requests, increase the <codeph>kerberos_reinit_interval</codeph> configuration setting
for the <cmdname>impalad</cmdname> daemons. Currently, the default for a cluster not managed by
Cloudera Manager is 60 minutes, while the default under Cloudera Manager is 10 minutes.
Consider using a higher value such as 360 (6 hours).
a new set of authentication requests, increase the <codeph>kerberos_reinit_interval</codeph>
configuration setting for the <cmdname>impalad</cmdname> daemons. Currently, the default is 60 minutes.
Consider using a higher value such as 360 (6 hours).
</p>
</conbody>
</concept>

6
docs/topics/impala_schema_design.xml
View File

@@ -229,12 +229,6 @@ under the License.
<p>
</p>
</section>
<section id="schema_design_cm">
<title>Use Cloudera Manager to monitor queries and overall performance.</title>
<p>
</p>
</section>
-->
</conbody>
</concept>

17
docs/topics/impala_timeouts.xml
View File

@@ -169,7 +169,7 @@ Trying to re-register with state-store</codeblock>
<title>Cancelling a Query</title>
<conbody>
<p>
Sometimes, an Impala query might run for an unexpectedly long time, tying up resources
in the cluster. You can cancel the query explicitly, independent of the timeout period,
@@ -177,21 +177,6 @@ Trying to re-register with state-store</codeblock>
default), and using the link on the <codeph>/queries</codeph> tab to cancel the running
query. For example, press <codeph>^C</codeph> in <cmdname>impala-shell</cmdname>.
</p>
<p audience="hidden">
Sometimes, an Impala query might run for an unexpectedly long time, tying up resources
in the cluster. You can cancel the query explicitly, independent of the timeout period,
by going into the web UI for the <cmdname>impalad</cmdname> host (on port 25000 by
default), and using the link on the <codeph>/queries</codeph> tab to cancel the running
query. Various client applications let you interactively cancel queries submitted or
monitored through those applications. For example, by pressing <codeph>^C</codeph> in
<cmdname>impala-shell</cmdname>, clicking the <uicontrol>Cancel</uicontrol> button from
the <uicontrol>Watch</uicontrol> page in Hue, clicking <uicontrol>Actions &gt;
Cancel</uicontrol> from the <uicontrol>Queries</uicontrol> list in Cloudera Manager, and
so on.
</p>
</conbody>

3
docs/topics/impala_troubleshooting.xml
View File

@@ -412,8 +412,7 @@ MEM_LIMIT: 0
<codeblock>
<![CDATA[
Useful test from beta at Visa.
SME: Jayant@
Useful test for I/O throughput of hardware.
Symptoms:
* Queries running slow

36
docs/topics/impala_txtfile.xml
View File

@@ -442,8 +442,8 @@ INSERT INTO csv SELECT * FROM other_file_format_table;</codeblock>
<indexterm audience="hidden">LZO support in Impala</indexterm>
<indexterm audience="hidden">compression</indexterm>
Impala supports using text data files that employ LZO compression. <ph rev="upstream">Cloudera</ph> recommends compressing
text data files when practical. Impala queries are usually I/O-bound; reducing the amount of data read from
Impala supports using text data files that employ LZO compression. Where practical, apply compression to
text data files. Impala queries are usually I/O-bound; reducing the amount of data read from
disk typically speeds up a query, despite the extra CPU work to uncompress the data in memory.
</p>
@@ -479,43 +479,15 @@ INSERT INTO csv SELECT * FROM other_file_format_table;</codeblock>
you establish, or by using packages. You must do these steps manually, whether or not you
are using cluster management software.
</p>
<p audience="hidden">
Before using LZO-compressed tables in Impala, do the following one-time setup for each machine in the
cluster. Install the necessary packages using either the Cloudera public repository, a private repository
you establish, or by using packages. You must do these steps manually, whether or not the cluster is
managed by the Cloudera Manager product.
</p>
<ol>
<li>
<b>Prepare your systems to work with LZO by downloading and installing the appropriate libraries:</b>
<p audience="hidden">
<b>On systems managed by Cloudera Manager using parcels:</b>
</p>
<p audience="hidden">
See the setup instructions for the LZO parcel in the Cloudera Manager documentation for
<xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_install_gpl_extras.html" scope="external" format="html">Cloudera
Manager 5</xref>.
</p>
<p>
<b>On systems using cluster management software or those not using cluster management
software:</b>
</p>
<p audience="hidden">
<b>On systems managed by Cloudera Manager using packages, or not managed by Cloudera Manager:</b>
</p>
<p>
Download and install the appropriate file to each machine on which you intend to use LZO with Impala.
These files all come from the Cloudera
<xref href="https://archive.cloudera.com/gplextras/redhat/5/x86_64/gplextras/" scope="external" format="html">GPL
These files all come from
<xref href="https://archive.cloudera.com/gplextras/redhat/5/x86_64/gplextras/" scope="external" format="html">this GPL
extras</xref> download site. Install the:
</p>
<ul>

18
docs/topics/impala_udf.xml
View File

@@ -161,7 +161,7 @@ select real_words(letters) from word_games;</codeblock>
<p>
Impala supports UDFs written in C++, in addition to supporting existing Hive UDFs written in Java.
<ph rev="upstream">Cloudera</ph> recommends using C++ UDFs because the compiled native code can yield higher performance, with
Where practical, use C++ UDFs because the compiled native code can yield higher performance, with
UDF execution time often 10x faster for a C++ UDF than the equivalent Java UDF.
</p>
</conbody>
@@ -437,21 +437,13 @@ and other examples demonstrating this technique in
(RHEL-based distributions) or <codeph>impala-udf-dev</codeph> (Ubuntu and Debian).
</li>
</ol>
<note>
The UDF development code does not rely on Impala being installed on the same machine. You can write and
compile UDFs on a minimal development system, then deploy them on a different one for use with Impala.
</note>
<note audience="hidden">
The UDF development code does not rely on Impala being installed on the same machine. You can write and
compile UDFs on a minimal development system, then deploy them on a different one for use with Impala. If
you develop UDFs on a server managed by Cloudera Manager through the parcel mechanism, you still install
the UDF development kit through the package mechanism; this small standalone package does not interfere
with the parcels containing the main Impala code.
</note>
<p>
@@ -1012,7 +1004,7 @@ BigIntVal CountFinalize(FunctionContext* context, const BigIntVal&amp; val);
// the StringVal intermediate type. When this UDA is registered, it would specify
// 16 bytes (8 byte sum + 8 byte count) as the size for this buffer.
//
// Usage: &gt; create aggregate function my_avg(double) returns string
// Usage: &gt; create aggregate function my_avg(double) returns string
// location '/user/cloudera/libudasample.so' update_fn='AvgUpdate';
// &gt; select cast(my_avg(col) as double) from tbl;

29
docs/topics/impala_webui.xml
View File

@@ -76,14 +76,14 @@ under the License.
</li>
</ul>
</p>
<note>
<p>
The web user interface is primarily for problem diagnosis and troubleshooting. The items listed and their
formats are subject to change.
</p>
</note>
<p outputclass="toc inpage"/>
</conbody>
@@ -191,7 +191,7 @@ under the License.
<codeph>http://<varname>impala-server-hostname</varname>:25000/logs</codeph> (non-secure cluster) or
<codeph>https://<varname>impala-server-hostname</varname>:25000/logs</codeph> (secure cluster).
</p>
<p>
This page shows the last portion of the <filepath>impalad.INFO</filepath> log file, the most detailed of
the info, warning, and error logs for the <cmdname>impalad</cmdname> daemon. You can refer here to see
@@ -199,7 +199,7 @@ under the License.
central page can be more convenient than looking around the filesystem for the log files, which could be
in different locations on clusters that use cluster management software.
</p>
</conbody>
</concept>
@@ -234,19 +234,12 @@ under the License.
<codeph>http://<varname>impala-server-hostname</varname>:25000/metrics</codeph> (non-secure cluster) or
<codeph>https://<varname>impala-server-hostname</varname>:25000/metrics</codeph> (secure cluster).
</p>
<p>
This page displays the current set of metrics: counters and flags representing various aspects of
<cmdname>impalad</cmdname> internal operation.
</p>
<p audience="hidden">
This page displays the current set of metrics: counters and flags representing various aspects of
<cmdname>impalad</cmdname> internal operation. For the meanings of these metrics, see
<xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_metrics_impala.html" scope="external" format="html">Impala
Metrics</xref> in the Cloudera Manager documentation.
</p>
</conbody>
</concept>
@@ -406,7 +399,7 @@ under the License.
the info, warning, and error logs for the <cmdname>impalad</cmdname> daemon. You can refer here to see
the details of the most recent operations, whether the operations succeeded or encountered errors. This
central page can be more convenient than looking around the filesystem for the log files, which could be
in different locations on clusters that use Cloudera Manager or not.
in different locations on different hosts.
</p>
</conbody>
</concept>
@@ -445,9 +438,7 @@ under the License.
<p>
This page displays the current set of metrics: counters and flags representing various aspects of
<cmdname>impalad</cmdname> internal operation. For the meanings of these metrics, see
<xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_metrics_impala.html" scope="external" format="html">Impala
Metrics</xref> in the Cloudera Manager documentation.
<cmdname>impalad</cmdname> internal operation.
</p>
</conbody>
</concept>
@@ -612,7 +603,7 @@ under the License.
the info, warning, and error logs for the <cmdname>impalad</cmdname> daemon. You can refer here to see
the details of the most recent operations, whether the operations succeeded or encountered errors. This
central page can be more convenient than looking around the filesystem for the log files, which could be
in different locations on clusters that use Cloudera Manager or not.
in different locations on different hosts.
</p>
</conbody>
</concept>
@@ -631,9 +622,7 @@ under the License.
<p>
This page displays the current set of metrics: counters and flags representing various aspects of
<cmdname>impalad</cmdname> internal operation. For the meanings of these metrics, see
<xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_metrics_impala.html" scope="external" format="html">Impala
Metrics</xref> in the Cloudera Manager documentation.
<cmdname>impalad</cmdname> internal operation.
</p>
</conbody>
</concept>