mirror of
https://github.com/apache/impala.git
synced 2025-12-30 03:01:44 -05:00
d484d2f684
This now gives a clean RAT check with bin/check-rat-report.py, which is one way for the Impala community to check compliance with ASF rules on intellectual property. Change-Id: I2ad06435f84a65ba126759e42a18fdaf52cd7036 Reviewed-on: http://gerrit.cloudera.org:8080/5232 Reviewed-by: Jim Apple <jbapple-impala@apache.org> Tested-by: Impala Public Jenkins Reviewed-by: John Russell <jrussell@cloudera.com>
79 lines
3.6 KiB
XML
79 lines
3.6 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!--
|
|
Licensed to the Apache Software Foundation (ASF) under one
|
|
or more contributor license agreements. See the NOTICE file
|
|
distributed with this work for additional information
|
|
regarding copyright ownership. The ASF licenses this file
|
|
to you under the Apache License, Version 2.0 (the
|
|
"License"); you may not use this file except in compliance
|
|
with the License. You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing,
|
|
software distributed under the License is distributed on an
|
|
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
KIND, either express or implied. See the License for the
|
|
specific language governing permissions and limitations
|
|
under the License.
|
|
-->
|
|
<!DOCTYPE concept PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">
|
|
<concept id="mem_limits">
|
|
|
|
<title>Controlling Impala Resource Usage</title>
|
|
<titlealts audience="PDF"><navtitle>Controlling Resource Usage</navtitle></titlealts>
|
|
<prolog>
|
|
<metadata>
|
|
<data name="Category" value="Impala"/>
|
|
<data name="Category" value="Performance"/>
|
|
<data name="Category" value="Memory"/>
|
|
<data name="Category" value="Scalability"/>
|
|
<data name="Category" value="Resource Management"/>
|
|
<data name="Category" value="Administrators"/>
|
|
<data name="Category" value="Developers"/>
|
|
<data name="Category" value="Data Analysts"/>
|
|
</metadata>
|
|
</prolog>
|
|
|
|
<conbody>
|
|
|
|
<p>
|
|
Sometimes, balancing raw query performance against scalability requires limiting the amount of resources,
|
|
such as memory or CPU, used by a single query or group of queries. Impala can use several mechanisms that
|
|
help to smooth out the load during heavy concurrent usage, resulting in faster overall query times and
|
|
sharing of resources across Impala queries, MapReduce jobs, and other kinds of workloads across a CDH
|
|
cluster:
|
|
</p>
|
|
|
|
<ul>
|
|
<li>
|
|
The Impala admission control feature uses a fast, distributed mechanism to hold back queries that exceed
|
|
limits on the number of concurrent queries or the amount of memory used. The queries are queued, and
|
|
executed as other queries finish and resources become available. You can control the concurrency limits,
|
|
and specify different limits for different groups of users to divide cluster resources according to the
|
|
priorities of different classes of users. This feature is new in Impala 1.3, and works with both CDH 4 and
|
|
CDH 5. See <xref href="impala_admission.xml#admission_control"/> for details.
|
|
</li>
|
|
|
|
<li>
|
|
<p>
|
|
You can restrict the amount of memory Impala reserves during query execution by specifying the
|
|
<codeph>-mem_limit</codeph> option for the <codeph>impalad</codeph> daemon. See
|
|
<xref href="impala_config_options.xml#config_options"/> for details. This limit applies only to the
|
|
memory that is directly consumed by queries; Impala reserves additional memory at startup, for example to
|
|
hold cached metadata.
|
|
</p>
|
|
</li>
|
|
|
|
<li>
|
|
<p>
|
|
For production deployment, <ph rev="upstream">Cloudera</ph> recommends that you implement resource isolation using mechanisms
|
|
such as cgroups, which you can configure using Cloudera Manager. For details, see the
|
|
<xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_service_pools.html" scope="external" format="html">Static
|
|
Resource Pools</xref> in the Cloudera Manager documentation.
|
|
</p>
|
|
</li>
|
|
</ul>
|
|
</conbody>
|
|
</concept>
|