mirror of
https://github.com/apache/impala.git
synced 2025-12-21 19:08:12 -05:00
94f67a3432
Before this patch Impala mainly used Thrift 0.9.3, but it was
possible to compile Impala shell with Thrift 0.11.0, so the 0.11.0
Thrift lib was already included in the toolchain.
Most of the changes are related to replacing boost:: with std::
shared_ptr-s in cpp code (this is a continuation of patch by Sahil).
The Thrift upgrade also needs an Impyla release with Thrift 0.11.0, as
Impala's test framework relies on Impyla. A thrift_sasl release is also
needed, because it currently pins Thrift version to 0.9.3 for Python 2.
The current patch uses alpha releases from Impyla and thrift_sasl that
use thrift 0.11.0.
Notable side effects:
- old logic to compile thrift for impala-shell with 0.11.0 was removed
- impala_shell's utf8 handling had to be updated as the new 0.11.0
compilation happens with no_utf8strings. This also made things a
bit faster, e.g the following is ~0.22s instead of ~0.25
shell/impala_shell.py \
-B -q "select * from functional_parquet.alltypes;" > /dev/null
- THRIFT-3921 changed the stream operators to print an enum's name
instead of its number, leading to slightly different messages
in some cases.
- "templates" was added to the thift generator's parameters to avoid
a compilation issue (related to IMPALA-10600). I didn't notice any
change in compilation time. This option generated .tcc files with
templetized readers/writers for Thrift types. Currently we don't
use these, but they could potentially speed up (de)serialization.
Testing:
- ran Impyla's test suite with Python 2 and 3
- ran core tests
Change-Id: Idd13f177b4f7acc07872ea6399035aa180ef6ab6
Reviewed-on: http://gerrit.cloudera.org:8080/17170
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
57 lines
2.3 KiB
Bash
Executable File
57 lines
2.3 KiB
Bash
Executable File
#!/bin/bash
|
|
#
|
|
# Licensed to the Apache Software Foundation (ASF) under one
|
|
# or more contributor license agreements. See the NOTICE file
|
|
# distributed with this work for additional information
|
|
# regarding copyright ownership. The ASF licenses this file
|
|
# to you under the Apache License, Version 2.0 (the
|
|
# "License"); you may not use this file except in compliance
|
|
# with the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing,
|
|
# software distributed under the License is distributed on an
|
|
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
# KIND, either express or implied. See the License for the
|
|
# specific language governing permissions and limitations
|
|
# under the License.
|
|
|
|
|
|
# This script runs the Impala shell. Python is required.
|
|
#
|
|
# This script assumes that the supporting library files for the Impala shell are
|
|
# rooted in either the same directory that this script is in, or in a directory
|
|
# in the environment variable ${IMPALA_SHELL_HOME}.
|
|
#
|
|
# The required directories are:
|
|
# gen-py/ -- containing the Python thrift stubs for connecting to Impalad
|
|
# lib/ -- containing the Hive Beeswax libraries, and the Python thrift 0.11.0 libraries
|
|
# note that this directory exists only in the shell tarball, not in the
|
|
# Impala repository, see make_shell_tarball.sh for details
|
|
# ext-py/ -- containing all the eggs for the modules packaged with the shell.
|
|
|
|
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
|
SHELL_HOME=${IMPALA_SHELL_HOME:-${SCRIPT_DIR}}
|
|
|
|
# Set the envrionment's locale settings to allow for utf-8 compatibility
|
|
export LC_CTYPE=${LC_CTYPE:-en_US.UTF-8}
|
|
|
|
# User can configure EGG_CACHE by setting PYTHON_EGG_CACHE.
|
|
# By default it is set to a per-user temporary location,
|
|
# which follows what hue does.
|
|
PYTHON_EGG_CACHE=${PYTHON_EGG_CACHE:-/tmp/impala-shell-python-egg-cache-${USER}}
|
|
|
|
if [ ! -d ${PYTHON_EGG_CACHE} ]; then
|
|
mkdir ${PYTHON_EGG_CACHE}
|
|
fi
|
|
|
|
# External module eggs are located in /ext-py, append them to the PYTHONPATH
|
|
EGG_PATH=''
|
|
for EGG in $(ls ${SHELL_HOME}/ext-py/*.egg); do
|
|
EGG_PATH="${EGG}:${EGG_PATH}"
|
|
done
|
|
|
|
PYTHONPATH="${EGG_PATH}${SHELL_HOME}/gen-py:${SHELL_HOME}/lib:${PYTHONPATH}" \
|
|
PYTHONIOENCODING='utf-8' exec python ${SHELL_HOME}/impala_shell.py "$@"
|