COMPUTE STATS is an async DDL command. When COMPUTE STATS fails it will set the
query status of the QueryExecState properly, but the original Beeswax::query() RPC
won't throw. The Impala shell sometimes did not pick up and display the
query status because no RPC actually threw. To fix this, I modified
Beeswax::get_log() to include the query status if it is not ok. The shell looks
for a special prefix to distinguish the query status from the runtime state error log.
Change-Id: I0d9dbf0801629a37de22ea4ebb6d2e5d53b836ef
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1899
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2063
The problem was that we were setting a flag marking the last_query_handle as closed, but
were not resetting the flag before the next query. This caused the first query to
be closed properly, but subsequent queries would not be closed. The fix is to change
where the flag is reset to the same place as where we assign last_query_handle.
Added a test case.
Change-Id: I870a96789489bfe4f388910b808409cd0584af8a
(cherry picked from commit 1439151af5b63112b0dd631fac9c7ab4d43bba37)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1976
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: jenkins
This change makes the fetch rpc interruptable. If the user cancels the query in
the middle of a fetch, the shell reconnects to the impalad and closes the
query. It also includes some code consolidation.
Change-Id: Iaaf0dfd4cba9ce2557e4a7d0447bc9c3ffda5e29
Reviewed-on: http://gerrit.ent.cloudera.com:8080/717
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: jenkins
Reindent is very expensive for large queries, removing it makes query parsing performant
for no loss of functionality.
Change-Id: I4d2bec6bbabaf949aa0f64193a6e2b3b3725407e
Reviewed-on: http://gerrit.ent.cloudera.com:8080/966
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
This patch reworks our Kerberos authentication layer to support multiple
authentication protocols, particularly PLAIN/SASL to support external
LDAP authentication.
There is now a system-wide AuthManager object, initialised by InitAuth()
which occurs during the usual InitCommonRuntime() setup. The AuthManager
is responsible for supplying AuthProvider objects to ThriftServers and
ThriftClients. The AuthProvider in turn generates Thrift transport
objects which are usually SASL-enabled, and which either employ GSSAPI
or PLAIN mechanisms.
In miscellaneous changes:
* Cyrus SASL now builds both with LDAP and the dummy '--enable-true'
external authentication mechanisms enabled.
* To test PLAIN/SASL authentication, you must now include
$IMPALA_HOME/thirdparty/${IMPALA_CYRUS_SASL_VERSION}/build/lib/sasl2 in
FLAGS_sasl_path.
* The shell now has an option to authenticate using LDAP, and will
prompt for a password at startup before doing so.
* Since the authentication code is almost entirely Thrift-specific, it
has been moved to the rpc lib.
Change-Id: I771de50f05630efdf1606ab9f0f48146ad54595e
Reviewed-on: http://gerrit.ent.cloudera.com:8080/716
Tested-by: jenkins
Reviewed-by: Henry Robinson <henry@cloudera.com>
quotation ending in a semi-colon.
Currently, the shell cannot parse a line which has an open quotation ending in a
semi-colon. It treats the semi-colon as a command terminator. With this change, the shell
will be able to detect an quotation which has not been closed, and parse the line
properly.
Change-Id: If0107952b9ae3144fb590f734a965d12c950c137
Reviewed-on: http://gerrit.ent.cloudera.com:8080/566
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
Rhel5x doesn't have the python ssl module installed by default. Thrift's TSSLSocket
module tries to import the ssl module. Currently, if the import fails, the exception
is not caught, making the shell unusable on Rhel5x. This change attempts to only import
TSSLSocket when the user wants an SSL-secured connection; If it's not found, the shell
exits with a warning.
Change-Id: I5cf30b2d0533b91d207a1aadb9ded7e753d7b01b
Reviewed-on: http://gerrit.ent.cloudera.com:8080/648
Tested-by: jenkins
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
exist or is uneditable.
Currently, the shell warns the user that it's unable to load the command history if the
command history file (~/.impalahistory) is not found. Moreover, if the file is not
editable, then an error is thrown after each the execution of each command. This change
disables readline if the history file is not editable instead of throwing repeated
errors, and removes the warning if the history file does not exist.
Change-Id: Ie4c94629431f2407b0679a7721a6bdf28907437f
Reviewed-on: http://gerrit.ent.cloudera.com:8080/532
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
This change has the following additions:
- If the user's connecting to a kerberized impalad, the Impala shell will check
whether a valid ticket exists by running 'klist -s'. If a valid ticket is not found,
then the shell will exit with an appropriate error message on the commandline.
- If the user's connecting to a kerberized impalad without the '-k' option, the Impala
Shell will issue a 'klist -s' to check if there are valid kerberos tickets in the
credentials cache. If a valid ticket is found, it will retry the connection with
kerberos enabled.
- The Impala shell encodes strings entered on the commandline as unicode. The sasl
module expects ascii strings as arguments. Explcitly encode any string sent to the
sasl module to ascii.
Change-Id: I1799b1e7988a19fa513b683afe1e3b66b68c1ffc
Reviewed-on: http://gerrit.ent.cloudera.com:8080/535
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
This patch allows Impala to start either Beeswax or HS2 on an
SSL-secured port. SSL is a certificate-based authentication scheme,
where the server provides a certificate to the client as part of the
handshake process. The client verifies that certificate, either by
contacting a trusted third-party certificate authority (CA), or by
accepting a 'self-signed' certificate from the server that is also
provided to the client out-of-band; the client simply compares the two
certificate copies.
Once the certificate is verified, the client and server negotiate an
encryption key for the session, using a public key provided by the
server to encrypt that negotiation. Therefore the server has to have
access to a private key in order to decrypt the encryption key.
Both certificate and key are stored in industry standard .PEM
format. Impala uses the same certificate and key for both Beeswax and
HS2, and the files containing the certificate and key are provided via
--ssl_server_certificate and --ssl_private_key. If either are non-blank,
SSL is enabled for Beeswax and HS2.
The Python shell supports SSL as of this patch via new --ssl and
--ca_cert flags.
Finally, this patch also adds support for Impala's ThriftClients to use
SSL, paving the way for having the backend service use encryption on the
wire as well (although such a configuration is not used by this
patch). The client SSL support is only currently used for the new test
case.
This patch does not enable 'mutual' authentication, where clients
provide certificates to the server in order to authenticate
themselves. Impala has other authentication mechanisms for that purpose.
Change-Id: I3942aa0d21b34b7cda748292f04a9523f35ee6d4
Reviewed-on: http://gerrit.ent.cloudera.com:8080/514
Tested-by: jenkins
Reviewed-by: Henry Robinson <henry@cloudera.com>
This adds support for CREATE TABLE AS SELECT to Impala. It supports all functionality a
regular CREATE TABLE statement includes, except it does not allow for for specifying
partition columns. Hive also has this limitation and it wouldn't be too hard to support
in the future.
Change-Id: I4ca3c3b8f1576441b8bb5ed9dc521d7dfa96ab74
Reviewed-on: http://gerrit.ent.cloudera.com:8080/157
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: Lenni Kuff <lskuff@cloudera.com>
This change adds Impala support for LOAD DATA statements. This allows the user
to load one or more files into a table or partition from a given HDFS location. The
load operation only moves files, it does not convert data to match the target
table/partition's file format.
