This patch reworks our Kerberos authentication layer to support multiple
authentication protocols, particularly PLAIN/SASL to support external
LDAP authentication.
There is now a system-wide AuthManager object, initialised by InitAuth()
which occurs during the usual InitCommonRuntime() setup. The AuthManager
is responsible for supplying AuthProvider objects to ThriftServers and
ThriftClients. The AuthProvider in turn generates Thrift transport
objects which are usually SASL-enabled, and which either employ GSSAPI
or PLAIN mechanisms.
In miscellaneous changes:
* Cyrus SASL now builds both with LDAP and the dummy '--enable-true'
external authentication mechanisms enabled.
* To test PLAIN/SASL authentication, you must now include
$IMPALA_HOME/thirdparty/${IMPALA_CYRUS_SASL_VERSION}/build/lib/sasl2 in
FLAGS_sasl_path.
* The shell now has an option to authenticate using LDAP, and will
prompt for a password at startup before doing so.
* Since the authentication code is almost entirely Thrift-specific, it
has been moved to the rpc lib.
Change-Id: I771de50f05630efdf1606ab9f0f48146ad54595e
Reviewed-on: http://gerrit.ent.cloudera.com:8080/716
Tested-by: jenkins
Reviewed-by: Henry Robinson <henry@cloudera.com>
quotation ending in a semi-colon.
Currently, the shell cannot parse a line which has an open quotation ending in a
semi-colon. It treats the semi-colon as a command terminator. With this change, the shell
will be able to detect an quotation which has not been closed, and parse the line
properly.
Change-Id: If0107952b9ae3144fb590f734a965d12c950c137
Reviewed-on: http://gerrit.ent.cloudera.com:8080/566
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
Rhel5x doesn't have the python ssl module installed by default. Thrift's TSSLSocket
module tries to import the ssl module. Currently, if the import fails, the exception
is not caught, making the shell unusable on Rhel5x. This change attempts to only import
TSSLSocket when the user wants an SSL-secured connection; If it's not found, the shell
exits with a warning.
Change-Id: I5cf30b2d0533b91d207a1aadb9ded7e753d7b01b
Reviewed-on: http://gerrit.ent.cloudera.com:8080/648
Tested-by: jenkins
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
exist or is uneditable.
Currently, the shell warns the user that it's unable to load the command history if the
command history file (~/.impalahistory) is not found. Moreover, if the file is not
editable, then an error is thrown after each the execution of each command. This change
disables readline if the history file is not editable instead of throwing repeated
errors, and removes the warning if the history file does not exist.
Change-Id: Ie4c94629431f2407b0679a7721a6bdf28907437f
Reviewed-on: http://gerrit.ent.cloudera.com:8080/532
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
This change has the following additions:
- If the user's connecting to a kerberized impalad, the Impala shell will check
whether a valid ticket exists by running 'klist -s'. If a valid ticket is not found,
then the shell will exit with an appropriate error message on the commandline.
- If the user's connecting to a kerberized impalad without the '-k' option, the Impala
Shell will issue a 'klist -s' to check if there are valid kerberos tickets in the
credentials cache. If a valid ticket is found, it will retry the connection with
kerberos enabled.
- The Impala shell encodes strings entered on the commandline as unicode. The sasl
module expects ascii strings as arguments. Explcitly encode any string sent to the
sasl module to ascii.
Change-Id: I1799b1e7988a19fa513b683afe1e3b66b68c1ffc
Reviewed-on: http://gerrit.ent.cloudera.com:8080/535
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
This patch allows Impala to start either Beeswax or HS2 on an
SSL-secured port. SSL is a certificate-based authentication scheme,
where the server provides a certificate to the client as part of the
handshake process. The client verifies that certificate, either by
contacting a trusted third-party certificate authority (CA), or by
accepting a 'self-signed' certificate from the server that is also
provided to the client out-of-band; the client simply compares the two
certificate copies.
Once the certificate is verified, the client and server negotiate an
encryption key for the session, using a public key provided by the
server to encrypt that negotiation. Therefore the server has to have
access to a private key in order to decrypt the encryption key.
Both certificate and key are stored in industry standard .PEM
format. Impala uses the same certificate and key for both Beeswax and
HS2, and the files containing the certificate and key are provided via
--ssl_server_certificate and --ssl_private_key. If either are non-blank,
SSL is enabled for Beeswax and HS2.
The Python shell supports SSL as of this patch via new --ssl and
--ca_cert flags.
Finally, this patch also adds support for Impala's ThriftClients to use
SSL, paving the way for having the backend service use encryption on the
wire as well (although such a configuration is not used by this
patch). The client SSL support is only currently used for the new test
case.
This patch does not enable 'mutual' authentication, where clients
provide certificates to the server in order to authenticate
themselves. Impala has other authentication mechanisms for that purpose.
Change-Id: I3942aa0d21b34b7cda748292f04a9523f35ee6d4
Reviewed-on: http://gerrit.ent.cloudera.com:8080/514
Tested-by: jenkins
Reviewed-by: Henry Robinson <henry@cloudera.com>
This adds support for CREATE TABLE AS SELECT to Impala. It supports all functionality a
regular CREATE TABLE statement includes, except it does not allow for for specifying
partition columns. Hive also has this limitation and it wouldn't be too hard to support
in the future.
Change-Id: I4ca3c3b8f1576441b8bb5ed9dc521d7dfa96ab74
Reviewed-on: http://gerrit.ent.cloudera.com:8080/157
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: Lenni Kuff <lskuff@cloudera.com>
This change adds Impala support for LOAD DATA statements. This allows the user
to load one or more files into a table or partition from a given HDFS location. The
load operation only moves files, it does not convert data to match the target
table/partition's file format.
This patch adds support for
- ALTER TABLE ADD|REPLACE COLUMNS
- ALTER TABLE DROP COLUMN
- ALTER TABLE ADD/DROP PARTITION
- ALTER TABLE SET FILEFORMAT
- ALTER TABLE SET LOCATION
- ALTER TABLE RENAME
