IMPALA-8538: HS2 + HTTP(S) + BASIC/LDAP based thrift server endpoint

This patch adds an additional hiveserver2 endpoint for clients to
connect to that uses HTTP. The endpoint can be disabled by setting
--hs2_http_port=0. HTTP(S) also works when external TLS is
enabled using --ssl_server_certificate.

Thrift's http transport is modified to support BASIC authentication
via ldap. For convenience of developing and reviewing, this patch
is based on another that copied THttpServer and THttpTransport into
Impala's codebase. Kerberos authentication is not supported, so the
http endpoint is turned off if Kerberos is enabled and LDAP isn't.

TODO
=====
- Fuzz test the http endpoint
- Add tests for LDAP + HTTPS

Testing
=======
- Parameterized JdbcTest and LdapJdbcTest to work for HS2 + HTTP mode
- Added LdapHS2Test, which directly calls into the Hiveserver2
  interface using a thrift http client.

Manual testing with Beeline client (from Apache Hive), which has
builtin support to connect to HTTP(S) based HS2 compatible endpoints.

Example
========

-- HTTP mode:
> start-impala-cluster.py
> JDBC_URL="jdbc:hive2://localhost:<port>/default;transportMode=http"
> beeline -u "$JDBC_URL"

-- HTTPS mode:
> cd $IMPALA_HOME
> SSL_ARGS="--ssl_client_ca_certificate=./be/src/testutil/server-cert.pem \
    --ssl_server_certificate=./be/src/testutil/server-cert.pem \
    --ssl_private_key=./be/src/testutil/server-key.pem --hostname=localhost"
> start-impala-cluster.py --impalad_args="$SSL_ARGS" \
    --catalogd_args="$SSL_ARGS" --state_store_args="$SSL_ARGS"
- Create a local trust store using 'keytool' and import the certificate
from server-cert.pem (./clientkeystore in the example).
> JDBC_URL="jdbc:hive2://localhost:<port>/default;ssl=true;sslTrustStore= \
    ./clientkeystore;trustStorePassword=password;transportMode=http"
> beeline -u "$JDBC_URL"

-- BASIC Auth with LDAP:
> LDAP_ARGS="--enable_ldap_auth --ldap_uri='ldap://...' \
    --ldap_bind_pattern='...' --ldap_passwords_in_clear_ok"
> start-impala-cluster.py --impalad_args="$LDAP_ARGS"
> JDBC_URL="jdbc:hive2://localhost:28000/default;user=...;password=\
    ...;transportMode=http"
> beeline -u "$JDBC_URL"

-- HTTPS mode with LDAP:
> start-impala-cluster.py --impalad_args="$LDAP_ARGS $SSL_ARGS" \
    --catalogd_args="$SSL_ARGS" --state_store_args="$SSL_ARGS"
> JDBC_URL="jdbc:hive2://localhost:28000/default;user=...;password=\
    ...;ssl=true;sslTrustStore=./clientkeystore;trustStorePassword=\
    password;transportMode=http"
> beeline -u "$JDBC_URL"

Change-Id: Ic5569ac62ef3af2868b5d0581f5029dac736b2ff
Reviewed-on: http://gerrit.cloudera.org:8080/13299
Reviewed-by: Thomas Marshall <tmarshall@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Bharath Vissapragada
2019-04-16 16:02:20 -07:00
committed by Impala Public Jenkins
parent 487547ec44
commit cce709a6e3
24 changed files with 683 additions and 147 deletions

View File

@@ -411,6 +411,9 @@ error_codes = (
("DISCONNECTED_SESSION_CLOSED", 135,
"Session closed because it has no active connections"),
("UNAUTHORIZED_SESSION_USER", 136,
"The user authorized on the connection '$0' does not match the session username '$1'"),
)
import sys