IMPALA-7460 part 1: require user to install Paramiko and Fabric

- Remove Fabric and Paramiko as requirements. They aren't needed by
  anything in buildall.sh.
- Add a means to install into the impala-python virtual environment by hand.
  impala-pip is fine for this.
- Add another requirements file for extended testing. The dependency
  situation is messy and untangling that out of impala-python and into
  lib/python should be out of the scope of IMPALA-7460.
- Update core tests, which cover real regressions that have happened in
  the past, to run against locations that don't require a Paramiko
  import. This moves some logic out of concurrent_select.py into a
  thinner module.
- Insulate ssh_util from globally-scoped import so that it only imports
  when needed.

Testing:
- This works in my development environment.
- This works in my downstream stress and query gen environments.
- This works when doing a full data load.
- Impala still builds on a variety of OSs.

Todo:
- A subsequent review will update the versions.

Change-Id: Ibf9010a0387b52c95b7bda5d1d4606eba1008b65
Reviewed-on: http://gerrit.cloudera.org:8080/11264
Reviewed-by: Michael Brown <mikeb@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
Michael Brown
2018-08-20 10:24:37 -07:00
committed by Impala Public Jenkins
parent 649f175df0
commit 971cf179f6
11 changed files with 157 additions and 82 deletions

View File

@@ -19,10 +19,6 @@
# after the toolchain is bootstrapped. Installed after requirements.txt
argparse == 1.4.0
Fabric == 1.10.2
paramiko == 1.15.2
ecdsa == 0.13
pycrypto == 2.6.1
impyla == 0.14.0
bitarray == 0.8.1
sasl == 0.1.3

View File

@@ -0,0 +1,24 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# Requirements for use of test tooling (tests/comparison and tests/stress).
# IMPALA-7460 TODO: upgrade Fabric and Paramiko
Fabric == 1.10.2
paramiko == 1.15.2
ecdsa == 0.13
pycrypto == 2.6.1