test_load was using /tmp as the staging directory, which did not cleaned up in Isilon,
leading to a build failure. This patch does the following:
- use /test-warehouse as the staging directory.
- replace calls to the hdfs commandline with calls to the in-house hdfs client.
- cleanup the test file and remove duplicates.
Additionally, a new method is introduced in the hdfs client to simulate hdfs dfs -cp, i.e,
it does a get and a put to mimic the hdfs command line's semantics.
Change-Id: I0cc27ab00df5f5ec3138b995144ab45ad622605d
Reviewed-on: http://gerrit.cloudera.org:8080/431
Reviewed-by: Taras Bobrovytsky <tbobrovytsky@cloudera.com>
Tested-by: Internal Jenkins
This patch introduces changes to run tests against Isilon, combined with minor cleanup of
the test and client code.
For Isilon, it:
- Populates the SkipIfIsilon class with appropriate pytest markers.
- Introduces a new default for the hdfs client in order to connect to Isilon.
- Cleans up a few test files take the underlying filesystem into account.
- Cleans up the interface for metadata/test_insert_behaviour, query_test/test_ddl
On the client side, we introduce a wrapper around a few pywebhdfs's methods, specifically:
- delete_file_dir does not throw an error if the file does not exist.
- get_file_dir_status automatically strips the leading '/'
Change-Id: Ic630886e253e43b2daaf5adc8dedc0a271b0391f
Reviewed-on: http://gerrit.cloudera.org:8080/370
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Internal Jenkins
This change adds Impala support for LOAD DATA statements. This allows the user
to load one or more files into a table or partition from a given HDFS location. The
load operation only moves files, it does not convert data to match the target
table/partition's file format.
