Zoltan Haindrich 7131ee34d2 IMPALA-10934 (Part 1): Enable table definition over a single file ...

Implements an abstraction layer to show files in a single directory.
Impala side part - filesystem drivers are in HIVE-25569.

Suppose that the filesystem has a directory in which there are multiple
files:
 hdfs://somedir/f1.txt
 hdfs://somedir/f2.txt

In case of a HMS backed table(s) - the contents of a directory could be
considered as table.

This patch enables a new file system wrapper 'sfs+' (sfs = single file
system) which provides a view of a single file in a directory.'  The '+'
indicates that this wrapper can be added on top of multiple underlying
file systems/object storage such as HDFS, S3 etc. The directory which
contains the file could be specified:
  sfs+hdfs://somedir/f1.txt/#SINGLEFILE#

This will be a directory containing only the f1.txt and nothing else.

This patch was tested locally - with a custom build of Hive version
which also had HIVE-25569.

Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Reviewed-on: http://gerrit.cloudera.org:8080/17878
Reviewed-by: Aman Sinha <amsinha@cloudera.com>
Tested-by: Aman Sinha <amsinha@cloudera.com>

2022-01-05 03:32:11 +00:00

..

hive-exec

IMPALA-10934 (Part 1): Enable table definition over a single file

2022-01-05 03:32:11 +00:00

s3a-aws-sdk

IMPALA-10198 (part 2): Add support for mvn versions:set

2020-10-15 19:30:13 +00:00