IMPALA-9709: Remove Impala-lzo from the development environment

This removes Impala-lzo from the Impala development environment.
Impala-lzo is not built as part of the Impala build. The LZO plugin
is no longer loaded. LZO tables are not loaded during dataload,
and LZO is no longer tested.

This removes some obsolete scan APIs that were only used by Impala-lzo.
With this commit, Impala-lzo would require code changes to build
against Impala.

The plugin infrastructure is not removed, and this leaves some
LZO support code in place. If someone were to decide to revive
Impala-lzo, they would still be able to load it as a plugin
and get the same functionality as before. This plugin support
may be removed later.

Testing:
 - Dryrun of GVO
 - Modified TestPartitionMetadataUncompressedTextOnly's
   test_unsupported_text_compression() to add LZO case

Change-Id: I3a4f12247d8872b7e14c9feb4b2c58cfd60d4c0e
Reviewed-on: http://gerrit.cloudera.org:8080/15814
Reviewed-by: Bikramjeet Vig <bikramjeet.vig@cloudera.com>
Tested-by: Joe McDonnell <joemcdonnell@cloudera.com>
This commit is contained in:
Joe McDonnell
2020-04-26 18:38:26 -07:00
parent 38b9617462
commit f15a311065
65 changed files with 88 additions and 346 deletions

View File

@@ -44,8 +44,7 @@ COMPRESSION_VALUES_MAP = {
"parquet": {
"none": "SNAPPY",
"snap": "SNAPPY",
"gzip": "GZIP",
"lzo": "LZO"
"gzip": "GZIP"
},
# Currently, only three codecs are supported in Hive for ORC. See Hive codes in
# org.apache.orc.impl.WriterImpl#createCodec (in module hive-orc)
@@ -397,7 +396,7 @@ if __name__ == "__main__":
source_db = args.source_db
target_db = args.target_db
file_format, compression_value = args.table_format.split("/")
# 'compression_value' is one of [none,def,gzip,bzip,snap,lzo]. We should translate it
# 'compression_value' is one of [none,def,gzip,bzip,snap]. We should translate it
# into values that can be set to Hive.
if file_format not in COMPRESSION_KEYS_MAP:
raise Exception("Nested types in file format %s are not supported" % file_format)