mirror of
https://github.com/apache/impala.git
synced 2026-01-08 12:02:54 -05:00
bb1ec12c1f
This patch adds the functionality to read compressed JSON files for the JSON scanner. Because the decompression code can largely be reused from HdfsTextScanner, this patch moves that part of the code from HdfsTextScanner to HdfsScanner so that HdfsJsonScanner can also call it. As it reuses the relevant code from the TEXT scanner, the compression formats supported by the Json scanner are the same as those supported by the TEXT scanner. Tests - Most of the existing end-to-end JSON format tests can run on compressed JSON format too. Change-Id: I2471855d97d4cdd51363b321055e6b06aa6d81e8 Reviewed-on: http://gerrit.cloudera.org:8080/20482 Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
2.7 KiB
2.7 KiB
| 1 | # Generated File. |
|---|---|
| 2 | file_format: text, dataset: functional, compression_codec: none, compression_type: none |
| 3 | file_format: text, dataset: functional, compression_codec: def, compression_type: block |
| 4 | file_format: text, dataset: functional, compression_codec: gzip, compression_type: block |
| 5 | file_format: text, dataset: functional, compression_codec: bzip, compression_type: block |
| 6 | file_format: text, dataset: functional, compression_codec: snap, compression_type: block |
| 7 | file_format: seq, dataset: functional, compression_codec: none, compression_type: none |
| 8 | file_format: seq, dataset: functional, compression_codec: def, compression_type: block |
| 9 | file_format: seq, dataset: functional, compression_codec: def, compression_type: record |
| 10 | file_format: seq, dataset: functional, compression_codec: gzip, compression_type: block |
| 11 | file_format: seq, dataset: functional, compression_codec: gzip, compression_type: record |
| 12 | file_format: seq, dataset: functional, compression_codec: bzip, compression_type: block |
| 13 | file_format: seq, dataset: functional, compression_codec: bzip, compression_type: record |
| 14 | file_format: seq, dataset: functional, compression_codec: snap, compression_type: block |
| 15 | file_format: seq, dataset: functional, compression_codec: snap, compression_type: record |
| 16 | file_format: rc, dataset: functional, compression_codec: none, compression_type: none |
| 17 | file_format: rc, dataset: functional, compression_codec: def, compression_type: block |
| 18 | file_format: rc, dataset: functional, compression_codec: gzip, compression_type: block |
| 19 | file_format: rc, dataset: functional, compression_codec: bzip, compression_type: block |
| 20 | file_format: rc, dataset: functional, compression_codec: snap, compression_type: block |
| 21 | file_format: avro, dataset: functional, compression_codec: none, compression_type: none |
| 22 | file_format: avro, dataset: functional, compression_codec: def, compression_type: block |
| 23 | file_format: avro, dataset: functional, compression_codec: snap, compression_type: block |
| 24 | file_format: parquet, dataset: functional, compression_codec: none, compression_type: none |
| 25 | file_format: orc, dataset: functional, compression_codec: def, compression_type: block |
| 26 | file_format: hbase, dataset: functional, compression_codec: none, compression_type: none |
| 27 | file_format: kudu, dataset: functional, compression_codec: none, compression_type: none |
| 28 | file_format: json, dataset: functional, compression_codec: none, compression_type: none |
| 29 | file_format: json, dataset: functional, compression_codec: def, compression_type: block |
| 30 | file_format: json, dataset: functional, compression_codec: gzip, compression_type: block |
| 31 | file_format: json, dataset: functional, compression_codec: bzip, compression_type: block |
| 32 | file_format: json, dataset: functional, compression_codec: snap, compression_type: block |