jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-19 18:12:08 -05:00
Code Issues Projects Releases Wiki Activity

IMPALA-10808: (addendum) Abort on illegal decimal parquet schemas

Browse Source 
The previous patch added checks on illegal decimal schemas of parquet
files. However, it doesn't return a non-ok status in
ParquetMetadataUtils::ValidateColumn if abort_on_error is set to false.
So we continue to use the illegal file schema and hit the DCHECK.

This patch fixes this and adding test coverage for illegal decimal
schemas.

Tests:
 - Add a bad parquet file with illegal decimal schemas.
 - Add e2e tests on the file.
 - Ran test_fuzz_decimal_tbl 100 times. Saw the errors are caught as
   expected.

Change-Id: I623f255a7f40be57bfa4ade98827842cee6f1fee
Reviewed-on: http://gerrit.cloudera.org:8080/17748
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This commit is contained in:
stiga-huang
2021-08-03 16:59:02 +08:00
committed by Impala Public Jenkins
parent 37758f0c0c
commit 599c84b4dd
8 changed files with 144 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
common/thrift/generate_error_codes.py
View File

@@ -68,7 +68,7 @@ error_codes = (
("PARQUET_WRONG_PRECISION", 17,
"File '$0' column '$1' has a precision that does not match the table metadata "
" precision. File metadata precision: $2, table metadata precision: $3."),
"precision. File metadata precision: $2, table metadata precision: $3."),
("PARQUET_BAD_CONVERTED_TYPE", 18,
"File '$0' column '$1' does not have converted type set to DECIMAL"),