mirror of
https://github.com/apache/impala.git
synced 2026-01-27 06:10:53 -05:00
4e8555031b
Despite the fact that special values such as Inf and NaN are not supported in standard JSON (they are considered invalid values), rapidjson does support them. However, it requires the parsing flag 'kParseNanAndInfFlag' to be enabled. This patch enables the flag to allows JsonParser to parse special numbers like Inf and NaN. Corresponding modifications have also been made to JsonSkipper to maintain consistent behavior when zero slots scans. Additionally, due to a minor issue in rapidjson v1.1.0 when parsing Inf and NaN, this patch also updates the rapidjson version to include the corresponding fix (see https://gerrit.cloudera.org/#/c/21980/). Testing: - Added and passed relevant test cases (BE, E2E). Change-Id: I05ee7c7c7fb7e78fff9570f659ce2d13c94a4e10 Reviewed-on: http://gerrit.cloudera.org:8080/21701 Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
34 lines
993 B
Plaintext
34 lines
993 B
Plaintext
====
|
|
---- QUERY
|
|
# Testing scanning of malformed JSON, if there is a data conversion failure, the scanner
|
|
# will report the error and use a NULL value to fill the field. If the JSON format itself
|
|
# is incorrect, rapidjson will stop parsing and report the corresponding error, because
|
|
# parsing errors are difficult to recover from, we just use NULL to fill the remaining
|
|
# fields, jump to next line and continue normal parsing.
|
|
select bool_col, int_col, float_col, string_col from malformed_json
|
|
---- TYPES
|
|
boolean, int, float, string
|
|
---- RESULTS
|
|
true,0,NULL,'abc123'
|
|
NULL,NULL,NULL,'NULL'
|
|
true,2,NULL,'NULL'
|
|
false,3,0.300000011921,'NULL'
|
|
true,4,0.40000000596,'NULL'
|
|
false,5,NULL,'NULL'
|
|
true,6,NULL,'NULL'
|
|
false,7,NULL,'NULL'
|
|
true,8,0.800000011921,'abc123'
|
|
false,9,0.899999976158,'abc123'
|
|
true,10,1.0,'abc123'
|
|
NULL,NULL,NULL,'NULL'
|
|
NULL,NULL,NULL,'abc123'
|
|
NULL,NULL,NaN,'NULL'
|
|
NULL,NULL,Infinity,'NULL'
|
|
====
|
|
---- QUERY
|
|
select count(*) from malformed_json
|
|
---- TYPES
|
|
bigint
|
|
---- RESULTS
|
|
15
|
|
==== |