IMPALA-1731,IMPALA-3868: Float values are not parsed correctly

Fixed StringToFloatInternal() not to parse strings like "1.23inf"
and "infinite" with leading/trailing garbage as Infinity. These
strings are now rejected with PARSE_FAILURE.
Only "inf" and "infinity" are accepted, parsing is case-insensitive.

"NaN" values are handled similarly: strings with leading/trailing
garbage like "nana" are rejected, parsing is case-insensitive.

Other changes:
- StringToFloatInternal() was cleaned up a bit. Parsing inf and NaN
strings was moved out of the main loop.
- Use std::numeric_limits<T>::infinity() instead of INFINITY macro
and std::numeric_limits<T>::quiet_NaN() instead of NAN macro.
- Fixed another minor bug: multiple dots are allowed when parsing
float values (e.g. "2.1..6" is interpreted as 2.16).
- New BE and E2E tests were added.

Change-Id: I9e17d0f051b300a22a520ce34e276c2d4460d35e
Reviewed-on: http://gerrit.cloudera.org:8080/3791
Reviewed-by: Michael Ho <kwho@cloudera.com>
Tested-by: Internal Jenkins
This commit is contained in:
Attila Jeges
2016-07-27 16:15:05 +02:00
committed by Internal Jenkins
parent 310edd5d00
commit 211f60d831
4 changed files with 205 additions and 41 deletions

View File

@@ -2453,4 +2453,32 @@ select base64decode('abc%')
STRING
---- ERRORS
UDF WARNING: Could not base64 decode input in space 4; actual output length 0
====
====
---- QUERY
# IMPALA-1731: test parsing infinity values
select cast('inf' as double), cast('InFinity' as float),
cast('inf ' as float), cast(' infinity ' as double),
cast('infinite' as double), cast('1.23inf' as double), cast('1inf' as float)
---- RESULTS
Infinity,Infinity,Infinity,Infinity,NULL,NULL,NULL
---- TYPES
double,float,float,double,double,double,float
====
---- QUERY
# IMPALA-1731: test parsing NaN values
select cast('nan' as double), cast('NaN' as float), cast(' nan ' as double),
cast('nana' as double), cast('1.23nan' as double), cast('1nan' as float)
---- RESULTS
NaN,NaN,NaN,NULL,NULL,NULL
---- TYPES
double,float,double,double,double,float
====
---- QUERY
# IMPALA-3868: test parsing float with multiple dots
select cast('1.23' as double), cast('.1.23' as float), cast('123.456.' as double),
cast('1.23.456' as double), cast('1.23.4.5' as float), cast('0..e' as double)
---- RESULTS
1.23,NULL,NULL,NULL,NULL,NULL
---- TYPES
double,float,double,double,float,double
====