156 Commits

Author SHA1 Message Date
Skye Wanderman-Milne
7d29de16bb Rewrite decimal functions/operators as UDFs.
This patch also adds a GetReturnType() method to FunctionContext. This
is staging for the expr refactoring.

Change-Id: I854e79ded409e151663c4ec99c4e08631ad9e03e
Reviewed-on: http://gerrit.sjc.cloudera.com:8080/3234
Reviewed-by: Skye Wanderman-Milne <skye@cloudera.com>
Tested-by: Skye Wanderman-Milne <skye@cloudera.com>
2014-08-17 12:44:18 -07:00
Paden Tomasello
879a40913c Implemented UDFs for timestamp functions.
FromUtc and ToUtc use thirdparty libraries which use inline asm which
isn't currently supported with JIT. The UDFs are included in this
commit, but the function symbols were not changed in
impala_functions.py

Change-Id: I0824a434d4a26a39abf29bc6e47d51b5ad7991d6
Reviewed-on: http://gerrit.ent.cloudera.com:8080/3390
Reviewed-by: Paden Tomasello <paden.tomasello@cloudera.com>
Tested-by: jenkins
(cherry picked from commit 8e149ccd78010b7a22d6fff1b0de5614848b02ac)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/3548
2014-07-21 15:27:46 -07:00
Victor Bittorf
778fba232c Adding MADlib vector operations.
These are basic UDFs which the user will need to interact with MADlib.

Change-Id: Iadcec2376e29d2f73f1bb04d5e695c58d9381952
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2368
Reviewed-by: Victor Bittorf <victor.bittorf@cloudera.com>
Tested-by: jenkins
(cherry picked from commit 2eac4ec1287e354c26c2b8916515307fe793fb93)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/3160
2014-06-19 14:48:59 -07:00
Skye Wanderman-Milne
a618d34f17 More decimal builtins.
Change-Id: Ie5b89ad7d1fc80fa646f7cf5f520db13b25b9565
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2764
Reviewed-by: Skye Wanderman-Milne <skye@cloudera.com>
Tested-by: jenkins
(cherry picked from commit 6e994ce7712047000d3a12b5eb677b5470687370)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2830
2014-06-06 19:42:45 -07:00
Nong Li
8f4dc0f2f0 IMPALA-974: Switch from FloatLiteral to DecimalLiteral.
Float/Doubles are lossy so using those as the default literal type
is problematic.

Change-Id: I5a619dd931d576e2e6cd7774139e9bafb9452db9
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2758
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
2014-05-31 22:19:06 -07:00
Nong Li
26ca559f38 Add decimal builtins: abs/round/ceil/floor/truncate.
Change-Id: I4fe0ee69475ff56d3dc0cd69ea21f677714ae8bc
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2748
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
2014-05-30 11:53:06 -07:00
Victor Bittorf
0bb66ef327 Adding aliases ADD_MONTHS and SUB_MONTHS
This is a request for consistency with oracle.

Change-Id: I463a66694a068cd773532d8f6f853a4b089b918a
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2400
Reviewed-by: Victor Bittorf <victor.bittorf@cloudera.com>
Tested-by: jenkins
(cherry picked from commit 1f0b643789596f96c54580b8c5262fada4dfc958)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2502
2014-05-09 17:35:29 -07:00
Henry Robinson
38befd2126 IMPALA-724: Support infinite / nan values in text files
This patch allows the text scanner to read 'inf' or 'Infinity' from a
row and correctly translate it into floating-point infinity. It also
adds is_inf() and is_nan() builtins.

Finally, we change the text table writer to write Infinity and NaN for
compatibility with Hive.

In the future, we might consider adding nan / inf literals to our
grammar (postgres has this, see:
http://www.postgresql.org/docs/9.3/static/datatype-numeric.html).

Change-Id: I796f2852b3c6c3b72e9aae9dd5ad228d188a6ea3
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2393
Reviewed-by: Henry Robinson <henry@cloudera.com>
Tested-by: jenkins
(cherry picked from commit 58091355142cadd2b74874d9aa7c8ab6bf3efe2f)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2483
2014-05-08 12:28:53 -07:00
Henry Robinson
35d986cc8d Add max / min_*int() builtins
It would have been convenient today to know the largest values that
Impala accepts for its integer types. This patch adds max and min
builtins for our numeric types.

[localhost:21000] > select max_bigint(), max_int(), max_smallint(),
max_tinyint();
Query: select max_bigint(), max_int(), max_smallint(), max_tinyint()
+---------------------+------------+----------------+---------------+
| max_bigint()        | max_int()  | max_smallint() | max_tinyint() |
+---------------------+------------+----------------+---------------+
| 9223372036854775807 | 2147483647 | 32767          | 127           |
+---------------------+------------+----------------+---------------+

Change-Id: I6df6df2728197529c6375dbb1b7d3c9ddb9833d2
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2381
Reviewed-by: Henry Robinson <henry@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2398
2014-04-29 16:54:31 -07:00
Victor Bittorf
46151dc7dd Adding EXTRACT builtin.
Change-Id: I6de20f336ecdfa3acd8d3a9166cff4a062baaacc
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2247
Reviewed-by: Victor Bittorf <victor.bittorf@cloudera.com>
Tested-by: jenkins
(cherry picked from commit f233955020ffbd1023f2d6adbbfb22e267986305)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2370
2014-04-25 15:38:51 -07:00
Victor Bittorf
c414c91931 Adding TRUNC builtin.
Includes additions to builtin UDF registration to support prepare/close.

Change-Id: I22668fa7ee033b3fa37050b7bccee935571ac453
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2243
Reviewed-by: Victor Bittorf <victor.bittorf@cloudera.com>
Tested-by: jenkins
2014-04-22 13:17:12 -07:00
Nong Li
87295a4e06 Decimal implementation.
This patch implements decimal support for text based formats.

Change-Id: I8e2c9e512ed149fe965216a72cb21fffd4f18e75
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1669
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: Nong Li <nong@cloudera.com>
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2238
Tested-by: jenkins
2014-04-14 21:07:32 -07:00
Srinath Shankar
23d394c572 IMPALA-1238: Impala supports length() but not char_length()
Adds aliases char_length and character_length for the length()
 builtin. These are valid for ASCII.

Change-Id: I934d997f2c6d372ed12e7221efc1a574d68e01f3
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1802
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1827
Reviewed-by: Srinath Shankar <sshankar@cloudera.com>
2014-03-08 16:53:49 -08:00
Srinath Shankar
985d90146e IMPALA-856:NULLIFZERO and ZEROIFNULL
Both nullifzero and zeroifnull apply only to numeric types.
nullifzero(arg) - Return NULL if arg == 0, arg otherwise
zeroifnull(arg) - Return 0 if arg is NULL, arg otherwise

Change-Id: I41260de1edca2f9fcf50594fe137ca1f68f76056
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1805
Reviewed-by: Srinath Shankar <sshankar@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1826
2014-03-08 13:09:33 -08:00
Nong Li
f0a67153d3 Decimal analysis changes.
Change-Id: Ib7d6a6a7650cc9058ff1486fc7546ab66c698d46
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1734
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
2014-03-03 21:15:00 -08:00
Nong Li
0d2919fe7f Refactor scalar and aggregate function analysis and execution.
This patch cleans up analysis and execution of scalar and aggregate functions
so that there is no difference between how builtins and user functions are
handled. The only difference is that the catalog is populated with the builtins
all the time.

The BE always gets a TFunction object and just executes it (builtins will have
an empty hdfs file location).

This removes the opcode registry and all of the functionality is subsumed by
the catalog, most of which was already duplicated there anyway.

This also introduces the concept of a system database; databases that the
user cannot modify and is populated automatically on startup.

Change-Id: Iaa3f84dad0a1a57691f5c7d8df7305faf01d70ed
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1386
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1577
2014-02-18 18:40:08 -08:00
Srinath Shankar
633d0e82e9 IMPALA-730: Implementation of NULLIF
Change-Id: I5df4261924e591ae0e9d707e9c75b350aac3f106
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1250
Reviewed-by: Srinath Shankar <sshankar@cloudera.com>
Tested-by: jenkins
2014-01-14 20:21:22 -08:00
Lenni Kuff
0ce83818a6 IMPALA-675: Add function to get current default database
This uses the same syntax as postgres: current_database()

Change-Id: Ic6ca8ce1fe8c10a496800c45c58b0df4d214b51c
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1274
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: jenkins
2014-01-13 22:27:22 -08:00
Matthew Jacobs
967346b0c4 IMPALA-630: Add fn to get the PID of the impalad to which the user is connected
Change-Id: I2d8b304bfb22883489bbbbe33e07478d164583b9
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1127
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: Matthew Jacobs <mj@cloudera.com>
2014-01-08 10:54:37 -08:00
Alex Behm
680c6c1a4c Added additional signatures to a few functions where implicit casting was annoying.
Functions modified: coalesce, nvl, greatest, least, positive, negative.

Change-Id: Ia9436a2cd392db2a81f8dec572d90c5f3be6f7c7
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1066
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: jenkins
2014-01-08 10:54:28 -08:00
Matthew Jacobs
2ea5ed5069 IMPALA-478: Create a Hash builtin function
Adds a builtin FNV hash function. Also renames HashUtil::FvnHash to HashUtil::FnvHash
since it was spelled incorrectly.

Change-Id: Ic6dbfbce58ceeded72442ff22d3cd04f1010ea78
Reviewed-on: http://gerrit.ent.cloudera.com:8080/995
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: jenkins
2014-01-08 10:54:15 -08:00
Skye Wanderman-Milne
656ae8b1c8 Cross-compiled UDF builtins.
When codegen is enabled, UDF builtins will be loaded from the IR
module rather than using the native functions. Since we cannot run
UDFs without codegen yet this means UDF builtins can only be run this
way, but once we add support for running UDFs without codegen this
will allow us to switch back to the native functions for
development/debugging.

Change-Id: I948b113c61603801b84f80982384bbc07596f119
Reviewed-on: http://gerrit.ent.cloudera.com:8080/605
Tested-by: jenkins
Reviewed-by: Nong Li <nong@cloudera.com>
2014-01-08 10:53:13 -08:00
Chris Channing
ba7c764279 IMP-651: Adding support for the greatest function.
Change-Id: Ia8c53db5504e28d8669e6013545da6b1164bcb23
Reviewed-on: http://gerrit.ent.cloudera.com:8080/570
Tested-by: jenkins
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
2014-01-08 10:53:07 -08:00
Chris Channing
dc055c57ff IMP-651: Adding support for the least function.
Change-Id: I51c12bdd2ed614e2885403b4f857abe7d8e5777c
Reviewed-on: http://gerrit.ent.cloudera.com:8080/552
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: Alex Behm <alex.behm@cloudera.com>
2014-01-08 10:53:07 -08:00
Nong Li
e959e49b7c Update opcode registry to support UDF-interface builtins.
There's a bigger change to migrate the rest of them but I think this is how
the builtins, when not running as cross compiled, should be run. This mode
is still useful when developing the builtin.

When run as cross compiled IR, we wouldn't do anything to distinguish between
a builtin and an external UDF.

Change-Id: I6aa336b22aa19b00507bad33c9df3978baa576cc
Reviewed-on: http://gerrit.ent.cloudera.com:8080/542
Tested-by: jenkins
Reviewed-by: Skye Wanderman-Milne <skye@cloudera.com>
2014-01-08 10:53:06 -08:00
Chris Channing
c81da10994 IMP-652: Adding support for the initcap function.
Change-Id: Icbfb7ceb83cc02ad9cccd2b199beab21bb29c5c7
Reviewed-on: http://gerrit.ent.cloudera.com:8080/502
Tested-by: jenkins
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
2014-01-08 10:52:58 -08:00
Alex Behm
79cee63e9b IMPALA-431: from_unixtime does not support BIGINT.
Change-Id: I64e1e06d179c7c11199ea535f82f6e880dd0fc6e
Reviewed-on: http://gerrit.ent.cloudera.com:8080/103
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: Lenni Kuff <lskuff@cloudera.com>
2014-01-08 10:52:01 -08:00
Alex Behm
bba0b2bc48 IMPALA-498: Added modulo for floats.
Change-Id: I51ae6a3bc6d27efc519c632fa0c86c23bea73d50
Reviewed-on: http://gerrit.ent.cloudera.com:8080/104
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: Lenni Kuff <lskuff@cloudera.com>
2014-01-08 10:52:00 -08:00
Alex Behm
65b11c61d4 IMPALA-463: Added dayname() function. 2014-01-08 10:51:57 -08:00
Alex Behm
3696e6a4b4 Added user() utility function. 2014-01-08 10:51:44 -08:00
Alex Behm
61b6c68ea3 IMPALA-455: Add function aliases for IsNull. 2014-01-08 10:51:41 -08:00
Nong Li
1e66f36581 Add translate builtin. 2014-01-08 10:51:03 -08:00
Nong Li
b3bd330cd1 Add dayofweek builtin. 2014-01-08 10:51:03 -08:00
Alex Behm
d173812f64 Added quotient() function. Fixed int divide and mod by zero. 2014-01-08 10:50:47 -08:00
Alex Behm
816141b9f5 Allow BIGINT type in functions that currently only accept INT. 2014-01-08 10:50:46 -08:00
Alex Behm
27f3840f83 Added IsNull() function. 2014-01-08 10:50:31 -08:00
Alex Behm
861ba05989 IMPALA-197: Outer join on constant expressions returns incorrect results. 2014-01-08 10:50:09 -08:00
Skye Wanderman-Milne
b872db2361 IMPALA-263: Add sleep(int ms) builtin 2014-01-08 10:49:59 -08:00
Skye Wanderman-Milne
60a4c823b5 IMPALA-245: day() function returns wrong result
Changes day() to return same result as dayofmonth() and adds dayofyear().
2014-01-08 10:49:46 -08:00
Skye Wanderman-Milne
982747c856 IMP-653: add CURRENT_TIMESTAMP() function as synonym for now() 2014-01-08 10:47:09 -08:00
Henry Robinson
2f339f2ed8 Add ASL license to all public files 2014-01-08 10:46:32 -08:00
ishaan
ccb020c4a0 Adding copyrights to remaining files. 2014-01-08 10:46:30 -08:00
Michael Ubell
c1852e2dcf Add from_unixtime and unix_timestamp(string, string) 2014-01-08 10:46:22 -08:00
Michael Ubell
48c454d319 IMP-267 Add version() function. 2014-01-08 10:45:57 -08:00
Alexander Behm
ee705e3083 Added timestamp arithmetic expressions. 2014-01-08 10:44:31 -08:00
Alexander Behm
5a92fee31c Added now() function. 2014-01-08 10:44:19 -08:00
Michael Ubell
78fd2c5d6d IMP-106 IMP-107, support of date and time strings. 2012-07-16 15:43:40 -07:00
Nong Li
2b119d552e LLVM Expr codegen cleanup. IR optimization passes now mimick clang passes. 2012-04-18 16:57:13 -07:00
Alexander Behm
cf9cbfe996 Added conditional functions. Disabled some tests due to issues with NULL and float literals. 2012-04-03 14:22:52 -07:00
Alexander Behm
4be1d84782 Added parse_url string function. 2012-03-26 22:24:53 -07:00