ext-data-source only needs a small subset of the thrift structures, so this
separates the dependencies between files so that just the necessary structs
are generated for ext-data-source. Afterwards, we can remove extra maven
dependencies which were using environment variables to get versions. While the
environment variables work when building the pom, they are not propagated to
dependencies so building fe/pom.xml ended up producing lots of warnings which
are now gone.
Change-Id: I267fe7bc7a54c3c21aad8c1ffce07cf1a1e07c5e
Reviewed-on: http://gerrit.sjc.cloudera.com:8080/3748
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: jenkins
(cherry picked from commit 1f738962ccb7a34834decfe6cb27307ed4548870)
Reviewed-on: http://gerrit.sjc.cloudera.com:8080/3767
Syntax is "CREATE TABLE name LIKE fileformat '/path/to/file'".
Supports all options that CREATE TABLE does. Currently only PARQUET is supported.
Run testdata/bin/create-load-data.sh after pulling this patch.
Change-Id: Ibb9fbb89dbde6acceb850b914c48d12f22b33f55
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2720
Reviewed-by: Victor Bittorf <victor.bittorf@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/3158
* Rename getStats() to prepare()
* Adds TRowBatch.num_rows to indicate number of rows when no cols are
materialized
* Changes api and sample poms to produce source jars
Change-Id: I02dcc89e27716978708386cfc3f7940ee5dbc023
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2406
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: jenkins
(cherry picked from commit 2d7fcba8b7442b54a388f8b994d0cfa08940bbd7)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2434
A few changes to the external data source thrift types:
* Change RowBatch to return entire columns. Adds Data.TColumnData to
represent an entire column.
* Makes all fields in ExternalDataSource (except for status fields on
the result structures) optional in case fields become deprecated in
the future.
* Adds a limit parameter to the TOpenParams structure in case the
data source needs to apply the limit itself.
Change-Id: I62db68bfb64d2190dfdd0c84be5925ad5db031ef
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2345
Reviewed-by: Nong Li <nong@cloudera.com>
Tested-by: jenkins
(cherry picked from commit faf220d628359be1368f898493900fc2e2913c53)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2385
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Adds the thrift structures for the public external data source API
and a new maven project containing the Java ExternalDataSource
interface and the generated Java thrift classes.
The ExternalDataSource.thrift structures can evolve in a backward
compatible way. The ExternalDataSource Java interface will always
contain a version number in the namespace (e.g.
com.cloudera.impala.extdatasource.v1 for V1) so we can potentially
make breaking changes to the interface in the future but still
support older versions.
A trivial implementation of the ExternalDataSource API is also
added for testing purposes.
TODO: Make the sample data source implementation realistic.
Change-Id: I827d6420a87ed7a2bce34e050362ca98ddc5dbcc
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2241
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: jenkins
(cherry picked from commit f29814e9ede9d4c889f2648606fcf511feeb47ae)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/2313
