Commit Graph

30 Commits

Author SHA1 Message Date
Nong Li
6e293090e6 Parquet writer.
Change-Id: I7117b545e3d3a7803a219234ad992040a6c7c4ec
2014-01-08 10:48:44 -08:00
Nong Li
868a99135a Add network benchmark 2014-01-08 10:47:56 -08:00
Alan Choi
be98df19c8 HiveServer2
This patch implements the HiveServer2  API.

We have tested it with Lenni's patch against the tpch workload. It has also
been tested manually against Hive's beeline with queries and metadata operations.

All of the HiveServer2 code is implemented in impala-hs2-server.cc. Beeswax
code is refactored to impala-beeswax-server.cc.

HiveServer2 has a few more metadata operations. These operations go through
impala-hs2-server to ddl-executor and then to FE. The logics are implemented in
fe/src/main/java/com/cloudera/impala/service/MetadataOp.java.

Because of the Thrift union issue, I have to modify the generated c++ file.
Therefore, all the HiveServer2 thrift generated c++ code are checked into
be/src/service/hiveserver2/. Once the thrift issue is resolved, I'll remove
these files.

Change-Id: I9a8fe5a09bf250ddc43584249bdc87b6da5a5881
2014-01-08 10:47:24 -08:00
Henry Robinson
7ba437a52e Code changes to build against thrift 0.9.0 in thirdparty/ 2014-01-08 10:47:22 -08:00
Henry Robinson
986f3cddf6 Move sparrow/ to statestore/ and remove sparrow namespace 2014-01-08 10:47:12 -08:00
Nong Li
2289906a5a Fix linker dependencies. 2014-01-08 10:46:56 -08:00
Henry Robinson
2f339f2ed8 Add ASL license to all public files 2014-01-08 10:46:32 -08:00
ishaan
05c65789bb Change Copyrights from 2011 ti 2012 2014-01-08 10:46:29 -08:00
Michael Ubell
ad46b98366 Add Kerberos authentication. 2014-01-08 10:45:10 -08:00
Marcel Kornacker
c004cdaa1c Thrift structures for the new planner interface. 2014-01-08 10:44:47 -08:00
Marcel Kornacker
fb32d40b03 Switching to an asynchronous plan fragment exec interface; this entails:
- making the coordinator asynchronous
- renamed ImpalaBackendService to ImpalaInternalService;
- new class ImpalaServer implements ImpalaService and ImpalaInternalService
- renaming ImpalaInternalService fields to conform to c++ style
- merged impala-service.{cc,h} and backend-service.{cc,h} into impala-server.{cc,h}
- added TStatusCode field to Status.ErrorDetail
- removed ImpalaInternalService.CloseChannel

Also removed JdbcDriverTest.java
2014-01-08 10:44:15 -08:00
Kay Ousterhout
073e38d6c2 Added the StateStore, a centralized repository for soft state.
The commit also adds the StateStoreSubscriber, a component that
runs alongside each impalad and handles communication with
the state store.
2012-07-13 09:26:16 -07:00
Alan Choi
f52286f72c This completes the Beeswax implementation for ODBC. All the ODBC tests
(CDH/hive-odbc-test) passes (except those with "create table" and "show table".

We should have nightly regression of the odbc test to run against impalad.

There're still a few issues:
1. running with num_node > 0 crashes the coordinator;
2. work around for a few ODBC jiras
3. no test for bool/timestamp because ODBC doesn't support them.

review: issue 110
2012-06-18 14:46:46 -07:00
Alan Choi
ef10afa439 This changes the Thrift from 0.6.1 to 0.7.0. Please uninstall the old thrift and download/install Thrift 0.7.0.
Beeswax service now depends on Hive metastore;
fix buildall.sh to clean generated-source in FE;
fix .gitignore to clean generated-source in BE;
2012-06-14 18:21:08 -07:00
Alan Choi
7af87c7dea Beeswax Service for Impala (partiial implementation)
review id: 82
2012-06-06 10:08:06 -07:00
Henry Robinson
3ff3559805 Add support for per-partition file formats to front end and backend.
At the same time, this patch removes the partitionKeyRegex in favour
of explicitly sending a list of literal expressions for each file path
from the front end.
2012-06-05 12:00:09 -07:00
Marcel Kornacker
4a4a07fde7 A number of changes for the Jenkins build:
- added option to run with derby metastore, based on whether env var METASTORE_IS_DERBY is set
- emoved hardwired file locations from planner tests
- switching to linking statically against libthrift.a

Also added script rebuild.sh, which contains the build steps of buildall.sh (against impala sources).
2012-03-08 16:19:47 -08:00
Nong Li
b410b62716 Add distributed profile counter for the BE. 2012-03-01 13:59:17 -08:00
Nong Li
88237350f0 Change the build to allow debug and release builds to coexist. 2012-02-17 18:14:04 -08:00
Nong Li
94db70c9fd Fix build. Dependencies don't propagate right on first build. 2011-12-30 21:28:18 -08:00
Nong Li
c84fec38d3 - Move thrift out of FE src and into impala/common
- Thrift files now build using cmake instead of mvn
- Added cmake build to impala/ which drives the build process
2011-12-30 19:35:20 -08:00
Marcel Kornacker
c056445612 Added m:n data streams:
- DataStreamSender: sender side (1:n) for a single stream
- DataStreamMgr: receiver side; singleton class for all incoming streams active at a node

Changed ExecNode::GetNext() to return eos indicator explicitly; this allows us to pass incoming TRowBatches (which may not be full) up w/o copying the data.

Added data-stream-test.
2012-01-10 18:00:20 -08:00
Alexander Behm
c7f7382c31 Added planner changes and data sinks for INSERT statements. 2011-12-12 15:14:49 -08:00
Nong Li
b1833d4de8 Implmented opcode registry. Added substr() and pi() functions. Added backend testing to buildall.sh 2011-11-20 13:44:41 -08:00
Marcel Kornacker
0914fedea9 Defining Impala backend service, which is exported by backend processes to service plan
fragment execution requests.
Changing thrift plan-related structs to pull out runtime parameters in preparation for parallel execution.
2011-11-02 14:47:32 -07:00
Marcel Kornacker
c534062c20 fixing build failure introduced by 7d9b7e2 2011-08-03 15:37:58 -07:00
Marcel Kornacker
cc141953de Adding plan service for be test driver
Adding mock implementation of libhdfs (only what's needed for text-scan-node)
in order to avoid having to make any jni calls.
Some bug fixes.
2011-07-22 12:09:55 -07:00
marcel
08e8a5db4c fixed jni and linker problems
some bug fixes and missing functions

some cleanup
2011-07-15 13:17:20 -07:00
Marcel Kornacker
c23616a30c deserializing plan request in c++
Coordinator.main(): util function to execute single query against test schema
removed dead code from TestSchemaUtils
2011-07-13 13:48:54 -07:00
marcel
3286190599 Initial version of backend. 2011-07-07 15:49:46 -07:00