Commit Graph

103 Commits

Author SHA1 Message Date
Matthew Jacobs
41d90312fa Admission controller: user to pool resolution, authorization, and pool configs
Adds RequestPoolUtils which exposes user to pool resolution, authorization,
and relevant pool configurations by wrapping Yarn classes that provide that
functionality. (To support CDH4, those Yarn classes will come from
thirdparty/cdh4-extras.) RequestPoolUtils is created once by the backend and
the instance lives for the duration of the process.

Change-Id: I53db075555578614356d33f9d939c5378b9ec797
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1566
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: jenkins
(cherry picked from commit 8e385bdb54ed97e567c672a76723936c24cfe45f)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1758
2014-03-06 14:21:31 -08:00
Matthew Jacobs
8ac929f095 Admission controller: use request memory estimate in admission
Change-Id: I86ba26df434e9297b11abe349ea237fea9b04b87
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1622
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: jenkins
(cherry picked from commit fc0b3f8c289f39fa07816e1cd9e7b0484b845470)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1689
2014-02-27 13:34:29 -08:00
Matthew Jacobs
9156cb94ca Admission controller functional tests
The test works by submitting a number of queries (parameterized) with
some delay between submissions (parameterized) and the ability to
submit to one impalad or many. The queries are set with the WAIT debug
action so that we have more control over the state that the admission
controller uses to make decisions.  Each query is submitted on a
separate thread. Depending on the test parameters a varying number of
queries will be admitted, queued, and rejected. Once queries are
admitted, the query execution blocks and we can cancel the query in
order to allow another queued query to be admitted. The test tracks
the state of the admission controller using metric counters on each
impalad.

Change-Id: I455484a7f899032890b22c38592fcea1875f5399
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1413
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: jenkins
(cherry picked from commit bc2a74d6da622de877422f926ff1892bed867bb1)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1624
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: Matthew Jacobs <mj@cloudera.com>
2014-02-20 14:48:30 -08:00