Commit Graph

18 Commits

Author SHA1 Message Date
Taras Bobrovytsky
e94de02469 Added execution summary, modified benchmark to handle JSON
- Added execution summary to the beeswax client and QueryResult
- Modified report-benchmark-results to handle JSON and perform
  execution summary comparison between runs
- Added comments to the new workload runner

Change-Id: I9c3c5f2fdc5d8d1e70022c4077334bc44e3a2d1d
Reviewed-on: http://gerrit.ent.cloudera.com:8080/3598
Reviewed-by: Taras Bobrovytsky <tbobrovytsky@cloudera.com>
Tested-by: jenkins
(cherry picked from commit fd0b1406be2511c202e02fa63af94fbbe5e18eee)
Reviewed-on: http://gerrit.ent.cloudera.com:8080/3618
2014-07-25 21:06:00 -07:00
ishaan
e3a0bedfe4 Add a time constraint while evaluating regressions in performance reports.
For very short queries, using percentage as a strict bound for detecting regression does
not cut it. This introduces a flag through which the user can set a value (in seconds),
which will only attempt to detect a regression if the time difference is equal to or
greater than the value.

Change-Id: I95bde65afb8fd564666223fe069537e2483ec98e
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1498
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: jenkins
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1579
2014-02-18 10:12:36 -08:00
Nong Li
6642c377d5 Update clear buffer cache plugin to only flush page cache.
More detail: http://linux-mm.org/Drop_Caches

Change-Id: I7fa675ccdc81f375d88e9cfab330fca3bc983ec8
Reviewed-on: http://gerrit.ent.cloudera.com:8080/1157
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Reviewed-by: Lenni Kuff <lskuff@cloudera.com>
Tested-by: Nong Li <nong@cloudera.com>
2014-01-08 10:54:37 -08:00
ishaan
1780084753 Fail a performance job when a plugin fails.
Change-Id: I81fd9ec742a9efbdb16466c8ee7387fabe3e16e4
Reviewed-on: http://gerrit.ent.cloudera.com:8080/342
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
2014-01-08 10:52:31 -08:00
ishaan
b7710dfa35 Generate a diff of runtimes profiles when comparing results.
The diffs files indicate whether the query has regressed, improved, or not changeds.

Change-Id: I3e71155277208ad7f2bdef84d54f6ff64274d5af
Reviewed-on: http://gerrit.ent.cloudera.com:8080/290
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
2014-01-08 10:52:30 -08:00
ishaan
70ac30f283 Fix benchmark reporting to ignore the t-test when reference and input standard deviations
are zero.

Change-Id: I5ebc158f62f22e9efb5088a50ca1847cfa73c620
Reviewed-on: http://gerrit.ent.cloudera.com:8080/273
Reviewed-by: Ishaan Joshi <ishaan@cloudera.com>
Tested-by: Ishaan Joshi <ishaan@cloudera.com>
2014-01-08 10:52:25 -08:00
Alex Leblang
a4e441cc15 VTune Plugin
This is a plugin that runs Intel's VTune from within the Impala test plugin runner framework.

Change-Id: I673a2f6c4785bfcf565e8b002c14ada7a7c8f48a
Reviewed-on: http://gerrit.ent.cloudera.com:8080/195
Reviewed-by: Alex Leblang <alex.leblang@cloudera.com>
Tested-by: Alex Leblang <alex.leblang@cloudera.com>
2014-01-08 10:52:12 -08:00
ishaan
6f9569ea6f Add a plugin framework for run-workload 2014-01-08 10:51:39 -08:00
Lenni Kuff
2f7198292a Add support for auxiliary workloads, tests, and datasets
This change adds support for auxiliary worksloads, tests, and datasets. This is useful
to augment the regular test runs with some additional tests that do not belong in the
main Impala repo.
2014-01-08 10:50:32 -08:00
Lenni Kuff
f6a4e20947 Use parameterized queries for saving perf execution results 2014-01-08 10:49:08 -08:00
Lenni Kuff
b877240ffd Increase the 'field_size_limit' for csv reader/writer used in benchmark result processing 2014-01-08 10:49:04 -08:00
Lenni Kuff
129c6473b9 Update run-workload to gather, display, and archive runtime profiles for each workload query 2014-01-08 10:48:46 -08:00
Nong Li
6e293090e6 Parquet writer.
Change-Id: I7117b545e3d3a7803a219234ad992040a6c7c4ec
2014-01-08 10:48:44 -08:00
Lenni Kuff
4cf7d2634e Update benchmark runner to use mean of all results if num_clients > 1 2014-01-08 10:48:30 -08:00
Lenni Kuff
d3b9de2222 Add support for detecting significant performance changes (regression and improvements) 2014-01-08 10:48:19 -08:00
Lenni Kuff
4d876e01d0 Add support for storing details on number of concurrent clients in backend perf db 2014-01-08 10:47:55 -08:00
Lenni Kuff
0398efb99d Add unknown queries/workloads to perf backend if they don't already exist 2014-01-08 10:47:24 -08:00
Lenni Kuff
9953224f68 Cleanup IMPALA_HOME/bin directory
Deleted some old files and moved some files out of the bin directory into better locations
2014-01-08 10:46:55 -08:00