This change corrects the improper rendering of nanosecond walltime
event timestamps in the text profile section of imported query profiles.
The timestamps are now displayed in minutes and seconds, instead of
being displayed in date format. (i.e. 2s120ms, 2ms498us)
The navbar rendering from incorrect declaration in webUI ES6 refactor
IMPALA-13389 has also been corrected.
Incorrectly added columns "Coordinator Slots" and "Executor Slots" in
IMPALA-13726: Add admission control slots to /queries page in webui
have been removed from the imported query profile section.
Change-Id: Id1ad10f469aec085e5b485b4c20d6ab89fe58034
Reviewed-on: http://gerrit.cloudera.org:8080/23067
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Currently, the scripts in the entire webUI contain variable and function
declarations using the ES5 standard.
In such declarations, all the variables are attached to the browser's
window object, polluting the global scope. Additionally, many unnamed
functions can be represented with cleaner and concise syntax.
This patch refactors such declarations, using the ES6 syntax.
-> Replacing 'var' declarations with 'let' and 'const'
- To improve browser's memory utilization
- For better scoping, immutability and readability
-> Replacing unnamed function declarations with arrow functions
- Better scoping by binding 'this' object to the surrounding context
These improve maintainability and browser's memory utilization.
Across many instances within the webUI scripts, no particular naming
scheme is being followed for variable and function identifiers.
Hence, they have been revised with the following naming scheme.
-> All function names have been declared using camel case.
-> All constant primitive values and strings have been declared
in uppercase.
-> All other types of variables have been declared using snake case.
This naming scheme allows easier distinction between functions,
constants and other variables, based on the identifiers.
These changes to code style are further enforced during code review
through IMPALA-13473.
Change-Id: Ie38f2c642ede14956a2c6d551a58e42538204768
Reviewed-on: http://gerrit.cloudera.org:8080/21851
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
This patch qualified some urls in template files.
Testing:
- Manully checked importing json profiles worked correctly with the
fix.
- Added new regex checks to test_knox_compatibility.
Change-Id: If53947c3b8429840990b361a936ba31e1df21522
Reviewed-on: http://gerrit.cloudera.org:8080/21574
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Imported query profiles are currently being stored in IndexedDB.
Although IndexedDB does not have storage limitations like other
browser storage APIs, there is a storage limit for a single
attribute / field.
For supporting larger query profiles, 'pako' compression library's
v2.1.0 has been added along with its associated license.
Before adding query profile JSON to indexedDB, it undergoes compression
using this library.
As compression and parsing profile is a long running process
that can block the main thread, this has been delegated to
a worker script running in the background. The worker script
returns parsed query attributes and compressed profile text sent to it.
The process of compression consumes time; hence, an alert message is
displayed on the queries page warning user to refrain from closing or
reloading the page. On completion, the raw total size, compressed
total size, and total processing time are logged to the browser console.
When multiple profiles are chosen, after each query profile insertion,
the subsequent one is not triggered until compression and insertion
are finished.
The inserted query profile field is decompressed before parsing on
the query plan, query profile, query statement, and query timeline page.
Added tests for the compression library methods utilized by
the worker script.
Manual testing has been done on Firefox 126.0.1 and Chrome 126.0.6478.
Change-Id: I8c4f31beb9cac89051460bf764b6d50c3933bd03
Reviewed-on: http://gerrit.cloudera.org:8080/21463
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
For query profile imports currently the following tabs are supported.
- Query Statement
- Query Timeline
- Query Text Plan
With the current patch "Query Profile" tab will also be supported.
In the "QueryProfileHandler", "query_id" is now added before verifying
its existence in the query log as in "QuerySummaryHandler" and others.
"getQueryID" function has been added to "util.js", as it is helpful
across multiple query pages for retrieving the query ID into JS scripts,
before the page loads up.
On loading the imported "Query Profile" page, query profile download
section and server's non-existing query ID alerts are removed.
All unsupported navbar tabs are removed and current tab is set to active.
The query profile is retrieved from the indexedDB's "imported_queries"
database. Then query profile is passed onto "profileToString" function,
which converts the profile into indented text for displaying on the
profile page.
Each profile and its child profiles are printed in the following order
with the right indentation(fields are skipped, if they do not exist).
Profile name:
- Info strings:
- Event sequences:
- Offset:
- Events:
- Child profile(recursive):
- Counters:
Change-Id: Iddcf2e285abbf42f97bde19014be076ccd6374bc
Reviewed-on: http://gerrit.cloudera.org:8080/21400
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
A button has been added to import query profiles on the queries page.
The button opens a system dialog to browse only "*.json" files from
client's local file system. This allows multiple JSON query profiles
to be imported.
JSON profiles are read from file system using FileReader API and then
these imported query profiles are stored in a NoSQL database using
browser's IndexedDB API.
If an error is encountered while parsing JSON profile, it is displayed
on the queries page.
IndexedDB is a large-scale NoSQL database system, that can be read
and written to asynchronously. Hence, the file size and number of JSON
profiles is only limited by the storage available on the client system.
The storage limit is browser dependent, in chromium based browsers
upto 80% of local storage may be used by IndexedDB.
The profile is parsed for the following attributes, which will be
displayed on the queries page. Number of imported queries is also
displayed.
- Query ID
- User
- Default Db
- Query Type
- Start Time
- End Time
- Bytes Read
- Bytes Sent
- Query State
- No. of rows fetched
- Resource Pool
- Query Statement
After importing or deleting query profiles, page will scroll back
to the imported query profiles section.
The queries are sorted based on descending order of the "Start Time".
A button is displayed beside the import button to easily clear all
the imported queries at once.
On opening any of the imported queries, its query timeline will be
displayed. Currently the following tabs are supported for imported
queries.
- Query Statement
- Query Timeline
- Query Text Plan
Syntax highlighting on query statement tab has also been supported.
Change-Id: Ife6eb59bf2030fd19fc92aaf134eb51c609e04d0
Reviewed-on: http://gerrit.cloudera.org:8080/20867
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
After upgrading Bootstrap to 4.3.1 in the previous patch, the query page
no longer highlights which tab is being used. This is due to the removal
of ".nav-tabs > li.active > a" in bootstrap.css. So li elements in
"active" class no longer has additional styles. Instead, the styles of
"active" class is moved into to "nav-link" class in Bootstrap 4. We
should add the "active" class in the "nav-link" elements instead.
Tests:
- Manually click throught all tabs to check the active state
Change-Id: I5c746bc4f49b30df54bd3c73ac6f1cf838260c65
Reviewed-on: http://gerrit.cloudera.org:8080/14582
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
For files that have a Cloudera copyright (and no other copyright
notice), make changes to follow the ASF source file header policy here:
http://www.apache.org/legal/src-headers.html#headers
Specifically:
1) Remove the Cloudera copyright.
2) Modify NOTICE.txt according to
http://www.apache.org/legal/src-headers.html#notice
to follow that format and add a line for Cloudera.
3) Replace or add the existing ASF license text with the one given
on the website.
Much of this change was automatically generated via:
git grep -li 'Copyright.*Cloudera' > modified_files.txt
cat modified_files.txt | xargs perl -n -i -e 'print unless m#Copyright.*Cloudera#i;'
cat modified_files_txt | xargs fix_apache_license.py [1]
Some manual fixups were performed following those steps, especially when
license text was completely missing from the file.
[1] https://gist.github.com/anonymous/ff71292094362fc5c594 with minor
modification to ORIG_LICENSE to match Impala's license text.
Change-Id: I2e0bd8420945b953e1b806041bea4d72a3943d86
Reviewed-on: http://gerrit.cloudera.org:8080/3779
Reviewed-by: Dan Hecht <dhecht@cloudera.com>
Tested-by: Internal Jenkins
This patch adds the /query_plan endpoint, which uses the Dagre and D3 JS
libraries to render the plan tree. The tree auto-refreshes once every
second, and the edges are labelled with the output cardinalities of each
operator. Plan fragment boundaries are marked by a different edge
colour, and nodes that belong to the same fragment are all coloured the
same.
This patch also restructures the summary and profile pages under one new
'details' heading that gives the user a selection of tabs to choose
between, each with some relevant query information. Those tabs contain
the plan tree, the query statement, query summary, plan text and the
profile.
Change-Id: I759970bf7a41874435baff700025ee0a3407d222
Reviewed-on: http://gerrit.cloudera.org:8080/68
Reviewed-by: Henry Robinson <henry@cloudera.com>
Tested-by: Internal Jenkins
