jprdonnelly/impala
1
0
Fork 0
mirror of https://github.com/apache/impala.git synced 2025-12-19 18:12:08 -05:00
Code Issues Projects Releases Wiki Activity
Files
36a63d0a3309e714862b0cdc05f3d4dc7464a7eb
impala/bin/rat_exclude_files.txt
Surya Hebbar 36a63d0a33 IMPALA-12182: Add CPU utilization chart for RuntimeProfile's sampled metrics 
This change adds support for a stacked area chart for CPU utilization
to the query timeline display, while also providing the ability to scale
timetick values, precision, and the ability to horizontally scale
the fragment timing diagram along with the utilization chart.

Rendering of different components within the diagram has been decoupled
to isolate scaling of timeticks, also to improve the overall
efficiency by making the rendering functions asynchronous
for better performance during resize events. Additionally,
re-rendering of fragment diagram is only triggered during new
fragment events.

The following are the associated key bindings to scale the timeline
with mouse wheel events.
- shift + wheel events on #fragment_diagram
- shift + wheel events on #timeticks_footer
- alt + shift + wheel events on #timeticks_footer for precision control

Note:
Ctrl + mouse wheel events and ctrl + '+'/'-' events can be used to
resize the timeline through the browser.

Mouse wheel events have been associated with respective components
for better efficiency and maintainability.

Constraints have been added to above attributes to limit scaling/zooming
for appropriate display and rendering across all DOM elements.

RESOURCE_TRACE_RATIO query option provides the utilization values to be
traced within the RuntimeProfile. It contains samples of CPU utilization
metrics for user, sys and iowait. These time series counters are available
within the profile having the following names.

Per Node Profiles -
  - HostCpuIoWaitPercentage
  - HostCpuSysPercentage
  - HostCpuUserPercentage

The samples are updated based on 'periodic_counter_update_period_ms'
providing the 'period' within profile's 'Per Node Profiles'.

These are retrieved from the ChunkedTimeSeriesCounter in the
RuntimeProfile. Currently, JSON profiles and webUI summary pages
contain the downsampled values.

Utilization samples are aligned with the fragment diagram by
associating the number of samples and the period.

Aggregate CPU usage for each node is being calculated
after accumulating the basis point values for user, sys
and iowait. These are being displayed after grouping the
associated counters for each node as a stacked line chart.

c3.js charting library based on d3.v5 is being used to plot
the utilization.

The license associated with d3 v5 during the related time frame
has been included along with the charting library's.

Support for experimental profile V2 is currently not included.

Scaling a large number of values to support profile V2 would be
possible with appropriate down-sampling in the back-end.

Testing: Manual testing with TPC-DS and TPC-H queries

Change-Id: Idea2a6db217dbfaa7a0695aeabb6d9c1ecf62158
Reviewed-on: http://gerrit.cloudera.org:8080/20008
Reviewed-by: Riza Suminto <riza.suminto@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
2023-07-11 00:06:24 +00:00

257 lines
8.2 KiB
Plaintext
Raw Blame History

# These are the globs that RAT ignores when doing a copyright
# audit. Comments start with "# ".
# http://www.apache.org/legal/src-headers.html: "A file without any degree of creativity
# in either its literal elements or its structure is not protected by copyright law;
# therefore, such a file does not require a license header."
.clang-format
.gitattributes
.gitignore
*/.gitignore
*/rat_exclude_files.txt
be/src/testutil/htpasswd
be/src/testutil/*.key
bin/generate_junitxml.py
bin/junitxml_prune_notrun.py
tests/*/__init__.py
testdata/common/__init__.py
fe/src/test/resources/regionservers
shell/__init__.py
ssh_keys/id_rsa_impala
testdata/__init__.py
tests/__init__.py
bin/diagnostics/__init__.py
lib/python/impala_py_lib/__init__.py
lib/python/impala_py_lib/jenkins/__init__.py
shell/packaging/MANIFEST.in
shell/packaging/requirements.txt
testdata/cluster/node_templates/cdh7/etc/init.d/kms
testdata/authentication/*
bin/banned_py3k_warnings.txt
# See $IMPALA_HOME/LICENSE.txt
be/src/gutil/*
be/src/thirdparty/datasketches/*
be/src/thirdparty/murmurhash/*
be/src/thirdparty/mpfit/*
be/src/thirdparty/fast_double_parser/*
be/src/kudu/gutil
www/highlight/*
www/DataTables*/*
www/datatables-*.*
www/bootstrap/css/bootstrap*
www/bootstrap/js/bootstrap*
www/favicon.ico
www/Chart*
tests/comparison/leopard/static/css/bootstrap*
tests/comparison/leopard/static/fonts/glyphicons-halflings*
tests/comparison/leopard/static/js/bootstrap*
shell/ext-py/bitarray-2.3.0/*
shell/ext-py/kerberos-1.3.1/*
shell/ext-py/prettytable-0.7.2/*
shell/ext-py/sasl-0.3.1/*
shell/ext-py/six-1.14.0/*
shell/ext-py/sqlparse-0.3.1/*
shell/ext-py/thrift-0.16.0/*
shell/ext-py/thrift_sasl-0.4.3/*
www/c3/*
www/d3.v3.min.js
www/d3.v5.min.js
www/jquery/jquery-3.5.1.min.js
tests/comparison/leopard/static/css/hljs.css
tests/comparison/leopard/static/js/highlight.pack.js
common/protobuf/kudu
be/src/kudu/util/array_view.h
be/src/kudu/util/cache-test.cc
be/src/kudu/util/cache.cc
be/src/kudu/util/cache.h
be/src/kudu/util/cloud/instance_detector-test.cc
be/src/kudu/util/coding.cc
be/src/kudu/util/coding.h
be/src/kudu/util/condition_variable.cc
be/src/kudu/util/condition_variable.h
be/src/kudu/util/debug/trace_event.h
be/src/kudu/util/debug/trace_event_impl.cc
be/src/kudu/util/debug/trace_event_impl.h
be/src/kudu/util/debug/trace_event_impl_constants.cc
be/src/kudu/util/debug/trace_event_synthetic_delay.cc
be/src/kudu/util/debug/trace_event_synthetic_delay.h
be/src/kudu/util/env.cc
be/src/kudu/util/env.h
be/src/kudu/util/env_posix.cc
be/src/kudu/util/nvm_cache.cc
be/src/kudu/util/random.h
be/src/kudu/util/slice.h
be/src/kudu/util/status-test.cc
be/src/kudu/util/status.cc
be/src/kudu/util/status.h
be/src/kudu/util/sse2neon.h
be/src/kudu/security/x509_check_host.h
be/src/kudu/security/x509_check_host.cc
be/src/util/cache/cache.h
be/src/util/cache/cache.cc
be/src/util/cache/cache-internal.h
be/src/util/cache/cache-test.h
be/src/util/cache/cache-test.cc
be/src/util/cache/rl-cache.cc
be/src/util/cache/rl-cache-test.cc
docs/css/dita-ot-doc.css
docs/shared/header.xml
# http://www.apache.org/legal/src-headers.html: "Short informational text files; for
# example README, INSTALL files. The expectation is that these files make it obvious which
# product they relate to."
be/src/testutil/certificates-info.txt
bin/README-RUNNING-BENCHMARKS
LOGS.md
README*.md
*/README
*/README.dox
*/README.txt
testdata/bin/README-BENCHMARK-TEST-GENERATION
testdata/scale_test_metadata/README.md
tests/comparison/ORACLE.txt
bin/distcc/README.md
tests/comparison/POSTGRES.txt
docs/README.md
docker/README.md
be/src/thirdparty/pcg-cpp-0.98/README.md
lib/python/README.md
lib/python/impala_py_lib/gdb/README.md
shell/packaging/README.md
bin/kerberos/README-kerberos.md
# http://www.apache.org/legal/src-headers.html: "Test data for which the addition of a
# source header would cause the tests to fail."
testdata/*.csv
testdata/*.sql
testdata/*.test
be/src/kudu/util/testdata/*.txt
be/src/testutil/*.pem
*.json
fe/src/test/resources/*.xml
fe/src/test/resources/adschema.ldif
fe/src/test/resources/adusers.ldif
fe/src/test/resources/hbase-jaas-client.conf.template
fe/src/test/resources/hbase-jaas-server.conf.template
fe/src/test/resources/users.ldif
java/toolchains.xml.tmpl
testdata/AllTypesError/*.txt
testdata/AllTypesErrorNoNulls/*.txt
*.avsc
*.parq
*.parquet
testdata/cluster/hive/*.diff
testdata/cluster/node_templates/cdh5/etc/hadoop/conf/*.xml.tmpl
testdata/cluster/node_templates/common/etc/kudu/*.conf.tmpl
testdata/cluster/node_templates/common/etc/hadoop/conf/*.xml.tmpl
testdata/cluster/ranger/setup/*.json.template
testdata/data/binary_tbl/000000_0.txt
testdata/data/chars-formats.txt
testdata/data/chars-tiny.txt
testdata/data/parent_table.txt
testdata/data/parent_table_2.txt
testdata/data/child_table.txt
testdata/data/date_tbl/*.txt
testdata/data/date_tbl_error/*.txt
testdata/data/decimal-tiny.txt
testdata/data/decimal_tbl.txt
testdata/data/decimal_rtf_tiny_tbl.txt
testdata/data/decimal_rtf_tbl.txt
testdata/data/overflow.txt
testdata/data/text-comma-backslash-newline.txt
testdata/data/text-dollar-hash-pipe.txt
testdata/data/dateless_timestamps.txt
testdata/data/text_large_zstd.txt
testdata/data/text_large_zstd.zst
testdata/data/widerow.txt
testdata/data/local_tbl/00000.txt
testdata/data/hudi_parquet/*
testdata/data/iceberg_test/*
testdata/data/sfs_d2.txt
testdata/data/sfs_d4.txt
testdata/datasets/functional/functional_schema_template.sql
testdata/impala-profiles/README
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.txt
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.json
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.pretty.json
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.pretty_extended.json
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_default.expected.txt
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_extended.expected.txt
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_v2
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_v2_default.expected.txt
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_v2_extended.expected.txt
testdata/hive_benchmark/grepTiny/part-00000
testdata/jwt/*.json
testdata/jwt/jwt_expired
testdata/jwt/jwt_signed
testdata/jwt/jwt_signed_untrusted
testdata/tzdb/2017c.zip
testdata/tzdb/2017c-corrupt.zip
testdata/tzdb_tiny/*
tests/pytest.ini
tests/shell/bad_impalarc
tests/shell/good_impalarc
tests/shell/good_impalarc2
tests/shell/good_impalarc3
tests/shell/good_impalarc4
tests/shell/impalarc_with_error
tests/shell/impalarc_with_error2
tests/shell/impalarc_with_query_options
tests/shell/impalarc_with_warnings
tests/shell/impalarc_with_warnings2
tests/shell/shell.cmds
tests/shell/shell2.cmds
tests/shell/shell_case_sensitive.cmds
tests/shell/shell_case_sensitive2.cmds
tests/shell/shell_error.cmds
tests/shell/test_close_queries.sql
tests/shell/test_file_comments.sql
tests/shell/test_file_no_comments.sql
tests/shell/test_var_substitution.sql
# Generated by Apache-licensed software:
be/src/transport/config.h
# BSD 2-Clause license:
bin/summarize-pstacks
# BSD 3-clause license that RAT can't seem to identify:
cmake_modules/FindJNI.cmake
# http://www.apache.org/legal/resolved.html#category-a : Python Software Foundation
# License is allowed.
shell/pkg_resources.py
# Notices in Impala as required by ASF rules:
DISCLAIMER
LICENSE.txt
NOTICE.txt
# Notices in thirdparty sources included in the Impala repo and called out in /LICENSE.txt
be/src/thirdparty/squeasel/LICENSE
be/src/thirdparty/pcg-cpp-0.98/LICENSE.txt
be/src/thirdparty/xxhash/xxhash.h
be/src/thirdparty/xxhash/README.md
# http://www.apache.org/legal/src-headers.html: 'Snippet' files that are combined as form
# a larger file where the larger file would have duplicate licensing headers.
www/all_child_groups.tmpl
www/blacklisted_tooltip.txt
www/common-footer.tmpl
www/form-hidden-inputs.tmpl
www/quiescing_tooltip.txt
# GNU tar artifact
pax_global_header
# Files in binary image formats:
tests/comparison/leopard/static/favicon.ico
docs/images/howto_access_control.png
docs/images/howto_per_node_peak_memory_usage.png
docs/images/howto_show_histogram.png
docs/images/howto_static_server_pools_config.png
docs/images/impala_arch.jpeg
docs/images/support_send_diagnostic_data.png
Reference in New Issue View Git Blame Copy Permalink