* Handle NotSupported exception in refresh_schema
- Add NotSupported exception handling to refresh_schema()
- Log unsupported datasources at DEBUG level
- Avoid error metrics for datasources without schema support
* Add test for NotSupported exception handling
- Test that NotSupported exceptions are caught and logged at DEBUG level
- Verify no warning logs are generated for unsupported datasources
* Fix import order (ruff)
* Remove test for NotSupported exception handling
As suggested by @yoshiokatsuneo, testing logging details for 3 lines of code
is excessive and may hurt maintainability. The existing tests already ensure
the functionality works correctly.
* fix(destinations): Handle unicode characters in webhook notifications
Previously, webhook notifications would fail if they contained unicode characters in the alert data. This was because the JSON payload was not UTF-8 encoded before being sent.
This commit fixes the issue by explicitly encoding the JSON data to UTF-8 and adds a test to verify the fix.
* move test function to new file
---------
Co-authored-by: gaojingyu <gaojingyu>
Add support for saving dashboard parameters after clicking the Apply button. Parameters are applied in the following order: URL, dashboard parameters, query parameters.
Persist the queued values only when “Done Editing” is clicked, keeping Query and Dashboard editors aligned.
* Add ibm-db package to enable DB2 as datasource:
* Review poetry format
* Added condition on platform for ibm-db, as support is restricted
---------
Co-authored-by: nicof38 <nicolas@FB-L-230557.soitec.net>
Co-authored-by: Tsuneo Yoshioka <yoshiokatsuneo@gmail.com>
* Update queries.latest_query_data on save
* Add wait on test as loading query and query results may re-render DOM and that makes test fraky
* Fix styling report by prettier
This prevents duplicate entries in the schema list. Materialized views are the
only table-like object not found information_schema. Also ensure that the schema
and table found in information_schema is accessible by the current user.
* Use 12-column layout for dashboard grid
Set minSizeX, minSizeY for widgets to 2 since a value of 1 breaks all
assumptions of the UI layout.
Migration provide transition from 6 to 12 columns for all widgets.
* Restyled by prettier
In commits fc1e1f7 and e44fcdb a new Selector option was added to
alerts, which may be "first", "min" or "max". This migration sets the
default to "first" for existing alerts.
* Update Azure Data Explorer query runner to latest version
* Fix black issue
* downgrade azure-kusto-data to 4.6.3
* Freeze numpy to 1.24.4 because of 2.0.0 incompatibility
* Fix failing test
* Reformat test
Preview images work for personal repositories, but we missed another variable
when publishing official images:
#34 [auth] arikfr/redash:pull,push token for registry-1.docker.io
#34 DONE 0.0s
#33 exporting to image
#33 pushing layers 15.5s done
#33 pushing manifest for docker.io/arikfr/redash
#33 pushing manifest for docker.io/arikfr/redash 1.6s done
#33 ...
#35 [auth] arikfr/preview:pull,push token for registry-1.docker.io
#35 DONE 0.0s
Fix query list item list sorting
- descending order, no triangle mark
- ascending order, up triangle mark(▲)
- descending order, down triangle mark(▼)
- ascending order, no triangle mark
- descending order, up triangle mark(▲)
- ascending order, down triangle mark(▼)
- descending order, no triangle mark
"sorting order" have 2-click cycle, but "triangle mark" have 3-click cycle.
* Sanitize NaN, Infinite, -Infinite causing error when saving as PostgreSQL JSON #7339 (2nd try)
* Move json nsanitaize to on the top of json_dumps
* Fix comment
This fixes the following exception:
```
UnboundLocalError: local variable 'value_is_number' referenced before assignment
File "rq/worker.py", line 1431, in perform_job
rv = job.perform()
File "rq/job.py", line 1280, in perform
self._result = self._execute()
File "rq/job.py", line 1317, in _execute
result = self.func(*self.args, **self.kwargs)
File "redash/tasks/alerts.py", line 36, in check_alerts_for_query
new_state = alert.evaluate()
File "redash/models/__init__.py", line 1002, in evaluate
new_state = next_state(op, value, threshold)
File "redash/models/__init__.py", line 928, in next_state
elif not value_is_number and op not in [OPERATORS.get("!="), OPERATORS.get("=="), OPERATORS.get("equals")]:
```
* prevent text values in payload being detected as 'set' on send.
Webex send ERROR:: Object of type set is not JSON serializable
Signed-off-by: Matt Nelson <metheos@gmail.com>
* add support for formatted QUERY_RESULT_TABLE in webex card
Signed-off-by: Matt Nelson <metheos@gmail.com>
* don't try to send to blank destinations
Signed-off-by: Matt Nelson <metheos@gmail.com>
* fix handling of the encoded QUERY_RESULTS_TABLE text
Signed-off-by: Matt Nelson <metheos@gmail.com>
* re-sort imports for ruff
Signed-off-by: Matt Nelson <metheos@gmail.com>
* change formatter to black
Signed-off-by: Matt Nelson <metheos@gmail.com>
* Add additional tests for Webex notification handling
ensure blank entries are handled for room IDs and person emails.
ensure that the API is not called when no valid destinations are provided.
ensure proper attachment formatting for alerts containing 2D arrays.
Signed-off-by: Matt Nelson <metheos@gmail.com>
* Add test for Webex notification with 1D array handling
This commit introduces a new test case to verify that the Webex
notification function correctly handles a 1D array input in the alert body.
The test ensures that the expected payload is constructed properly and that
the requests.post method is called with the correct parameters.
Signed-off-by: Matt Nelson <metheos@gmail.com>
---------
Signed-off-by: Matt Nelson <metheos@gmail.com>
* Support result reuse
* Update pyathena to 2.25.2
* Separate options
* Regenerate the Poetry lock file
---------
Co-authored-by: SeongTae Jeong <seongtaejg@gmail.com>
I've experience this on my Redash in production. I'm not sure what can cause the value to exist, but be None. I guess it depends on the SQL query.
I followed the same idea of returning a self.UNKNOWN_STATE for cases that we can't know what's happening.
Something changed in python-rq and the old code was behaving in a way that if a job ran for longer than 2 min it would be automatically set as failed, but it would continue running.
This causes a problem in the UI because it is as if the job stopped, but it actually didn't
* Snapshot: 24.08.0-dev
* no more Running...0 or runtime0
* also missing a space
* Restyled by prettier
* check if data_scanned is defined
otherwise we could get "Data Scanned ?" if it's not supported
by some data sources
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <commits@restyled.io>
The Dependabot alert for sentry-sdk says that the security fix has
been backported to the 1.x series as well, in version 1.45.1.
So, lets use that as it should be more compatible that jumping to
a new major series version.
This change involved adding an extra option to the GridSettings editor,
adding the "fixed" option to columns, and adding styling for the fixed
columns. In order to change the number of fixed columns, which will
default to 0, one has to go to Edit visualization -> Grid -> Choose
number of columns to fix -> Save.
This covers cases where partitioned tables are part of a schema that is
not accessible by the current user.
CREATE SCHEMA xyz;
CREATE TABLE xyz.tab (
id bigint GENERATED ALWAYS AS IDENTITY,
ts timestamp NOT NULL
) PARTITION BY LIST ((ts::date));
CREATE TABLE xyz.tab_default PARTITION OF xyz.tab DEFAULT;
Updated from Python 3.8 to 3.10. Python 3.10 is the default for Ubuntu 22. This change necessitated upgrading to
SQLAlchemy_Utils 0.38.3, and importing the sort_query function from an older version of SQLAlchemy_Utils because it was dropped in newer versions.
Co-authored-by: Ezra Odio <eodio@starfishstorage.com>
"Query in queue" should switch to "Executing query", but does not.
Commands:
git revert --no-commit bd17662005
git revert --no-commit 5ac5d86f5e
vim tests/handlers/test_query_results.py
git add tests/handlers/test_query_results.py
Co-authored-by: Justin Clift <justin@postgresql.org>
This reverts commit bd115e7f5f, as
it turns out to be a useful security feature.
In order to remove this in a better way, we'll need to replace it
with something that provides equivalent functionality.
The 'codecov-action@v4' requires an organization-level upload token, not
a single repo upload token, so we're temporarily downgrading it until we
can generate an organization-level upload token.
Reference: https://github.com/codecov/codecov-action/issues/1273
Use Webpack configuration for locating this asset in the same way that
client/app/index.html does.
This code path is when REDASH_MULTI_ORG=true.
Co-authored-by: github-actions <github-actions@github.com>
This PR changes the default (tcp) port for the web user interface back to port 5001.
The recent change to port 5000 (to match an old default) turned out to be more painful than it's worth.
So, lets keep using port 5001 after all.
* filter widged results to fix tests during repeatable execution
* minor fix
* made queryName variable
---------
Co-authored-by: Andrew Chubatiuk <andrew.chubatiuk@motional.com>
The qds-sdk-py package along with the rest of the Qubole project is no longer
maintained:
3c6a34ce33
Removing this eliminates these warnings when running Redash management commands:
./qds_sdk/commands.py:1124: SyntaxWarning: "is" with a literal. Did you mean "=="?
if options.mode is "1":
./qds_sdk/commands.py:1137: SyntaxWarning: "is" with a literal. Did you mean "=="?
if options.db_update_mode is "updateonly":
./qds_sdk/commands.py:1424: SyntaxWarning: "is" with a literal. Did you mean "=="?
if (total is 0) or (downloaded == total):
Co-authored-by: github-actions <github-actions@github.com>
Format the Slack message using the "mrkdwn" type, which will make
hyperlinks clickable.
New test for Slack destination.
Co-authored-by: github-actions <github-actions@github.com>
To avoid warnings in the CI pipeline
> Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20
Co-authored-by: github-actions <github-actions@github.com>
Since #6687, we don't serialize query results as JSON
before returning them. This is fine, except for the
query results data source which needs to pass the
data directly to sqlite3, and doesn't know how to
do that with the decimal types that are occasionally
returned by (at least) the PostgreSQL query runner:
https://www.psycopg.org/docs/faq.html#problems-with-type-conversions
Using github.repository_owner name was convenient for testing this
action, but is not correct since account names do not match.
Git Hub: getredash/
Docker Hub: redash/
Co-authored-by: github-actions <github-actions@github.com>
* Only respond to new tags ending with -dev
* Use github account name to allow easier testing in a fork
* Allow preview image to be referenced by a specific tag, or by latest tag
redash/preview:24.02.0-dev
redash/redash:preview
Co-authored-by: github-actions <github-actions@github.com>
To allow this workflow to run even though normal contributors
are required to create a pull request.
Steps:
1. Generate SSH key pair: ssh-keygen -t ed25519. No need for passphrases etc.
2. Add public key (.pub one) as a deploy key at Your repo -> Settings ->
Security -> Deploy keys, check "Allow write access".
3. Add private key as a secret at Your repo -> Settings -> Security -> Secrets
and variables -> Actions
https://stackoverflow.com/a/76135647/1809872
Co-authored-by: github-actions <github-actions@github.com>
* Fixed notification template
* Made if-clause equal to append
Like Slack and email notification
* Add custom_body attribute to discord test
* Add missing attribute
* Adds influxdb v2 query runner as a default query runner.
* Deletes TableList object for typing, because it is possible, that the import of influxdb client fails.
---------
Co-authored-by: Masayuki Takahashi <masayuki038@gmail.com>
* Adds ssl support for prometheus query runner.
- Adds possibilty to upload and use of ssl cert, key and ca file in redash ui
* Extends test cases for prometheus query runner.
- Adds secret attribute to configuration schema.
* Fixes wrong timestamps in different timezones in prometheus' testcases.
- Dynamically calculates timestamps in testcases to be robust in
different timezones.
- Adds now datetime function to make it more testable.
* Fixes timestamp in prometheus' testcases which can be wrong depending on timezone.
---------
Co-authored-by: Masayuki Takahashi <masayuki038@gmail.com>
* Snapshot: 23.11.0-dev
* dataframe_to_result function moved outside python query runner
* added yandex disk query runner
* moved file_extension check
* skip unsupported extensions in schema
* removed unused variable
* added support for xlsx with multiple sheets
* moved pandas-converters to utils file
* added tests
* fixed backend tests
* fixed pandas to redash type conversion
* added more tests
* added tests for pandas
* added tests for pandas converter and yandex disk
* added tests for read_file and multiple sheets
* pandas: do not load if lib is not installed
* added test for yaml read
* fixed test for yaml read
---------
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
* Update rockset.py with support for multi-VI
Rockset introduced compute-compute separation which allows multiple Virtual Instances (VIs) to query data independently. This a commit introduces support for executing queries on specific VIs from Redash.
Changes include:
- added new configuration element (not required): Virtual Instance ID
- if this VI ID is configured, Redash will use that specific VI to execute the query
- if it's not specified, Redash will execute the query on the main/default VI (same behaviour as before)
* Update rockset.py
Removed comment from the query method to pass lint test
* Update rockset.py - for linting
Added more formatting to pass the black lint test
* Update rockset.py - linting update
* Update rockset.py - updated lint
* Allow Query.options to be None
Query.options may not have a value on a database created by 10.1.0
* Ensure counter widget is not compressed in query preview tabs
Also bump minimum counter font size to 14
The required sql expressions for sqlalchemy_searchable are only created
on the sqlalchemy pre_create hook. When upgrading from a previous redash
version no CREATE statements may be run, so these expressions are never
created.
Add a migration to manually execute the sql_expressions DDL from
sqlalchemy_searchable.
* Add column type to a query result of InfluxDB
* Remove an unused import
* Migrate to pytest
* Rename test names to descriptive
---------
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
update greenlet to 2.0.2 -
Because redash depends on gevent (23.9.1) which depends on greenlet (>=2.0.0), greenlet is required.
So, because redash depends on greenlet (1.1.3), version solving failed.
__file__ will resolve to tests/test_cli.py, which is already opened by Python.
When tests are run on a network file system the test runner may deadlock while
waiting for an advisory lock to be released on this file.
* change author and add maintainers
* remove bin/upgrade
* update project author/maintainers
* comment on how to get added/removed from the mailing list
* Check all Trino catalogs if no catalog configured
- If catalog is not set, query 'SHOW CATALOGS' to get catalogs
- For each catalog, discover schema, table, columns
- If catalog has a `.`, then quote the catalog name
- New table name returned includes catalog name
- If catalog is set, same result as before
* Use self._handle_run_query_error(error)
* Fix lint problem & format w/ black
* Fix condition so catalog_prefixes are [""] when catalog is set
* Fix for loop / Remove default values for catalog & schema
* Add untested tests for Trino's get_schema
* Black formatting
* Add test_get_schema_catalog_set test & fix tests
* Parameterize the side_effect fn thoroughly
* Get catalogs (not catalog_prefixes) bc confusing
* Trino._get_catalogs added for testability
There's a few advantages of using ruff over these tools:
* It's way faster
* It's easier to configure
* It includes support for a bunch of other linters (for example bugbear)
right out of the box, which catches some things and makes our code
more consistent.
Ruff works great with black, which I'd recommend we continue using.
Fixed a few minor issues that the new linter combo picked up.
* poetry init
* add all dependencies
* update pyproject.toml
* add poetry.lock
* remove requirements txt fixer
* add ldap3 group and make all groups optional
* remove requirements files
* convert dockerfile to use poetry instead of pip
* fix ldap 3 dependency group typo
* update ldap3 inline error
* update cypress to install only main poetry group
* convert test_all_deps
* update redis/rq
* don't create virtualenv
* add -dev suffix and make version snapshot work
* fix typo in snapshot commit message
* remove importlib-resources
This is not required for Python >= 3.8 since the goal of this project is
to back port functionality. Also the version specified is not
compatible with later versions of Python.
https://pypi.org/project/importlib-resources/
oauth2client was deprecated in 2017 and is no longer maintained.
This rewrites the service credentials code and replaces it with google-auth.
It also makes the JSON key file optional for the data source and use the
application's default credentials (ADC) if the JSON key file is omitted.
This enables support for a variety of GCP authentication methods, including
the gcloud CLI, GCE metadata servers, and GKE or AWS Workload Identity.
Note that the bigquery_gce functionality is covered by ADC, but is retained
for compatibility reasons.
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
* convert unexpected validation errors to InvalidParameterError
* get rid of local except blocks - any failure to validate is caught in _valid()
* avoid bare exception
* don't assign exception if we're not going to use it
* preferentially raise querydetachedfromdatasource error if it is present
---------
Co-authored-by: Omer Lachish <omer@rauchy.net>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
This needs the appropriate Docker Hub login credentials set in our GitHub settings before it will work properly.
* DOCKER_USER environment variable
* DOCKER_PASS secret
* Use multi_byte_search_enabled option for My Queries search and Favorite List search
* make format
* add search by user tests
---------
Co-authored-by: Justin Clift <justin@postgresql.org>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
* Support to find a worksheet by the title for google_spreadsheets
* Add tests for parse_query
* Fix types
* Add tests when finding a worksheet by a title is failed
* Proxy by a wrapper instead of using spreadsheet directly. Do not use gspread in tests
* Add tests. Fix format of quoted strings for titles
* Fix an error format
* Add a newline
* Fix formatting
---------
Co-authored-by: Justin Clift <justin@postgresql.org>
* Add Asana alert destination
* change icon
* fix format
* add except in log
* fix flake8
* add metadata kwarg to notify
* update asana image
* make pat and project id required
* rename asana destination icon
* reuse Alert
* extract api base url to property method
* use required options
* add asana destination test
* make format
* define metadata
* fix test
* remove json.dumps from asana test
---------
Co-authored-by: Konstantin Smirnov <46676677+konnectr@users.noreply.github.com>
Co-authored-by: konnectr <1konnectrl@gmail.com>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
* [client] add friendly console log on '<' syntax error
Issue: when a javascript file is served with invalid contents, sometimes
it means that a fallback html file was returned from server instead.
This can be hard to determine because the stack trace is not helpful and
the `filename` information was previously lost from the global error
handler.
Added a specific error for this case to produce a friendlier error
message on the console.
* Add ace define override for custom mode
* Apply prettier changes
---------
Co-authored-by: Justin Clift <justin@postgresql.org>
* increase duplicate column names based on the original column name and not the number of duplicates detected
* add fetch columns test for base query runner
---------
Co-authored-by: Omer Lachish <omer@rauchy.net>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
* Fix python query runner crashing the GUI due to missing 'rows' and/or 'columns' in the JSON returned data.
* Fix typo of previous commit.
* Throw exception when python query runner has invalid result.
* Update test_python.py
---------
Co-authored-by: YuhengChen <yuheng.chen@imaygou.com>
* Fix: Query results in public dashboard not loading when query ends with semicolon
* fix flake8 error for commit 7a50b18d3
---------
Co-authored-by: Alexander Strickner <alexander.strickner@outlook.com>
The current get_schema method of Trino query runner is missing table columns' data types information, so these data types won't be seen in Queries editing GUI. After the modification, these info are back.
Co-authored-by: V <4979571+vnnw@users.noreply.github.com>
* Incomplete URL substring sanitization
* fix hostname
* Only match on the exact host name
---------
Co-authored-by: Justin Clift <justin@postgresql.org>
* Do not makedsn. instead self.configuration["servicename"]
in host == "_donotmakedsn"
* add comment
* mod properties in `host`
* mod properties in `host`
* mod word. in host == "_donotmakedsn" to "_useservicename"
* mod properties in `host`
* fix format
For some unknown reason the `--colors` and `-d` options are no longer recognised by webpack (v4) in our configuration.
This is likely due to some other non-obvious problem in our configuration, as those options should exist (they're documented).
For now though, this gets things working again.
The config file format for webpack has changed slightly, so webpack was throwing this error:
Invalid options object. Less Loader has been initialized using an
options object that does not match the API schema.
This commit updates the config file format so webpack works again.
Yarn seems to miss a lot of deduplication opportunities, regardless of what the docs claim.
This can leave old, insecure versions of some dependencies in use when they shouldn't be.
* add try except for data_source
* add try except for data_source
* fix errir handle
* add test for no data source
* changed error handle
* fix format
* fix test
---------
Co-authored-by: Konstantin Smirnov <46676677+konnectr@users.noreply.github.com>
Co-authored-by: konnectr <1konnectrl@gmail.com>
Dependabot couldn't find the original pull request head commit, b868bb5edd4a3fc036ed40754f099f37541a2814.
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump Cypress to 6.0.1
Unfortunately, due to a change in how Cypress treats elements with opacity 0, we need to disable two of the front end tests.
Hopefully someday, someone with better knowledge of JS and Cypress is able to fix these. :)
- Escape all variables by default since Mustache only has a syntax for raw values
- Generate a generic HTML result table to allow an alert template to display any query
- Optionally allow alert template to be defined to be set using REDASH_ALERTS_DEFAULT_MAIL_BODY_TEMPLATE
- Formatting updated by black
Generating the query hash from the query text with no lowercasing of the query text
allows case-sensitive parameter values in the dashboard to have different cache entries.
Fixes#2137
* Fix persisting datasource selection choice
Closes getredash/redash#5646.
We were already storing this in localStorage, however we weren't
accounting for the fact that it was stored as a string, rather than
an integer.
* Restyled by prettier (#6182)
Co-authored-by: Restyled.io <commits@restyled.io>
---------
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <commits@restyled.io>
* Reencrypt Database Safely
* Try with only one statement
* remove success logging
Signed-off-by: Ye Sijun <junnplus@gmail.com>
---------
Signed-off-by: Ye Sijun <junnplus@gmail.com>
Co-authored-by: Jun <junnplus@gmail.com>
Also adds the POSTGRES_HOST_AUTH_METHOD environment variable to the CI docker compose files. Modern PostgreSQL docker images need it, at least as they're used by our CI.
Problem: setting the `REDASH_REFERRER_POLICY` environment variable also sets the feature policy in addition to the referrer policy. Conversely, there is no way to set the feature policy on its own.
It looks like a copy/paste error from the line above.
Solution: change the feature policy to be set by a `REDASH_FEATURE_POLICY` environment variable.
* Remove DynamoDB as a data source for now, due to dependency incompatibilities
* Remove Firebolt as a data source for now, due to dependency incompatibilities
* Remove pymapd / pyomnisci / heavyai, due to dependency incompatibilities
* Remove pycrypto, upgrade cryptography
* Remove pyarrow (for now), which we no longer need
* Require an (older) version of black, to avoid test failures with more recent click library
* Upgrade gevent to 21.12.0
* Upgrade greenlet to 1.1.2
* Upgrade httpli2 to 0.18.1
* Upgrade requests to 2.31.0
* Upgrade snowflake-connector-python to 3.0.4
* Upgrade urllib3 to 1.25.11
Note, the warning message caused by the older versions of gevent and greenlet was:
RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 40 from C header, got 144 from PyObject
* Pin boto3 dependencies in dev requirements as well
Testing to see if this fixes CI, where it appears a more recent (too
recent) version of boto3 is being installed.
* Add circle step to list installed packages
This takes almost no time, and helps debug dependency issues (a
longer-term fix might be to switch to pip-compile).
* Update ngines definition to allow for newer versions of Node.
With Node version 19 I stumbled into some issues so for now bumped it to v16, until we get to updating the libraries we use.
* docker-compose.yml updates:
1. Switch to newer maildev docker image.
2. Update local port to 5001 as 5000 seems to be used by a system process now.
* Update pymssql and pyarrow. Also commented out ibm-db until we have a way to not install it only on ARM.
* handle query execution error in one place. increase ability to debug issues with schema retrieval
* split message and details for error reporting
Co-authored-by: Dmitriy Apollonin <dmitriy.apollonin@aspireiq.com>
* Microsoft Teams Webhook alert destination
* Text formatting and new image for Microsoft Teams Webhook
* Comment on how to build frontend
* Add title to clarify webhook URL
* Make the message into a configurable template.
- A runner supporting the newest versions of ES,
aggregation, nested aggregations and nested fields.
- A runner for the SQL OpenDistro flavor
- A runner for the SQL X-Pack flavor
Co-authored-by: Nicolas Le Manchet <nicolas@lemanchet.fr>
Co-authored-by: wwl717195673 <717195673@qq.com>
ClickHouse query runner splits query into several and execute each query in turn. The result of the last execution is returned. Implementation uses ClickHouse sessions in the HTTP protocol. `session_id` is generated for the first query and then it is used with the subsequent queries (together with the `session_check` parameter).
If query runner gets a success response with empty body from ClickHouse (for example, in case of temporary table creation request) query runner returns empty response.
authored-by: Liubov Ulitina <ulitinalm@vl.ru>
* Fix hard-coding of amd64 platform and make amd64 package installation conditional
* Cleanup Dockerfile for best practices
* Enable BuildKit for docker building
This change took place in steps:
1. Change order of content and sidebar.
Sidebar appears first, then content.
2. Fix padding
* Before: content was jutted against the sidebar. The sidebar was double-
padded from the edge of the content area.
After: Content has 15px pad against the sidebar. Sidebar has the same pad
as the page title.
3. Don't pad the content on small screens.
Otherwise the content appears off-center and doesn't use all of the
available space.
4. Allow Create buttons to have varying width
This makes the Query, Dashboard, and Alert list pages share the same style
Makes the details field a JSONB field per pg doc recommendations.
Update model.all() method to work properly now that profile_image_url
is not an independent field.
Closes#4469
* Added firebolt-sdk in place of firebolt-sqlalchemy
* fixed connection issue
* fixed connection issue
* final commit
* Moved firebolt-sdk's imports to try block
Co-authored-by: rajeshSigmoid <rajeshk@sigmoidanalytics.com>
Add verify option to json datasource runner to allow query developers the option of skipping certificate verification
Co-authored-by: Kevin Chiang <kchiang@tesla.com>
Co-authored-by: kevinchiang <kevinchiang@outlook.com>
If you run the docker-compose on a Mac with the new M1 chip, you will get the "Unable to locate package msodbcsql17" error. Because there are currently no msodbcsql17 packages for arm64 architecture. The solution was to change the base image in the Dockerfile to change the installation to the older AMD architecture.
FROM --platform=linux/amd64 python:3.7-slim-buster
* WIP: break the flask_oauthlib behavior
* Refactor google-oauth to use cryptographic state.
* Clean up comments
* Fix: tests didn't pass because of the scope issues.
Moved outside the create_blueprint method because this does not depend
on the Authlib object.
* Apply Arik's fixes. Tests pass.
* Fix: pagination is broken on the dashboard list page (#5516)
* Add test that reproduces issue #5466
* Fix: Duplicate dashboard rows were returned by Dashboard.all() (#5466)
* Update changelog for V10
* Update changelog for #5516
protobuf package with a dependency of google-api-python-client released a new version (3.18.0) on September 16, 2021. Since then, the Docker build is failing, and it is presumed that there is a conflict with other DataSource packages that use protobuf. (phoenixdb, pydgraph)
* Guard against empty totalProcessedBytes in BigQuery responses
This field will be empty on query responses for tables with
row level access controls enabled.
* Fix whitespace
* Update redash/query_runner/big_query.py
Co-authored-by: Jesse <jwhitehouse@airpost.net>
* Excel query runner
* Param handling for read_excel
* CSV query runner
* Fix wrong module name
* Use yaml as query language
* Use yaml as query language for CSV
* Added icon and required modules
* Local address filtering
* Fix syntax error
* Add PlainButton
* refactor close icons
* reorder import
* refactor remaining anchors
* refactor: replace remaining <button> and TODOs
* refactor: changed applicable elements to type link
* fix: minor details
* bug: fix tooltip ternary
* refactor: improve interactivity and semantics of schema list item
useEffect() doesn't run until _after_ the component renders. Before the
hook runs, the value of `groups` === []. And this is passed to
<DynamicForm>'s `initialValue` prop. The `initialValue` is not re-evaluated
after useEffect() completes. So the users groups are never updated.
This change pulls the user's current groups from `user` prop on the
page.
* feature: add external link
* refactor: split external link into own component
* refactor: added link with icon
* refactor: remove reduntant tab index
* refactor: simplify props
* refactor: fix types
* refactor: bring types and components together
* refactor: improve treatment of target
* add Corporate Memory Runner based on cmempy 21.2.3
* fix code style
* apply some code nice ups
* use extendedEnum, boolean and extra_options for schema description
* use lower case sorting for data source types list
This correctly orders data source names which starts with lower
chars (such as eccenca Corporate Memory)
* add missing dblogo
* Added screen reader CSS
* Added description to external links
* Added spinner icon accessibility
* Added accessibility to exclamation and big message
* Added question and exclamation accessibility
* Hide decorative icons
* Standardized link design
* Added a11y to refresh icons
* Added aria-label to anchors and buttons
* Added a11y to conditional icons
* Added applicable labels to Ant Icons
* Changed escape to interpolation
* Replaced external links with opens in new tab
* Improved Tooltip hosts
* Added aria live to temporary elements
* Removed mistakenly added redundant helper
* Undoes unnecessarily added interpolation
* Replaced empty label with hidden
* Improved full icon label
* Improved display of live regions
* Added note
* remove unused class
* Created unique id
* Remove TODOs
* Proper action label
* Improved feedback for autocomplete toggle
* feature: add id hook
* refactor: use id hook
* standardize white space
* Add styles for focused ant menus
* Add disabled styles to clickable button
* Improved dashboard header syntax and added focus
* Improved CSS syntax
* Add interactive styles
* Improved anchor dependent styles
* Improved styles of widget (gray more/delete btns)
* Add interactive style for favorite star
* Improved style of delete btn
* Make table content fill all space
* Added focus and active styles
* Scoped query snippets list
* Fixed behavior for all major browsers
* Replaced button styles with plain button
* Scoped items list styles
* Added focus styles to ant table
* Add plain button (#5419)
* Minor syntax improvements
* Refactor of Link component (#5418)
* reset failure counter when query completes successfully via adhoc
* Use "query_id" in metadata, but still allow "Query ID" for transition/legacy support
* Fixed jsx-a11y problems
* Changed tabIndex to type number
* Initial improvements to DesktopNavbar accessibility
* Added accessibility to favorites list
* Improved accessibility in Desktop Navbar
* Improvements in Desktop navbar semantics
* Added aria roles to tags list
* Fixed tabindex type
* Improved aria labels in query control dropdown
* Added tab for help trigger close button
* Fixed typo
* Improved accessibility in query selector
* Changed resizable role to separator
* Added label to empty state close button
* Removed redundant and mistaken roles
* Used semantic components
* Removed tabIndex from anchor tags
* Removed mistakenly set menuitem role from anchors
* Removed tabIndex from Link components
* Removed improper hidden aria label from icon
* Reverted button and link roles in anchors for minimal merge conflicts
* Replaced alt attr with aria-label for icons
* Removed redundant menu role
* Improved accessibility of CodeBlock
* Removed improper role from schema browser
* Reverted favorites list to div
* Removed improper presentation role in query snippets
* Tracked changes for further PR
* Revert "Improved accessibility of CodeBlock"
* Add aria-labelledby to the associated code labels
This reverts commit 00a1685b1b.
* Wrapped close icon into button
* upgrade RQ to v1.5
* set job's started_at
* update healthcheck to match string worker names
* delay worker healthcheck for 5 minutes from start to allow enough time to load in case many workers try to load simultaneously
* log when worker cannot be found
* Add My Dashboards filter option to the Dashboards list. Added API endpoint to get the list of a user's dashboards, similar to the My Queries feature.
* Update empty dashboard list state to show an invite to create a new dashboard, like My Queries
* Update to Levko's suggested approach. Clean up some of the formatting for consistency. Put the 'My Queries/Dashboards' item before the Favorites since that organization seems cleaner to me.
* Address Levko's comments
Fixes#5300 and fixes#5307
There have been upstream (`python:37-slim` image) changes that
bring in `pip` version 20.3.1, which makes new `2020-resolver`
the default. Due to that, un-resolvable dependency conflicts
in `requirements_all_ds.txt` now cause the build to fail.
This is a workaround until the package versions can be updated
to work with the new pip resolver.
* added type casting to coerce number string into nuber
* Merge branch 'master' into fix-inconsistent=sankey-behavior
* typed map viz options
* Partially typed what was possible
* reworked data coercion
* improved MapOptionsType types
* readaqueted sankey rows so as to allow strings again
* created bar-chart e2e test boilerplate
* refactored assertions
* added snapshots and dashboard
* refactored assertions to properly deal with async
* replaced loops with getters for proper workings of cypress
* added a couple other bar charts
* ran prettier
* added a better query for bar charts
* removed leftovers
* moved helpers to support folder
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* fixed QueryBasedParamterInput optionFilterProp
* added optionFilterProp fallback for SelectWithVirtualScroll
* simplified syntax
* removed optionFilterProp from QueryBasedParameterInput.jsx
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* restricted SelectWithVirtualScroll props
* Added e2e test for parameter filters
* moved filter assertion to more suitable place
* created helper for option filter prop assertion
* moved option filter prop assertion to proper place, added result update assertion
* refactor openAndSearchAntdDropdown helper
* Fix parameter_spec
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* Enable graceful shutdown of rq workers
* Use `exec` in the `worker` command of the entrypoint to propagate
the `TERM` signal
* Allow rq processes managed by supervisor to exit without restart on
expected status codes
* Allow supervisorctl to contact the running supervisor
* Add a `shutdown_worker` command that will send `TERM` to all running
worker processes and then sleep. This allows orchestration systems
to initiate a graceful shutdown before sending `SIGTERM` to
supervisord
* Use Heroku worker as the BaseWorker
This implements a graceful shutdown on SIGTERM, which simplifies
external shutdown procedures.
* Fix imports based upon review
* Remove supervisorctl config
* Change front-end and data model for SAML2 auth - static configuration
* Add changes to use inline metadata.
* add switch for static and dynamic SAML configurations
* Fixed config of backend static/dynamic to match UI
* add ability to encrypt/decrypt SAML assertions with pem and crt files. Upgraded to pysaml2 6.1.0 to mitigate signature mismatch during decryption
* remove print debug statement
* Use utility to find xmlsec binary for encryption, formatting saml_auth module
* format SAML Javascript, revert want_signed_response to pre-PR value
* pysaml2's entityid should point to the sp, not the idp
* add logging for entityid for validation
* use mustache_render instead of string formatting. put all static logic into static branch
* move mustache template for inline saml metadata to the global level
* Incorporate SAML type with Enabled setting
* Update client/app/pages/settings/components/AuthSettings/SAMLSettings.jsx
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Chad Chen <chad.chen@databricks.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* added bar chart boilerplate
* added x/y manipulation
* replaced x/y management to inner series preparer
* added tests
* moved axis inversion to all charts series
* removed line and area
* inverted labels ui
* removed normalizer check, simplified inverted axes check
* finished working hbar
* minor review
* added conditional title to YAxis
* generalized horizontal chart for line charts, resetted state on globalSeriesType change
* fixed updates
* fixed updates to layout
* fixed minor issues
* removed right Y axis when axes inverted
* ran prettier
* fixed updater function conflict and misuse of getOptions
* renamed inverted to swapped
* created mappingtypes for swapped columns
* removed unused import
* minor polishing
* improved series behaviour in h-bar
* minor fix
* added basic filter to ChartTypeSelect
* final setup of filtered chart types
* Update viz-lib/src/components/visualizations/editor/createTabbedEditor.jsx
* added proptypes and renamed ChartTypeSelect props
* Add missing import
* fixed import, moved result array to global scope
* merged import
* clearer naming in ChartTypeSelect
* better lodash map syntax
* fixed global modification
* moved result inside useMemo
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
* Extra actions for Query View and Query Source pages
* Convert Queries List page to functional component
* Convert Dashboards List page to functional component
* Extra actions for Query List page
* Extra actions for Dashboard List page
* Extra actions for Dashboard page
* Pass some extra data to Dashboard.HeaderExtra component
* CR1
* Test Cypress on package list
* Skip Puppeteer Chromium as well
* Put back missing npm install on netlify.toml
* Netlify: move env vars to build.environment
* Remove cypress:install script
* Update Cypress dockerfile
* Copy package-lock.json to Cypress dockerfile
* Directly map query results column to GeoJSON property
* Use cache for geoJson requests
* Don't handle bounds changes while loading geoJson data
* Choropleth: fix map "jumping" on load; don't save bounds if user didn't edit them; refine code a bit
* Improve cache
* Optimize Japan Perfectures map (remove irrelevant GeoJson properties)
* Improve getOptions for Choropleth; remove unused code
* Fix test
* Add US states map
* Convert USA map to Albers projection
* Allow to specify user-friendly field names for maps
* Ask user to log in when session expires
* Update implementation
* Update implementation
* Minor fix
* Update modal
* Do not intercept calls to api/session as Auth.requireSession() relies on it
* Refine code; adjust popup size and position
* expire CSRF tokens after 6 hours
* use axios' built-in cookie to header copy mechanism
* add axios-auth-refresh
* retry CSRF-related 400 errors by refreshing the cookie
* export the auth refresh interceptor to support ejecting it if neccessary
* reject the original request if it's unrelated to CSRF
* Convert TagsList to functional component
* Convert TagsList to typescript
* Allow to unselect all tags
* Add title to Tags block and explicit "clear filter" button
* Some tweaks
* add default limit 1000
* Add frontend changes and connect to backend
* Fix query hash because of default limit
* fix CircleCI test
* adjust for comment
* Refactor CardsList - pass a suffix for list item
Adding :id to an item to be used as a key suffix is redundant and the same
can be accomplished by using :index from the map function.
* Move CardsList to typescript
* Convert CardsList component to functional component
* CR1
* CR2
* Support multiple queries in a single query box
* Implement statement splitting function and add tests for it
* Add a test for databricks-specific syntax
* Split statements before running query
* Introduce Link component
* Use Link component for external links as well
* Remove unused file (I hope it's really not needed)
* Use Link component in visualizations library
* Simplify Link component implementation
* CR1
* Trigger build
* CR2
* Set corejs version in .babelrc so Jest doesn't complain.
* Rewrite services/routes in TypeScript.
* Add TypeScript definitions for DialogComponent.
* Make image paths more portable
* Add current route context and hook.
* Make EmptyState more flexible by being able to pass in getSteps function.
* Rewrite ItemsList in TypeScript.
* Introduce the possibility to add custom sorters for a column.
* Rearrange props to be friendly to TypeScript.
* Type definitions for NotificationApi.
* Use Databricks query editor components for databricks_internal type of query runner.
* URL Escape password in Alembic configuration.
* Compare types in migrations.
* allow non-sequential IDs for DataSources in Cypress tests
* refactor redash-api to a set of Cypress commands
* support mounting Redash endpoints in Cypress routes
* fix some parameter specs by waiting for schema to load
* extract baseUrl from cypress.json
* Restyled by prettier (#5110)
Co-authored-by: Restyled.io <commits@restyled.io>
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <commits@restyled.io>
* add lock table header
* Move styling to a new class
* Update renderer.less
* Move class to table and fix top border
* Update renderer.less
* Update viz-lib/src/visualizations/table/renderer.less
Thanks, this change is good to me.
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* add flask-wtf
* add CSRF tokens to all static forms
* add CSRF tokens to all axios requests
* disable CSRF validation in unit tests
* support CSRF-protected requests in *most* cypress tests
* don't enfroce CSRF checks by default
* avoid CSRF enforcement in unit tests
* remove redundant spread
* some camel casing hiccups
* always yield the CSRF cookie, but avoid enforcing it if CSRF toggle is off
* Restyled by prettier (#5056)
Co-authored-by: Restyled.io <commits@restyled.io>
* set a CSRF header only if cookie is present
* enforce CSRF in CI
* install lodash directly for Cypress
* install request-cookies directly for Cypress. We should probably start loading package.json deps
* enable CSRF support when logout and login happen within the same spec
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <commits@restyled.io>
When using some of the customized login flows such as `REMOTE_USER` the deployed site breaks due to not finding template files. This change updated the app default to use the existing Flask templates directory rather than the compiled static assets directory which only contains an index.html file.
* Add refresh button in the bottom
* Add caching
* Drop allSettled
* Simplify refresh button
* Update error to return 500
* Load tables before loading columns
* Don't mutate schema
* Reset db name and schemas when changing data source
* Load both tables and columns
* Return error with code 200
* Code review updates
* Add expiration time to the cache Keys
* Back with RQ
* getredash/redash#5026 Fix wrong Y-axis range for stacked bar chart
* Update tests
* Use Plotly's built-in algorinthm to compute Y-axis range
* Update tests
* Revert previous solution (yRange-related code)
* Revert other unrelated changes
* Revert other unrelated changes
* Move chart rendering to own file and ensure that rendering steps will occur in necessary order
* Reduce amount of plot updates by mergin separate updates into a sigle cumulative update
* Give better names for several functions
* TASK Add typescript dependencies to package.json
* TASK Add typescript to build process and npm scripts and TASK Move example components to typescript and add an example definition file.
* TASK Move back to ts-loader instead of babel typescript preset
* FIX Remove unnecessary changes
* FIX Explicitly mention tsconfig file in webpack.config.js to avoid `error while parsing tsconfig.json, The 'files' list in config file 'tsconfig.json' is empty`
See (https://github.com/TypeStrong/ts-loader/issues/405#issuecomment-330108362)
* FIX Move tsconfig to client subdirectory to make it accessible in docker container (only webpack.config.js is copied over from root folder in Dockerfile)
* TASK Move from ts-loader to babel to reduce compatibility issues between ES6/7 and typescript compilation.
* TASK Add types for classnames, hoist-non-react-statics and lodash. Fix default export of DashboardList and run prettier on eslintrc
* Run npm install
* Trigger tests
* Run npm install 2
* Trigger tests
* on dashboard api calls - take the id from the beginning of the slug, unless there is no number in it - in that case, take the entire slug as id
* add dashboard id when showing links to dashboards
* change path to include new name when renaming dashboards
* move slug generation to backend
* redirect to new name after changing (this time with a proper promise)
* oh right, we already have a slug function
* add spec that makes sure that renamed dashboards are redirected to the
url which contains their new name
* use id-slug in all Cypress specs
* move dashboards from /dashboard/:slug to /dashboards/:id-:name_as_slug
* Update dashboard url as its name changes
* Update separator to be "/"
* Update missing dashboard urls
* Update api not to depend on int id
* Use '-' instead of '/' as separator and update Dashboard.get calls
* slug -> name_as_slug
* Keep slug urls on cypress
* Update route path
* Use legacy attr for GET
* Use getter for urlForDashboard
* Update dashboard url when loaded by slug
* Update Dashboard routes to use id instead of slug
* Update Dashboard handler tests
* Update Cypress tests
* Fix create new dashboard spec
* Use axios { params }
* Drop Ternary operator
* Send updated slug directly in 'slug' attr
* Update multiple urls Dashboard test name
* Update route names
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
* allow overriding the type of key used for primary/foreign keys of the different models
* rename key_types to singular key_type
* add some documentation for `database_key_definitions`
* Textbox: confirm close if text was changed
* Update texting (with @gabrieldutra)
* Update texting
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* Move components specific to UserProfile page to corresponding folder
* Split UserProfile page into components
* Rename components, refine code a bit
* Add some extension points
* Fix margin
* Split OrganizationSettings page into components
* Update change handling: use objects instead of string keys, move some logic to more appropriate place
* Convert OrganizationSettings page to functional component and refine code a bit
* Add some extension points
* Improve onChange handler
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* Fix CLI command for "status"
CLI command "status" can fail due to incorrect connection information to RQ.
This change matches the behavior from line 65 and solves the connection error.
* Move connection up to CLI entrypoint
* ErrorMessage is not centered
* Adjust ErrorMessage size on large screens
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* Vertical navbar
* Update vertical menu look and add create menu.
* Make query editor work with vertical nav.
* Dark mode
* Fix create menu & make sidebar fixed.
* Update Alert pages layout
* Update System status pages
* Update Queries and Dashboards list pages
* Update Query Source and Query View pages
* Use dark theme for mobile navbar
* Update Dashboard page: fix Add widget/textbox panel positioning
* Dashboard page: fix layout issues when container changes its size (fixes known issues: navbar expand/collapse, scrollbar appears/hides)
* Fix dashboard page sticky header (there was a 15px space above it)
* Fix embeds
* Extract desktop navbar component; move mobile navbar and its styles to ApplicationLayout folder
* Remove old app header
* Fix tests
* Restore version info block
* Make Percy capture entire page
* Make vertical navbar expand/collapse animation smoother (as it's currently impossible to disable it :-( )
* Fix misc UI issues (show Create label on expanded menu; fix some CSS; don't select items on click)
* Allow to override navbars with DynamicComponent
* Fix misc UI issues: expand/collapse button animation, menu items styles, menu expand/collapse animation
* Hide submenu arrow; show username when menu is expanded
* Refine CSS and make it more isolated; adjust colors
* Update tests
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* V9 Changelog: Initial Draft from Jesse
* V9 Changelog: Add later updates
* Adjust title spacing
* Apply Jesse's suggestions
Co-authored-by: Jesse <jesse@whitehouse.dev>
* provide an explanation on how to switch from Celery to RQ when upgrading to v9
* Update CHANGELOG.md
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* Add contributor names
* Update version.
* Update CHANGELOG
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Jesse <jesse@whitehouse.dev>
Co-authored-by: Omer Lachish <omer@rauchy.net>
* Plotly Charts: use .bg to determine legends size
* Test: remove hack for legend below plotly
* Revert "Test: remove hack for legend below plotly"
This reverts commit d8efb0c032.
* Use .legend to calculate bounds
* Also update plots without legend
* Sankey: Make sure last stage has "Exit"
* Sankey: Use 2 as min stage width size to render
* Use null instead of "Exit"
* Add comment about corresponding exit node
* Add multiple stages on Cypress test
* don't join underlying exception message with commas when invalid parameter errors happen
* Revert "don't join underlying exception message with commas when invalid parameter errors happen"
This reverts commit 71a21b7ce6.
* when a problem occurs during refresh_queries, report it as a RefreshQueriesError
* Fix: table viz crashing when search is enabled
* Replace that weird hack with more controlled code
* Don't clear search input, apply search when data changes
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
* Add build arg to Dockerfile to control if we should build frontend assets
* Move more env settings into the shared one.
* Use build arg in docker-compose to skip frontend build.
* CirlceCI: Skip building frontend assets in backend tests
* Create dummy template files
* Expand file names manually.
* Add build arg to skip dev dependencies.
* Update Dockerfile
* Reverse logic of skip_dev_deps to what it should be.
* run queries through adhoc SSH tunnels
* reduce indent by losing try/else clause
* document host/port getters and setters
* handle forceful schema refreshes in RQ and poll for their results using the /jobs endpoint
* set schema refresh timeout to 5 minutes
* Restyled by prettier (#4847)
Co-authored-by: Restyled.io <commits@restyled.io>
* send schema refresh errors as part of API response
* Use correct get_schema call.
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <commits@restyled.io>
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* Add option to explicitly set chart legend position
* Revert some chanes in order to fix tests
* Leave only "auto" and "below the plot" legend placement options
* Move Show legend checkbox to select; fix spelling
* Add visualizations project settings
* Move visualizations to redash-visualizations
* Delete shared components
* Remove antd from deps
* Remove p-r-5 from table utils
* Remove visualization deps from package.json
* Rename package and change its version
* Test preinstall script
* Update Dockerfile build for frontend
* Test adding dockerignore
* Update jest tests
* Add step for jest tests
* Include viz-lib on dev commands
* User prettier v1 for now
* Delete unused libs on the app
* Add readme draft (to be finished)
* Add getOptions to Editor
* Add required libraries and finish basic example
* Bump version
* A new intro paragraph to explain what Redash is
We have been using Redash at Databricks and really love it. I took the time to work with Arik to create a better, more up-to-date description of the project.
* Add data sources
* Fix: showing current settings tab broken in MULTI_ORG.
* Revert "Fix: showing current settings tab broken in MULTI_ORG."
This reverts commit a88defd0b5.
* Add test for SettingsMenu#isActive
* Use stripBase to remove slug from url
* ODBC Based Databricks connector.
* Install Databricks' ODBC driver in Docker image
* Add useragent string.
* Add Types enum to redash.query_runner to replace the seprate constants.
* Databricks connector:
1. Parse types.
2. Send additional connection options.
3. Correctly parse errors.
* Switch to TYPE constants to use code with Python 2.
* Add note about the Databricks driver terms and conditions.
* Show message about Databricks driver terms and conditions.
* Handle cases when the query doesn't return any results.
* Update redash/query_runner/databricks.py
Co-Authored-By: Jesse <jesse@whitehouse.dev>
* Use new Databricks logo
* Fix connection string options
Co-authored-by: Jesse <jesse@whitehouse.dev>
* Move Dashboard off `subqueryload()` loader in all() method due to inconsistent results bug in SQLAlchemy when leveraging distinct within a subqueryload call through paginate.
* Added source reference to Presto Query Runner connection through the pyhive client to announce to presto that the query is coming from `redash` instead of `pyhive`.
* Removing source line from presto query runner to refactor based on feedback.
Let the notifications go into browser/OS notification trays so users can click on them from there if they miss the initial notification. Modern browsers generally use OS notifications so the user is in control of the notification at the OS level.
* Fix comparison error when scale is None
Prevents `'>' not supported between instances of 'NoneType' and 'int'` error when scale is `None`
* Update oracle.py
* Fix scale logic.
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* add pyexasol datasource, ensure that integer dont overflow in javascript
* support setting encryption for Exasol connections
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* feature: add ability to make the restriction of api calls to private addresses optional
* chore: fix typo
* Update redash/settings/__init__.py
Co-authored-by: lprice92 <lprice92@iastate.edu>
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* Limit filters to 40% of query fixed layout space
* Add check for height to determine fixed layout
* Add maxTagCount of 5 to Filters
* Update maxTagCount settings to be similar to Parameters
* feat: provide ssl options for Cassandra data source
* remove Log and prints
* Refactor to create module methods and unit tests
* Switch to using Enumerator and temp file
* Fix temporary file lifecycle for cert
* Align with changes on master
* Fix non certificate but ssl enabled usecase
* Fix query based param with no results crashing page
* Add message for empty dropdown parameters
* Handle 500 no results case with empty result set
* Cypress: Make sure it shows the message
* Use .ant-select-selection to open dropdown
Redash's docker-compose file will no longer bring up an environment from
a cold start due to recent upstream changes to the postgres image that
force the user to either set a password for the default superuser or
opt-in to allowing all connections without a password via environment
variable.
Upstream PR: https://github.com/docker-library/postgres/pull/658
Related Discussion: https://github.com/docker-library/postgres/issues/681
* Cohort: add settings for tooltips, value formats and placeholder
* Cohort: add settings for colors
* Cohort: change all settings tabs to use horizontal inputs
* Cohort: show color labels in editor
* Trigger lint error on warnings on CI
* Test removing pip3 command from frontend unit
* Test eslint warning
* Revert "Test eslint warning"
This reverts commit 89d407345a.
* Revert "Test removing pip3 command from frontend unit"
This reverts commit 424c900200.
* Run apt update before installing pip3
* Stop using route.resolve feature (pages should load all the data themselves)
* Remove route.resolve feature
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* Allow touch action on dashboard grid
* Deactivate touch when resizing widgets
* Disable touch interactions on Plotly
* Update Plotly and use dragmode: false
* Remove autoFocus from ItemsList search
* Fix spacing for queries and dashboard favorites
* Make sure admin pages don't go over 100% width
'postgres' is a default database name in the Docker image, but if an
external database server is used, than Redash database can have
a different name (specified in REDASH_DATABASE_URL).
* Fix: when default value is false make sure it's still stringified.
* Fix: when extra field is of type boolean make sure it's different from default value to decide if it's open.
* Use isNil to check the default value
* Restyled by prettier (#4704)
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
* Snowflake: use different method of showing columns if no schema specified in db name
* Update redash/query_runner/snowflake.py
Co-Authored-By: Omer Lachish <omer@rauchy.net>
* Update redash/query_runner/snowflake.py
Co-authored-by: Omer Lachish <omer@rauchy.net>
* move filtering of invalid schedules to the query
* simplify retrieved_at assignment and wrap in a try/except block to avoid one query blowing up the rest
* refactor refresh_queries to use simpler functions with a single responsibility and add try/except blocks to avoid one query blowing up the rest
* avoid blowing up when job locks point to expired Job objects. Enqueue them again instead
* there's no need to check for the existence of interval - all schedules have intervals
* disable faulty schedules
* reduce FP style in refresh_queries
* report refresh_queries errors to Sentry (if it is configured)
* avoid using exists+fetch and use exceptions instead
* Update oracle.py
The reason I propose this change is to fix an issue when oracle password has an @
example of connection string: user/p@ssword@host
* Update oracle.py
Fixing init after comments
* Remove empty constructor.
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* Realign Data Source and Refresh Schedule
* Adjust execution status height
* Rewrite Query Page Header flexibility
* Remove margin from QuerySource parameters
* Cypress: Visit visualization instead of click in tab
* Fix wrong css class name in dashboard-grid
Because we already call USE DATABASE before running SHOW COLUMNS adding IN DATABASE is redundant, but causes an error if the user specifies a schema along with database name.
* Adds logic to sort column names returned by the query runner. If `sorted`
raises an Exception it returns the column names unaltered from the query
runner.
* Moves table name sorting from model code into schema handler.
* Moves token sorting into the model code.
* Replaces single-quotes with double-quotes for consistency.
* Applies black formatting to changes.
* Moves schema sort into separate method. Adds test.
* Fixes output schema variable name. Without this the sorted cache is never returned!
____ ____ ____ _____
/ __ \/ __ \/ __ \/ ___/
/ /_/ / /_/ / /_/ (__ )
\____/\____/ .___/____/
/_/
* Adds test case guaranteeing that the model actually _uses_ the schema sorter.
Related to a31f90178c
* add meta information to executing queries
* add a table for running queries
* add pagination to queues table
* sort the queues table
* add pagination to all tables
* remove legacy session identifier support
* remove redundant test
* redirect to login to support any invalid session identifiers
* be more specific with caught errors
* reject empty values in DynamicForm
* don't submit form values if they are empty (unless they are
intentionally set to empty string)
* set empty values to null to clear out data source option in the model
* check explicitly for null
* - Added support to specify read preference when query a replicaset database (for example, secondaryPreferred - to try and read data from secondary before primary).
- Removed old code that used MongoClientReplicaSet as it is now just a reference to MongoClient
- Fixed a documentation type :-)
* Moving to PyMongo 3.3.1 which also supports MongoDB 3.2
* Changed the readPreference config to use an enum
* Pass readPreference to MongoClient
* primaryPreferred is now the default
* Move each hook to own file; move hooks and components to own folders
* Update URL and timer only when refresh rate changes
* Skip dashboard refresh if previous refresh is still running
* Fix test
* Add redshift role use option
* Update requirements for SSL socket wrap issue fixes
* Split Redshift class into User and IAM logins
* Update incorrect register
* Change type names
* Correct class name to inherit
* Render IAM redshift image and field order correct
* Update redash/query_runner/pg.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* Update redash/query_runner/pg.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* Remove need for specified urllib - specify pyopenssl is enough
* Pyopenssl back down to 19.0.0
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* Migrate router and <app-view> to React: skeleton
* Update layout on route change
* Start moving page routes from angular to react
* Move page routes to react except of public dashboard and visualization embed)
* Move public dashboard and visualization embed routes to React
* Replace $route/$routeParams usages
* Some cleanup
* Replace AngularJS $location service with implementation based on history library
* Minor fix to how ApplicationView handles route change
* Explicitly use global layout for each page instead of handling related stuff in ApplicationArea component
* Error handling
* Remove AngularJS and related dependencies
* Move Parameter factory method to a separate file
* Fix CSS (replace custom components with classes)
* Fix: keep other url parts when updating location partially; refine code
* Fix tests
* Make router work in multi-org mode (respect <base> tag)
* Optimzation: don't resolve route if path didn't change
* Fix search input in header; error handling improvement (handle more errors in pages; global error handler for unhandled errors; dialog dismiss 'unhandled rejection' errors)
* Fix page keys; fix navigateTo calls (third parameter not available)
* Use relative links
* Router: ignore location REPLACE events, resolve only on PUSH/POP
* Fix tests
* Remove unused jQuery reference
* Show error from backend when creating Destination
* Remove route.resolve where not necessary (used constant values)
* New Query page: keep state on saving, reload when creating another new query
* Use currentRoute.key instead of hard-coded keys for page components
* Tidy up Router
* Tidy up location service
* Fix tests
* Don't add parameters changes to browser's history
* Fix test (improved fix)
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* enforce hard limits on non-responsive work horses by workers
* move differences from Worker to helper methods to help make the specialization clearer
* move HardLimitingWorker to redash/tasks
* move schedule.py to /tasks
* explain the motivation for HardLimitingWorker
* pleasing CodeClimate
* pleasing CodeClimate
* port query execution to RQ
* get rid of argsrepr
* avoid star imports
* allow queries to be cancelled in RQ
* return QueryExecutionErrors as job results
* fix TestTaskEnqueue and QueryExecutorTests
* remove Celery monitoring
* get rid of QueryTask and use RQ jobs directly (with a job serializer)
* Revert "remove Celery monitoring"
This reverts commit 37a74ea403.
* reduce occurences of the word 'task'
* use Worker, Queue and Job instead of spreading names that share behavior details
* remove locks for failed jobs as well
* did I not commit that colon? oh my
* push the redis connection to RQ's stack on every request to avoid verbose connection setting
* use a connection context for tests
* remove Celery monitoring
* 👋 Celery
* remove Celery from Cypress
* black it up
* some more black
* return all started/queued job ids (for future monitoring
* Restyled by prettier (#4522)
* remove celery.py
* remove some frontend residuals that reappeared after a merge
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
* launch and monitor multiple workers using supervisor
* run supervisord in non-daemon mode
* redirect all output to stdout/stderr
* no need to log supervisord's output because it is redirected to stdout anyway
* updated and less brittle healthcheck
* add supervisor healthchecks
* remove redundant supervisor installation as it is installed by pip
* add a 5 minute check gate
* enforce hard limits on non-responsive work horses by workers
* move differences from Worker to helper methods to help make the specialization clearer
* move HardLimitingWorker to redash/tasks
* move schedule.py to /tasks
* explain the motivation for HardLimitingWorker
* pleasing CodeClimate
* pleasing CodeClimate
* port query execution to RQ
* get rid of argsrepr
* avoid star imports
* allow queries to be cancelled in RQ
* return QueryExecutionErrors as job results
* fix TestTaskEnqueue and QueryExecutorTests
* remove Celery monitoring
* get rid of QueryTask and use RQ jobs directly (with a job serializer)
* Revert "remove Celery monitoring"
This reverts commit 37a74ea403.
* reduce occurences of the word 'task'
* use Worker, Queue and Job instead of spreading names that share behavior details
* remove locks for failed jobs as well
* did I not commit that colon? oh my
* push the redis connection to RQ's stack on every request to avoid verbose connection setting
* use a connection context for tests
* black it up
* run RQ on all queues when running in Cypress
* Allow executing query with either view_query or execute_query permissions.
* Render AuthHeader according to permissions.
* Don't return dashboards where you only have access to textbox widget.
Closes#4099.
* Prettier all the JS files
* Add GitHub Action to autoformat code pushed to master
* Fix eslint violation due to formatting.
* Remove GitHub actions for styling
* Add restyled.io config
* Remove app/service/query-string (unused) and its dependency.
* Fix usage of mixed operators.
* eslint --fix fixes for missing dependencies for react hooks
* Fix: useCallback dependency passed to $http's .catch.
* Satisfy react/no-direct-mutation-state.
* Fix no-mixed-operators violations.
* Move the decision of whether to render Custom chart one level up to make sure hooks are called in the same order.
* Fix: name was undefined. It wasn't detected before because there is such global.
* Simplify eslint config and switch to creat-react-app's eslint base.
* Add prettier config.
* Make sure eslint doesn't conflict with prettier
* A few updates post eslint (#4425)
* Prettier command in package.json
* crude unit tests for counter visualisation utils
* improve type safety with default param values for getCounterData()
* fix count rows never shows zero
* remove default values for getCounterData() params
* Remove --max-old-space-size=4096 from build
Looks like it's no longer needed.
* Update to node v12.
* Add build:old-node-version for those who have Node < 12.
* Add support for configuring a Username/Password for the connection to Druid
* Bump pydruid version for username/password support
* Deal with missing/empty configuration parameters
* refreshing snowflake schema w/o waking cluster
Have also added a new internal method to not select a
warehouse while executing query
Using 'show columns' to fetch database schema instead of
executing a select query in information schema
show columns does not require a warehouse to run
* modularising snowflake code to avoid repetitions
fixing internal function syntax and avoiding
code repetition
* removing user object in snowflake schema query
* Migrate Funnel visualization to React: Editor
* Migrate Funnel visualization to React: Renderer
* Replace Auto sort options with Sort Column + Reverse Order
* Add option for items limit (instead of hard-coded value)
* Add number formatting options
* Replace d3.max with lodash.maxBy; fix bug in prepareData
* Add options for min/max percent values
* Debounce inputs
* Tests
* Refine Renderer: split components, use Ant Table for rendering, fix data handling
* Extract utility function to own file
* Fix tests
* Fix: sometimes after updating options, funnel shows "ghost" rows from previous dataset
* Sort by value column by default
* Migrate Cohort to React: Editor
* Extract prepareData and getOptions to own files
* Refine CohortRenderer Angular component (js, less, prepareData) for easier migration
* Migrate Cohort to React: Renderer
* Migrate Cornelius to React: styles
* Migrate Cohort to React: Cornelius library
* Cornelius: add licence info; remove unused style
* Cornelius: use numeral to format numbers; revisit styles
* Cornelius: use moment to format date labels
* Cornelius: use chroma for cell backgrounds; update options; update proptypes; minor fixes
* Tidy up
* Tests
* add some logging to scheduler
* schedule jobs only if they are not already scheduled
* jobs scheduled with an interval over 24 hours were not repeated
* schedule version_check using standard scheduling
* clean up old jobs that are not part of the definition anymore
* add some tests
* add one more test to verify that reschedules are not done when not neccesary
* no need to check for func existence - all jobs have a func to run
* Migrate settings-screen to React
* Use black instead of blue color for active item
* Revert "Use black instead of blue color for active item"
This reverts commit 0e4ececa6a.
* Add selectable=false to the Menu
* add some logging to scheduler
* clean failed RQ job data from Redis
* move stale job purging to tasks/general.py
* provide better documentation on why we don't reject keys in FailedJobRegistry at the moment
* pleasing the CodeClimate overlords
* simplified clenaup by deleting both job data and registry entry
* use FailedJobRegistry as source of truth for purging
* remove redundant key deletion
* Update redash/settings/__init__.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* CircleCI workflow improvements
- Don't automatically build the Docker image.
- Make the Python lint step requirement for the follow up steps. When it fails it usually means there is a code error which will prevent the next steps anyway.
* Fix YAML syntax error.
* Add separate build Docker image step for master branch
* Start draft for new Parameter structure
* Add the rest of the methods
* EnumParameter
* QueryBasedDropdownParameter
* DateParameter
* DateRangeParameter
* Update Parameter usage on code
* Merge dynamicValue into normalizedValue
* Add updateLocals and omit unwanted props
* Allow null NumberParameter and omit parentQueryId
* Rename parameter getValue to getExecutionValue
* Update $$value to normalizedValue + omit on save
* Add a few comments
* Remove ngModel property from Parameter
* Use value directly in DateRangeParameter
* Use simpler separator for DateRange url param
* Add backward compatibility
* Use normalizeValue null value for isEmpty
* Start creating jest tests
* Add more tests
* Normalize null value for multi mode in Enum
* Use saved value for param isEmpty
* Make core app compatible with Python 3
No backward compatibility with Python 2.7 is kept.
This commit mostly contains changes made with 2to3 and manual
tweaking when necessary.
* Use Python 3.7 as base docker image
Since it is not possible to change redash/base:debian to Python 3
without breaking future relases, its Dockerfile is temporarly
copied here.
* Upgrade some requirements to newest versions
Some of the older versions were not compatible with Python 3.
* Migrate tests to Python 3
* Build frontend on Python 3
* Make the HMAC sign function compatible with Python 3
In Python 3, HMAC only works with bytes so the strings and the
float used in the sign function need to be encoded.
Hopefully this is still backward compatible with already generated
signatures.
* Use assertCountEqual instead of assertItemsEqual
The latter is not available in Python 3.
See https://bugs.python.org/issue17866
* Remove redundant encoding header for Python 3 modules
* Remove redundant string encoding in CLI
* Rename list() functions in CLI
These functions shadow the builtin list function which is
problematic since 2to3 adds a fair amount of calls to the builtin
list when it finds dict.keys() and dict.values().
Only the Python function is renamed, from the perspective of the
CLI nothing changes.
* Replace usage of Exception.message in CLI
`message` is not available anymore, instead use the string
representation of the exception.
* Adapt test handlers to Python 3
* Fix test that relied on dict ordering
* Make sure test results are always uploaded (#4215)
* Support encoding memoryview to JSON
psycopg2 returns `buffer` objects in Python 2.7 and `memoryview`
in Python 3. See #3156
* Fix test relying on object address ordering
* Decode bytes returned from Redis
* Stop using e.message for most exceptions
Exception.message is not available in Python 3 anymore, except
for some exceptions defined by third-party libraries.
* Fix writing XLSX files in Python 3
The buffer for the file should be made of bytes and the actual
content written to it strings.
Note: I do not know why the diff is so large as it's only a two
lines change. Probably a white space or file encoding issue.
* Fix test by comparing strings to strings
* Fix another exception message unavailable in Python 3
* Fix export to CSV in Python 3
The UnicodeWriter is not used anymore. In Python 3, the interface
provided by the CSV module only deals with strings, in and out.
The encoding of the output is left to the user, in our case
it is given to Flask via `make_response`.
* (Python 3) Use Redis' decode_responses=True option (#4232)
* Fix test_outdated_queries_works_scheduled_queries_tracker (use utcnow)
* Make sure Redis connection uses decoded_responses option
* Remove unused imports.
* Use Redis' decode_responses option
* Remove cases of explicit Redis decoding
* Rename helper function and make sure it doesn't apply twice.
* Don't add decode_responses to Celery Redis connection URL
* Fix displaying error while connecting to SQLite
The exception message is always a string in Python 3, so no
need to try to decode things.
* Fix another missing exception message
* Handle JSON encoding for datasources returning bytes
SimpleJSON assumes the bytes it receives contain text data, so it
tries to UTF-8 encode them. It is sometimes not true, for instance
the SQLite datasource returns bytes for BLOB types, which typically
do not contain text but truly binary data.
This commit disables SimpleJSON auto encoding of bytes to str and
instead uses the same method as for memoryviews: generating a
hex representation of the data.
* Fix Python 3 compatibility with RQ
* Revert some changes 2to3 tends to do (#4261)
- Revert some changes 2to3 tends to do when it errs on the side of caution regarding dict view objects.
- Also fixed some naming issues with one character variables in list comprehensions.
- Fix Flask warning.
* Upgrade dependencies
* Remove useless `iter` added by 2to3
* Fix get_next_path tests (#4280)
* Removed setting SERVER_NAME in tests setup to avoid a warning.
* Change get_next_path to not return empty string in case of a domain only value.
* Fix redirect tests:
Since version 0.15 of Werkzeug it uses full path for fixing the location header instead of the root path.
* Remove explicit dependency for Werkzeug
* Switched pytz and certifi to unbinded versions.
* Switch to new library for getting country from IP
`python-geoip-geolite2` is not compatible with Python 3, instead
use `maxminddb-geolite2` which is very similar as it includes
the geolite2 database in the package .
* Python 3 RQ modifications (#4281)
* show current worker job (alongside with minor cosmetic column tweaks)
* avoid loading entire job data for queued jobs
* track general RQ queues (default, periodic and schemas)
* get all active RQ queues
* call get_celery_queues in another place
* merge dicts the Python 3 way
* extend the result_ttl of refresh_queries to 600 seconds to allow it to continue running periodically even after longer executions
* Remove legacy Python flake8 tests
* add rq and an rq_worker service
* add rq_scheduler and an rq_scheduler service
* move beat schedule to periodic_jobs queue
* move version checks to RQ
* move query result cleanup to RQ
* use timedelta and DRY up a bit
* move custom tasks to RQ
* do actual schema refreshes in rq
* rename 'period_jobs' to 'periodic', as it obviously holds jobs
* move send_email to rq
* DRY up enqueues
* ditch and use a partially applied decorator
* move subscribe to rq
* move check_alerts_for_query to rq
* move record_event to rq
* make tests play nicely with rq
* 👋 beat
* rename rq_scheduler to plain scheduler, now that there's no Celery scheduler entrypoint
* add some color to rq-worker's output
* add logging context to rq jobs (while keeping execute_query context via get_task_logger for now)
* move schedule to its own module
* cancel previously scheduled periodic jobs. not sure this is a good idea.
* rename redash.scheduler to redash.schedule
* allow custom dynamic jobs to be added decleratively
* add basic monitoring to rq queues
* add worker monitoring
* pleasing the CodeClimate overlords
* adjust cypress docker-compose.yml to include rq changes
* DRY up Cypress docker-compose
* add rq dependencies to cypress docker-compose service
* an odd attempt at watching docker-compose logs when running with Cypress
* Revert "an odd attempt at watching docker-compose logs when running with Cypress"
This reverts commit 016bd1a93e.
* show docker-compose logs at Cypress shutdown
* Revert "DRY up Cypress docker-compose"
This reverts commit 43abac7084.
* minimal version for binding is 3.2
* remove unneccesary code reloads on cypress
* add a command which errors if any of the workers running inside the current machine haven't been active in the last minute
* SCHEMAS_REFRESH_QUEUE is no longer a required setting
* split tasks/queries.py to execution.py and maintenance.py
* fix tests after query execution split
* pleasing the CodeClimate overlords
* rename worker to celery_worker and rq_worker to worker
* use /rq_status instead of /jobs
* show started jobs' time ago according to UTC
* replace all spaces in column names
* fix query tests after execution split
* exit with an int
* general lint
* add an entrypoint for rq_healthcheck
* fix indentation
* delete all existing periodic jobs before scheduling them
* remove some unrequired requires
* move schedule example to redash.schedule
* add RQ integration to Sentry's setup
* pleasing the CodeClimate overlords
* remove replication settings from docker-compose - a proper way to scale using docker-compose would be the --scale CLI option, which will be described in the knowledge based
* revert to calling a function in dynamic settings to allow periodic jobs to be scheduled after app has been loaded
* don't need to depend on context when templating failure reports
* set the timeout_ttl to double the interval to avoid job results from expiring and having periodic jobs not reschedule
* whoops, bad merge
* describe custom jobs and don't actually schedule them
* fix merge
* don't need to depend on context when templating failure reports
* extract a render_template function with some docs
* CodeClimate has really outdone itself this time. Removed a whitespace character in order to fix 2 CodeClimate errors
* apparently whitespace doesn't count as a character
* pagerduty.py: Change default summary text
Change is made to use alert name in PagerDuty's alert
destination default summary text instead of
query id and name
* Update default description text & field description
* Add interface to implement custom persistence for QueryResult data
Co-authored-by: Omer Lachish <omer@rauchy.net>
* Deserialize query results data in the model
* Change order of mixins.
* Make DBPersistence.data setter in sycn with getter + tests
* The sender email address has to be None for the test of "is email server
configured" to be correct. Moved the dev setting into docker-compose.yml.
* Move the REDASH_MAIL_SERVER setting into docker-compose.yml to revert the default value to its original value in case anyone was using it.
* Make worker dependant on email as it's the one that actually using it.
* DynamicForm support for advanced options
* Randomly select a few options to be advanced
* Merge conditions with the same logic
* Address some comments
* Update styling for the button
* Some style adjustments (#4162)
* Don't set default value to additional settings
* Rename advanced -> extra
* Show extra fields by default when they are filled
* Update hasFilledExtraField logic
* Add example field from destination as extra
* Support for predefined options in data sources
* DynamicForm Select: title -> name
* Make it work with "enum" prop
* Make it work for "extendedEnum" prop
* Not JS
* Deep copy the configuration schema
* Handle non-array in multi-value QueryBasedParameter
* Use state value in QueryBasedParameterInput
* Normalize array in parameter structure
* Add Multi-selection test
* Remove unnecessary not null check
* npm install react-pivottable
* Initiate Pivot Table Migration
* Update renderer with editor options
* Clean up
* Remove old pivottable from package.json
* Test Percy Snapshot with Pivot Table in a Dashboard
* Tmp: use cy.wait to make sure dashboard is loaded
* Clean up Percy snapshot test
* Small improvements
- cy.all with multiple args
- add controls to pivot valid options
* Watch for options in the Renderer
* Improve sizing for Number inputs
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Migrate WidgetDialog
* Start migrating Widget
* Update textbox to use HtmlContent
* QueryLink migration and some updates
* Add visualization rendering
* Render widget
* Add delete button
* Update AutoHeight
* Add widget bottom
* Add Drodpown button
* Split Widget component
* Update with #4056 and trigger netlify
* In progress: use composition
* Add header and footer
* Update widget actions positioning
* Re-render when refreshing from widget
* Add workaround to force DashboardGrid re-render
* VisualizationWidgetFooter component
* VisualizationWidget menu
* Separate RestrictedWidget
* Update tests
* Update margin for Parameters
* Remove widget files
* Revert "Improve sizing for Number inputs"
This reverts commit a02ce8f0aa.
* Some cleanup
* Move refresh logic to the Dashboard
* Add loadingWidgets logic to the public dashboard
* Add onLoadWidget
* Remove parameter from URL when empty
* Recreate widget array instead of loadingWidgets
* Add comment about re-rendering + whitespace missing
* CR changes
* Use plain html instead of string syntax
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Snippets: Don't change url when not needed
* Revert "Snippets: Don't change url when not needed"
This reverts commit 2f346f3bb4.
* Query Snippets: use onClick instead of link
* Initial commit of BeaconConsent component
* Add comment about being able to change setting
* Use <Text> correctly
* Final version of consent screen
* Show beacon consent message on homepage only if it wasn't enabled already.
* Add consent setting to organization settings screen.
* Add support for custom message in OrgSetting.save.
* Implmenet consent saving.
* If consent given, send extra data
* Add HelpTrigger
* Make CodeClimate happy
* Wrap everything with DynamicComponent
* getredash/redash#2629 Refactor Chart visualization, add option for handling NULL values (keep/convert to 0.0)
* Handle null values in line/area stacking code; some cleanup
* Handle edge case: line/area stacking when last value of one of series is missing
* Mjnor update to line/area stacking code
* Fix line/area normalize to percents feature
* Unit tests
* Refine tests; add tests for prepareLayout function
* Tests for prepareData (heatmap) function
* Tests for prepareData (pie) function
* Tests for prepareData (bar, line, area) function
* Tests for prepareData (scatter, bubble) function
* Tests for prepareData (box) function
* Remove unused file
* Code formatting
* Move annotation logic into query runner, so it can be overriden in the query runner.
* Add mixin to __all__
* Switch to flag instead of mixin
* Feature (Redshift): option to set query group for adhoc/scheduled queries (#4114)
* Add scheduled status to query job metadata.
* Add: option to set query group for adhoc/scheduled Redshift queries
* Scheduled might not be set for already enqueued queries.
* refresh only affected queries in dashboard when parameters are changed
* rename pendingParameters to updatedParameters
* select which widgets to update according to their mapping as a dashboard-level parameter
* use lodash's include
* [Qubole] - Adding support to process Quantum query types.
Quantum is a serverless interactive service that offers
direct SQL access to user's data lake. Changes are made
to accept `quantum` query type from user which makes
`Cluster Label` as optional.
* -Making quantum as defult query.
-Dictionary safe access to connection parmeters
* keeping pep8 standards
* Maintainig pep8 std
* Use latest version of qds-sdk
* Use qds-sdk v1.13.0
* Use qds-sdk v1.12.0
* Use qds-sdk v1.13.0
* Updating SDK with verified version
* hive as default query type
* qds-sdk : Locking most recent release version
* qds-sdk : Locking recent release version
* falling back to original version of qds-sdk
* [Data Sources] Add: Azure Data Explorer (Kusto) query runner
* CodeClimate fixes
* Remove TODO
* Fixed configuration properties names for Azure Kusto
* Azure Kusto: get_schema in one query
* azure-kusto-data update to 0.0.32
* Add Kusto to the default query runners list
kombu is a dependency of Celery and usually we let them declare the version, but their version is pinned and the most recent release (4.6.4) has a severe regression where workers stop responding to inspect commands.
* Initial commit for adding Dgraph support
* Made suggestions from https://codeclimate.com/github/getredash/redash/pull/3964
* feedback from @arikfr
* added logo for Dgraph from Twitter
* Better conversion of Dgraph JSON to Redash's internal JSON
* made recommendations from @arikfr
* removed unused function
* Fix loading of periodic tasks and clean up extension loading.
This does a few things:
- add tests for extension loading
- refactor the extension and periodic task loading
- better handle assertions raised by extensions (e.g. when an extension tries to override an already registered view)
- attach exception traceback to error log during loading for improved debugging
* Use site.addsitedir instead of calling pip.
* Use sys.path instead of site.addsitedir and also the setup.py egg_info command.
* allowing to specify a custom work group for AWS Athena queries
* Fixing title + adding correct position in the UI
* Adding assume role configuration to Athena query runner.
* removing extra blank lines
* fixes based on comments to the PR
With the default Docker installed from sources on Ubuntu 19.04 it failed starting the project when asking for Compose version 3.7, but everything worked fine with 3.2.
* Fix W292 no newline at end of file
* Fix extra whitespace
* Fix E305 expected 2 blank lines after class or function definition
* Fix W391 blank line at end of file
* Fix E231 missing whitespace after
* Fix E303 too many blank lines
* Fix E302 expected 2 blank lines
* Fix E128 continuation line under-indented for visual indent
* avoid using 'abort' in parameterized query - raise an exception instead
* when facing invalid parameters or detached dropdown queries - continue to refresh the rest of the outdated queries
* test that dropdown queries detached from data source raise an exception when fetch values is attempted
* test that queries with invalid parameters arent refreshed
* test that queries with dropdown query parameters which are detached from the data source are skipped
* fix stale test double name
* newlines. newlines everywhere.
* pass org into dropdown_values
* pass in org in every ParameterizedQuery usage
* Update redash/tasks/queries.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* reduce refresh_queries log noise
* track failure count for queries that failed to apply parameters, and also notify the failures
* Update redash/tasks/queries.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* newlines. newlines everywhere.
* Allow multiple values for enum parameter
* Allow multi-select for Query dropdown parameters
* CR + make sure list values are allowed
* Add prefix, suffix and separator
* Rename multipleValues and cast options as strings
* Replicate serialization logic on frontend
* Add Quote Option Select
* Make sure it's enum or query before join
* Add a couple of tests
* Add help to quote option
* Add min-width and normalize empty array
* Improve behavior when changing parameter settings
- Set parameter value again to pass through checks
- Add setValue check for multi values
* Validate enum values on setValue + CodeClimate
* Ran wording suggestions
* Updates after Apply Changes
* Fix failing Cypress tests
* Make sure enumOptions exists before split
* Improve propTypes for QueyBasedParameterInput
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* CR
* Cypress: Test for multi-select Enum
* Fix multi-selection Cypress spec
* Update Refresh Schedule
* initial work on e-mail report for failed queries
* send failure report only for scheduled queries and not for adhoc queries
* add setting to determine if to send failure reports
* add setting to determine interval of aggregated e-mail report
* html templating of scheduled query failure report
* break line
* support timeouts for failure reports
* aggregate errors within message and warn if approaching threshold
* handle errors in QueryExecutor.run instead of on_failure
* move failure report to its own module
* indicate that failure count is since last report
* copy changes
* format with <code>
* styling, copy and add a link to the query instead of the query text
* separate reports with <hr>
* switch to UTC
* move <h2> to actual e-mail subject
* add explicit message for SoftTimeLimitExceeded
* fix test to use soft time limits
* default query failure threshold to 100
* use base_url from utils
* newlines. newlines everywhere.
* remove redundant import
* apply new design for failure report
* use jinja to format the failure report
* don't show comment block if no comment is provided
* don't send emails if, for some reason, there are no available errors
* subtract 1 from failure count, because the first one is represented by 'Last failed'
* don't show '+X more failures' if there's only one
* extract subject to variable
* format as text, while we're at it
* allow scrolling for long exception messages
* test that e-mails are scheduled only when beneath limit
* test for indicating when approaching report limits + refactor
* test that failures are aggregated
* test that report counts per query and reason
* test that the latest failure occurence is reported
* force sending reports for testing purposes
* Update redash/templates/emails/failures.html
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Update redash/templates/emails/failures.html
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Update redash/tasks/failure_report.py
* add org setting for email reports
* remove logo from failure report email
* correctly use the organization setting for sending failure reports
* use user id as key for failure reports data structure
* Update redash/tasks/failure_report.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* build comments while creating context for e-mail templates
* figure out the base url when creating the e-mail
* no need to expire pending failure report keys as they are deleted anyway when sent
* a couple of CodeClimate changes
* refactor key creationg to a single location
* refactor tests to send e-mail from a single function
* use beat to schedule a periodic send_aggregated_errors task instead of using countdown per email
* remove pending key as it is no longer required when a periodic task picks up the reports to send
* a really important blank line. REALLY important.
* Revert "a really important blank line. REALLY important."
This reverts commit c7d8ed8972.
* a really important blank line. REALLY important. It is the best blank line.
* don't send failure emails to disabled users
* Draft for Date Dynamic values
* Use value with prefix instead of specific attr
* Fix not possible to select static value
* Update antd version
* Cleanup and DateRangeParameter
* Dynamic DateTimeRange
* Add Dynamic options to Date Parameters
* UI refinements
* Add getDynamicValue function
* Add 'This' options and prevent text clipping
* Make allowClear available
* Update ScheduleDialog snapshot
* Add some protections and separate Date/DateRange
* Accept null values on date or daterange parameters
* Handle undefined values on Moment propType
* Move export to end of files
* Remove Today/Now option
* Update with Apply Changes
* Show name instead of value for dynamic values
* Add comment about supporting useCurrentDateTime
* Cypress Tests: Date Parameters
* Cypress Tests: Date Range Parameters
* Don't put null params in the url
* Add workaround comments to Cypress tests
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Fix Dynamic Value as default for global parameters
* Update Back to Static Value
* Add isValid to value on Date and DateRange inputs
* CR suggestions
* Fix Back to Static Value for Dates
* Update Dynamic Value Styling
* Fix failing Date tests
* Fix selectedDynamicValue
* Parameter spec: Remove date range clickThrough
* Add transition
* Fix failing Cypress tests
* Back with 'width: auto'
* Check value is valid on Back to Static value
* CR
* Update Date Range width
* add a deprecated flag to query runners and show only non-deprecated query runners when adding a new data source
* add a deprecated flag to alert destinations and show only non-deprecated alert destinations when adding a new alert destination
* add a deprecated() decorator for a more succint way to deprecate
* deprecate URL query runner and HipChat alert destination
* use class properties instead of class methods for deprecation
* I <3 newlines
* return message explaining unsafe sharing
* use backend-generated message for public dashboards
* use backend-generated message for embeds
* Update redash/handlers/query_results.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* refactor simple (non-interpolated) query result handler error messages to a single location
* use error_messages to test out unsafe error messages (along with a couple of others)
* Update redash/handlers/query_results.py
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Update redash/handlers/query_results.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* avoid catching errors on text widgets' load(), as they don't have a visualization and therefore do not return any promise
* throw error when failing to load widgets on public dashboards - in case something needs to be done with it at a later time, and it's the right thing to do anyway
* use Promise.resolve instead of checking for undefined
* call serialize_query_result instead of directly calling to_dict
* filter unneeded query result fields for unauthenticated users
* test for serialization filtering
* lint
* use project instead of list comprehension
* pick up *.py file changes and restart scheduler
* only watch /redash in order to avoid reloading on other file changes (e.g. tests)
* add dev_scheduler entrypoint
* use exec
* Update bin/docker-entrypoint
* rename dev_scheduler to dev_worker
* use same defaults as worker
* avoid catching errors on text widgets' load(), as they don't have a visualization and therefore do not return any promise
* throw error when failing to load widgets on public dashboards - in case something needs to be done with it at a later time, and it's the right thing to do anyway
* use Promise.resolve instead of checking for undefined
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* use the textless endpoint (/api/queries/:id/results) for pristine
queriest
* Revert "use the textless endpoint (/api/queries/:id/results) for pristine"
This reverts commit cd2cee7738.
* go to textless /api/queries/:id/results by default
* change `run_query`'s signature to accept a ParameterizedQuery instead of
constructing it inside
* raise HTTP 400 when receiving invalid parameter values. Fixes#3394
* enqueue jobs for ApiUsers
* rename `id` to `user_id`
* support executing queries using Query api_keys by instantiating an ApiUser that would be able to execute the specific query
* show deprecation messages for ALLOW_PARAMETERS_IN_EMBEDS. Also, move
other message (email not verified) to use the same mechanism
* add link to forum message regarding embed deprecation
* change API to /api/queries/:id/dropdowns/:dropdown_id
* split to 2 different dropdown endpoints and implement the second
* add test cases for /api/queries/:id/dropdowns/:id
* use new /dropdowns endpoint in frontend
* first e2e test for sharing embeds
* Pleasing the CodeClimate overlords
* All glory to CodeClimate
* remove residues from bad rebase
* add query id and data source id to serialized public dashboards
* add global parameters directive to public dashboards page
* allow access to a query by the api_key of the dashboard which includes
it
* rename `object` to `obj`
* simplify permission tests once `has_access` accepts groups
* support global parameters for public dashboards
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* rename `object` to `obj`
* simplify permission tests once `has_access` accepts groups
* no need to log `is_api_key`
* send parameters to public dashboard page
* allow access to a query by the api_key of the dashboard which includes it
* disable sharing if dashboard is associated with unsafe queries
* remove cypress test added in the wrong place due to a faulty rebase
* add support for clicking buttons in cy.clickThrough
* Cypress test which verifies that dashboards with safe queries can be shared
* Cypress test which verifies that dashboards with unsafe queries can't be shared
* remove duplicate tests
* use this.enabled and negate when needed
* remove stale comment
* add another Cypress test to verify that unauthenticated users have access to public dashboards with parameters
* obviously, I commit 'only' the first time I use it
* search for query access by query id and not api_key
* no need to fetch latest query data as it is loaded by frontend from the textless endpoint
* test that queries associated with dashboards are accessible when supplying the dashboard api_key
* propagate `isDirty` down to `QueryBasedParameterInput`
* go to /api/:id/dropdown while editing a query, since dropdown queries might still not be associated with the parent. see #3711
* show helpful error message if dropdown values cannot be fetched
* use backticks instead of line concatenation
* remove requirement to have direct access to dropdown query in order validate it. parent query association checks are sufficient
* remove isDirty-based implementation and allow dropdown queries through nested ACL even if they aren't associated yet (given that the user has _direct_ access to the dropdown query)
* fix tests to cover all cases for /api/queries/:id/dropdowns/:id
* fix indentation
* require access to the query, not the data source
* resolve dashboard user by query id
* apply new copy to Cypress tests
* if only something would have prevented me from commiting an 'only' call 🤔
* very important handling of whitespace
* respond to parameter's Apply button
* text widgets are safe for sharing
* remove redundant event
* add a safety check that object has dashboard_api_keys before calling it
* supply a parameter value for text parameters to have it show up
* add parameter values for date and datetime
* use the current year and month to avoid pagination
* use Cypress.moment() instead of preinstalled moment()
* explicitly create parameters
* refresh query data if a querystring parameter is provided
* avoid sending a data_source_id - it's only relevant to unsaved queries, since a saved query's data_source is available in the backend
* remove empty query text workaround
* provide default value to parameter
* add a few more dashboard sharing specs
* lint
* wait for DynamicTable to appear to reveal that actual results are displaying
* override error message for unsafely shared widgets
* Support multi-byte search for query names and descriptions
* add multi_byte_support_enabled option to organization settings
* add `ilike %...%` to query search conditions when the option is enabled
* Improve description for multi_byte_search_enabled option
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* Remove tsvector from search when multi_byte_search_enabled
* Add a multi-byte search test case
* Allow calling query results endpoint without parameters.
* Fix: allow serializing empty or bad dates
* Revert "Fix: allow serializing empty or bad dates"
This reverts commit cc49319d9e.
* supply a parameter value for text parameters to have it show up
* add parameter values for date and datetime
* use the current year and month to avoid pagination
* use Cypress.moment() instead of preinstalled moment()
* capture time before clicking on Now
* use now from input
* use now from input for another test
* Add regenerate function of query's API Key
* Add regenerate API Key button
* Add regenerate Query API Key tests
* Fix too long line
* Replace with this
* Return a simple version query
* Update only API Key
* Update API Key via query
* Celery task to clear schedule was added
* fix formating
* empty_schedules task was put in separate task
* worker interval changed, new tests added
* past artifact deleted
* test queries moved to right class, lambda was used to filter data
* unnecessary changes eliminated
* more unnecessary files deleted
* line shortened
* Line shortened more
* codeclimate changes
* Unused test deleted, logs added
* remove legacy session identifier support
* remove redundant test
* redirect to login to support any invalid session identifiers
* be more specific with caught errors
* use authorization according to api_key (if provided) over session
* flatten lists and dicts to json to be used with SQLite's json_extract functions
* add test that verifies that lists and dicts are saved
* add test that verifies that lists and dicts are saved
* remove legacy session identifier support
* remove redundant test
* redirect to login to support any invalid session identifiers
* be more specific with caught errors
* fix refresh for public dashboards
* remove legacy session identifier support
* remove redundant test
* redirect to login to support any invalid session identifiers
* be more specific with caught errors
* Update PromiseRejectionError to show error message from API response
* Update version to 8.0.0-beta.
* Revert "Update version to 8.0.0-beta."
This reverts commit c8fa74967f.
* Decouple extensions from Flask app.
This separates the extension registry from the Flask app and also introduces a separate registry for preriodic tasks.
Fix#3466.
* Address review feedback.
* Update redash/extensions.py
Co-Authored-By: jezdez <jannis@leidel.info>
* Minor comment in requirements.
* Refactoring after getting feedback.
* Uncoupled bin/bundle-extensions from Flas app instance.
* Load bundles in bundle script and don’t rely on Flask.
* Upgraded to importlib-metadata 0.9.
* Add missing requirement.
* Fix TypeError.
* Added requirements for bundle_extension script.
* Install bundles requirement file correctly.
* Decouple bundle loading code from Redash.
* Install bundle requirements from requirements.txt.
* Use circleci/node for build-docker-image step, too.
* Dashboard grid React migration
* Updated tests
* Fixes comments
* One col layout
* Tests unskipped
* Test fixes
* Test fix
* AutoHeight feature
* Kebab-cased
* Get rid of lazyInjector
* Replace react-grid-layout with patched fork to fix performance issues
* Fix issue with initial layout when page has a scrollbar
* Decrease polling interval (500ms is too slow)
* Rename file to match it's contents
* Added some notes and very minor fixes
* Fix Remove widget button (should be visible only in editing mode); fix widget actions menu
* Fixed missing grid markings
* Enhanced resize handle
* Updated placeholder color
* Render DashboardGrid only when dashboard is loaded
* Add touch state to parameters and autoupdate query
* Use values change event instead of $watch
* Remove getQueryResultDebounced
* Add Apply button
* Remove Input Number spinners for Parameters
* Make Apply Button optional
* Update share_embed_spec
* Change debounce to the Parameters component
* Remove unnecessary click on Execute query
* Add apply button to the remaining places
* Update dashboard_spec
* Use onKeyUp for InputNumber
* Simplify onParametersValuesChanged
* Update DateTime onChange function
* Don't apply when modifier key is pressed
* Remove refresh Button from Parameters
* Update apply button styling
* Update apply right distance
* Remove debounce for testing
* Use data-dirty instead of classNames for styling
* Make sure $apply runs before calling onChange
* If MySQL returns multiple resultSets (eg when executing multiple statements, or calling stored procedures) then use the last resultSet that has columns
* Make cancellation of a MySQL query work (the same way the C client does it)
* Address code climate moans
* Migrate Organization Settings to React
* Fix failing spec and replace default values from inputs
* Add HelpTrigger and handleChange to SAML options
* Undo changes to ant-variables.less
* Add time format to OrganizationSettings
* propagate `isDirty` down to `QueryBasedParameterInput`
* go to /api/:id/dropdown while editing a query, since dropdown queries might still not be associated with the parent. see #3711
* show helpful error message if dropdown values cannot be fetched
* use backticks instead of line concatenation
* remove requirement to have direct access to dropdown query in order validate it. parent query association checks are sufficient
* remove isDirty-based implementation and allow dropdown queries through nested ACL even if they aren't associated yet (given that the user has _direct_ access to the dropdown query)
* fix tests to cover all cases for /api/queries/:id/dropdowns/:id
* fix indentation
* require access to the query, not the data source
* use require_access instead of has_access
* provide queryId when fetching query results in order to allow query-based api-key authentication to work properly
* cypress test to verify that embeds without parameters are shared succesfully
* rename Percy snapshot
* add an endpoint for fetching job using a query's api_key
* when unauthenticated, use api_key to get job, and fetch the latest query
result (as opposed to fetching the query result by ID)
* add 'refresh dataset' button to parameters directive
* fix scope error introduced by earlier commit
* show timer when refreshing results
* Show input for missing parameters in embedded visualizations (#3741)
* Redirect to default parameter values when parameters are missing in
embedded visualizations
* Revert "Redirect to default parameter values when parameters are missing in"
This reverts commit 43c65500b7.
* load all data after page is loaded
* return no data only when parameters are missing
* data binding no longer required
* show an error on embeds that fail to load
* data binding no longer required
* present full-page error when dealing with unsafe queries
* don't render the execute button for each parameter
* show 'missing parameter value' error
* Don't reload the whole page when parameter value changes.
* Set API key and load config before rendering.
* Add Query#hasParameters method.
* Don't show download controls for parameterized queries (they won't work).
* Use getUrl to construct a correct query link.
* WIP: have a single way to load results
1. This preloads the query before rendering the page, so we can benefit from using default parameters & make the logic in component simpler.
2. Use a single way to load results, to make sure we do polling when try to load the query results for the first time.
* Show persistent errors and finish loading logic.
* Check if query is safe and show message otherwise.
* Fix test for unsafe parameters embed.
* wait for query results to return before taking snapshot
## What type of PR is this? (check all applicable)
<!-- Please leave only what's applicable -->
- [x] Refactor
- [x] Bug Fix
## Description
This basically makes sure that when import the redash package we don't accidentally trigger import-time side-effects such as requiring Redis.
Refs #3569 and #3466.
## What type of PR is this? (check all applicable)
- [x] Bug Fix
## Description
Without this change the Help Drawer couldn't load content anymore.
## Related Tickets & Documents
#3404
* Add rate limits for user resources.
* Disable rate limiting in tests (except for tests that need it).
* Update strings to unicode to avoid SQLA warnings
* extract time limit decisions to a dynamic settings function
* introduce environment variable for scheduled query time limits
* pass in org_id to query_time_limit
* add an interaction test that verifies that time limits are applied to
jobs
* really important newlines according to CodeClimate
* Don't force an order by created date - any usecases that want
this already request it explicitly and it breaks the search
function that wants to order by best match.
* Fix the logic so that we fall back to a default search order when there's
no search term, rather than when there is one and we should use the
best-match ordering.
* Remove accidentially added blank line
* support date ranges for parameterized embeds
* add qs
* remove hideous implementation and use qs
* get rid of apiKey querystring parameter and introduce query-string
module
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* use the textless endpoint (/api/queries/:id/results) for pristine
queriest
* Revert "use the textless endpoint (/api/queries/:id/results) for pristine"
This reverts commit cd2cee7738.
* go to textless /api/queries/:id/results by default
* change `run_query`'s signature to accept a ParameterizedQuery instead of
constructing it inside
* raise HTTP 400 when receiving invalid parameter values. Fixes#3394
* support querystring params
* extract coercing of numbers to function, along with a friendlier
implementation
* wire embeds to textless endpoint
* allow users with view_only permissions to execute queries on the
textless endpoint, as it only allows safe queries to run
* enqueue jobs for ApiUsers
* add parameters component for embeds
* include existing parameters in embed code
* fetch correct values for json requests
* remove previous embed parameter code
* rename `id` to `user_id`
* support executing queries using Query api_keys by instantiating an ApiUser that would be able to execute the specific query
* bring back ALLOW_PARAMETERS_IN_EMBEDS (with link on deprecation coming up)
* show deprecation messages for ALLOW_PARAMETERS_IN_EMBEDS. Also, move
other message (email not verified) to use the same mechanism
* add link to forum message on setting deprecation
* rephrase deprecation message
* add link to forum message regarding embed deprecation
* change API to /api/queries/:id/dropdowns/:dropdown_id
* split to 2 different dropdown endpoints and implement the second
* add test cases for /api/queries/:id/dropdowns/:id
* use new /dropdowns endpoint in frontend
* first e2e test for sharing embeds
* Pleasing the CodeClimate overlords
* All glory to CodeClimate
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* split has_access between normal users and ApiKey users
* remove residues from bad rebase
* allow access to safe queries via api keys
* rename `object` to `obj`
* support both objects and group dicts in `has_access` and `require_access`
* simplify permission tests once `has_access` accepts groups
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* rename `object` to `obj`
* support both objects and group dicts in `has_access` and `require_access`
* simplify permission tests once `has_access` accepts groups
* fix bad rebase
* send embed parameters through POST data
* no need to log `is_api_key`
* move query fetching by api_key to within the Query model
* fetch user by adding a get_by_id function on the User model
* pass parameters as POST data (fixes test failure introduced by switching
from query string parameters to POST data)
* test the right thing - queries with safe parameters should be embeddable
* introduce cy.clickThrough
* add another Cypress test to make sure unsafe queries cannot be embedded
* serialize Parameters into query string
* set is_api_key as the last parameter to (hopefully) avoid
backward-dependency problems
* Update redash/models/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* attempt to fix empty percy snapshots
* snap percies after DOM is fully loaded
* handle an edge case where dropdown queries are connected to data sources
that no longer exist
* Rethinking it, an empty result set makes no sense and it's better to
throw an error
* remove redundant import
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* rename `object` to `obj`
* support both objects and group dicts in `has_access` and `require_access`
* simplify permission tests once `has_access` accepts groups
* Migrate TypePicker to React
* Migrate DataSources and Destinations List
* Fixes to DestinationsList
* Add CreateDataSource (testing with Steps)
* Render the form after type selection
* Add HELP_LINKS to CreateDataSource
* Add Done behavior
* Add scrollToTop to CreateDataSource
* TypePicker styling adjusts
* Add CreateDestination
* Update resouce gets to componentDidMount
* Create EditForm components
* Migrate Edit pages
* Remove angular logic from DynamicForm
* Add actions to EditPages
* TypePicker title style adjustments
* Add Empty and Loading state
* UX improvements
* Review changes
* Styling updates on TypePicker, forms background fix
* Add blank line removed by mistaken
* Reorganize TypePicker
* Hide Search on List Pages
* Fix spacing in Forms
* Update Create Data Source and Destination to be a Dialog
* Remove max-height from the form
* Fix DynamicForm import in CreateUserDialog
* Route /new to open CreateSourceDialog
* Add HelpTrigger + refine styling and Edit Pages
* Remove help links from data source resource
* Update Cypress specs
* TypePicker -> CardsList
* Remove old TypePicker styling and change CardsList styling to less
* Test if Percy shows Dialogs
* Personal review cleanup
* CR
* Remove unnecessary query on dialog success
* Handle resource errors in Edit Pages
* Add CreateDestination policy
* Add placeholder and separator to the Name field
* Use cy.click instead of cy.wait
* Revert "Use cy.click instead of cy.wait" (Didn't work)
This reverts commit 77285d9fa3.
* Align help trigger on the right and rename Steps
* Refine behavior for long names
* Update toastr calls to use notification instead
* Redirect to target after creation
* Remove autoFocus on DynamicForm for Edit Pages
* Add eslint-disable for cy.wait
* Fix Celery worker CLI parameter name that was changed in Celery 4.x.
* Set Celery worker --max-memory-per-child to 1/4th of total system memory.
* Review fixes.
* Review fixes.
* check that e-mail server is configured before marking the email address
as not verified and sending out a verification e-mail
* use helper method in `invite_user`
* move email_server_configured helper to settings
* add test to verify that email addresses arent marked as unverified if
there's no e-mail server to verify them
* simplify a couple of tests with patch
* combine conditions into single variable
* Booleans, gotta love 'em
* Browser support config
* Removed some offending code
* Added unsupported html page and redirect for IE
* Typo in regex
* Made html page static
* Added redirect script to multi_org
* Moved static html page to client/app
* Normalize Flask initialization API use.
* Use Flask-Talisman.
* Enable HSTS when HTTPS is enforced.
* More details about how CSP is formatted and write CSP directives as a string.
* Use CSP frame-ancestors directive and not X-Frame-Options for embedable endpoints.
* Add link to flask-talisman docs.
* set remember_token cookie to be HTTP-Only and Secure
* Reorganize secret key configuration to be forward thinking and backward compatible.
* Fix a few more inconsistencies when loading and dumping JSON.
Refs #2807. Original work in: #2817
These change have been added since c2429e92d2.
* Review fixes.
* change API to /api/queries/:id/dropdowns/:dropdown_id
* extract property
* split to 2 different dropdown endpoints and implement the second
* make access control optional for dropdowns (assuming it is verified at a
different level)
* add test cases for /api/queries/:id/dropdowns/:id
* use new /dropdowns endpoint in frontend
* require access to dropdown queries when creating or updating parent
queries
* rename Query resource dropdown endpoints
* check access to dropdown query associations in one fugly query
* move ParameterizedQuery to models folder
* add dropdown association tests to query creation
* move group by query ids query into models.Query
* use bound parameters for groups query
* format groups query
* use new associatedDropdowns endpoint in dashboards
* pass down parameter and let it return dropdown options. Go Levko!
* change API to /api/queries/:id/dropdowns/:dropdown_id
* split to 2 different dropdown endpoints and implement the second
* use new /dropdowns endpoint in frontend
* pass down parameter and let it return dropdown options. Go Levko!
* fix bad rebase
* add comment to clarify the purpose of checking the queryId
* Process extra column metadata for a few sql-based data sources.
* Add Table and Column metadata tables.
* Periodically update table and column schema tables in a celery task.
* Fetching schema returns data from table and column metadata tables.
* Add tests for backend changes.
* Front-end shows extra table metadata and uses new schema response.
* Delete datasource schema data when deleting a data source.
* Process and store data source schema when a data source is first created or after a migration.
* Tables should have a unique name per datasource.
* Addressing review comments.
* Update migration file for mixins.
* Appease PEP8
* Upgrade migration file for rebase.
* Cascade delete.
* Adding org_id
* Remove redundant column and table prefixes.
* Non-existing tables and columns should be filtered out on the server side not client side.
* Fetching table samples should be optional and should happen in a separate task per table.
* Allow users to force a schema refresh.
* Use updated_at to help prune old schema metadata periodically.
* Using settings.SCHEMAS_REFRESH_QUEUE
* convert all dropdown values to strings to support parameter lookup.
solves #3562
* unicode all the way down
* show correct default values in QueryBasedParameterInput by converting
them to strings
* Remove QueryTaskTracker
* Remove scheudling of cleanup_tasks
* Add Celery introspection tools
* First iteration of updating the admin API.
* Show more details
* Add option to skip building npm in Dockerfile
* Show started_at
* update the refresh schedule, as it's too fast
* Update Celery monitor to report on all active tasks.
* Update task parsing for new format
* WIP: improved celery status screen
* Fix property name.
* Update counters
* Update tab name
* Update counters names
* Move component to its own file and fix lint issues
* Add migratin to remove Redis keys
* Improve columns layout
* Remove skip_npm_build arg as it's not used anymore.
* Convert query from SQL to Python
* Simplify column definition.
* Show alert on error.
## What type of PR is this? (check all applicable)
- [x] Refactor
- [x] Bug Fix
## Description
Move favorites list handlers to their relevant modules (`redash.handlers.queries` and `redash.handlers.dashboards`) and applied `order_results` to dashboards.
* Add support for Google Hangouts Chat as alert destination
* Remove redundant imports
* Remove code used for debugging
* Fix pep8 warnings
* Update redash/destinations/hangoutschat.py
Add friendly name by separating type and description
Co-Authored-By: pieter-venter <pieterventer@geotab.com>
* Fix pep8 warnings. Rename image to match desitnation type.
* Show message for unknown alert state in default color
* Create React version for the EmailSettingsWarning
* Migrate the Create User Page
* Migrate UserProfile to React
* Add /users/me to the routes (Percy ftw)
* Fix UserShow test spec
* Remove Error Messages component
* Show invitation link if email server not setup (#3519)
* return invite link to client if e-mail server is not set up
* add a couple of tests to make sure invite links are only returned when neccessary
* show invite link when e-mail is not configured
* remove "an e-mail has been sent" when there's no e-mail configured
* return invite_url in re-invites as well. Also refactor to reuse the code.
* Use CreateUserDialog instead of Page
* Render invite link on Resend Invitation click
* Add email validation to DynamicForm
* Fix EmailWarning position + update user list with user creation success
* Fix console error on UserProfile
* Redirect from /users/new + rename createUser -> showCreateUserDialog
* Use alert instead of toastr for user creation errors
* Remove logic from CreateUserDialog
* CR
* Use Promise.reject instead of throw to avoid console error
* append slug to /verification_email endpoint when in MULTI_ORG mode
* Revert "append slug to /verification_email endpoint when in MULTI_ORG mode"
This reverts commit 817fb034c4.
* fix for /email_verification in MULTI_ORG setups
* Make sure master builds tarball/docker image only when backend and frontend tests pass.
* Build a redash/preview image alongside redash/redash image.
* Fix version variable
* Updated npm to support react hooks
* Added <QuerySelector />
* Changed selectQuery to also clear, completed 2->3 dots in msg, avoiding setSearching on stale rejection.
* Removed unused highlight lib
* Add two new envars. REDASH_LDAP_USE_SSL which determines if the connection will use ssl and LDAP_AUTH_BIND which determines if the binding is SIMPLE or ANONYMOUS
* Add use_ssl paremeter
* Rename LDAP_AUTH_BIND to LDAP_AUTH_METHOD and modify LDAP_SSL using parse_boolean
* Fix typo
* import ANONYMOUS constant from ldap3
* Add NTLM authentication
* Add comment to authentication method envar
* adding gevent worker in requirements.txt, adding some gunicorn configurable parameters with defaults
* reverting the change as it's going to be set via env variable(s)
* use the textless endpoint (/api/queries/:id/results) for pristine
queriest
* reverse conditional. not not is making me the headaches.
* add ParameterizedQuery#is_safe with an inital naive implementation which
treats any query with a text parameter as not safe. This will be
remedied later when DB drivers will handle these parameters.
* allow getting new query results even if user has only view permissions
to the data source (given that the query is safe)
* fix lint error - getDerivedStateFromProps should be placed after state
* Revert "use the textless endpoint (/api/queries/:id/results) for pristine"
This reverts commit cd2cee7738.
* move execution preparation to a different function, which will be soon
reused
* go to textless /api/queries/:id/results by default
* let the query view decide if text or textless endpoint is needed
* allow safe queries to be executed in the UI even if the user has no
permission to execute and create new query results
* change `run_query`'s signature to accept a ParameterizedQuery instead of
constructing it inside
* use dict#get instead of a None guard
* use ParameterizedQuery in queries handler as well
* test that /queries/:id/results allows execution of safe queries even if
user has view_only permissions
* lint
* raise HTTP 400 when receiving invalid parameter values. Fixes#3394
* remove unused methods
* avoid cyclic imports by importing only when needed
* verify that a ParameterizedQuery without any parameters is considered
safe
* introduce query.parameter_schema
* encapsulate ParameterizedQuery creation inside Query
* Fix collection fields retreival bug when Views are present in MongoDB
* fixing _is_collection_a_view function
* Update redash/query_runner/mongodb.py
Co-Authored-By: jodevsa <jodevsa@gmail.com>
* using options() instead of try/except on collstats to detect if a collection is a view
* monitor "schemas" queue to run refresh_schema
`refresh_schema` tasks won't run because "schemas" queue isn't consumed
with default settings.
and it cause leaking redis storage, a "schemas" list is growing with time.
this PR fix it, adds "schemas" queue to monitor by celery.
* use scheduled_worker for "schemas" queue
instead of "adhoc_worker"
* Add check for repository url
* Use CIRCLE_REPOSITORY_URL directly
* Change percy waiting to be with resources instead of time
* Add data sources types resource to Edit Data Source + eslint fixes
* Separate Page Screenshots in different spec files
* Moved component from services->components
* Added help trigger to header
* Reverted tooltip-anchor swap
* Moved trigger location in html (to mirror saas)
* Removed white space
* Changed tooltip text to “Help”
* Update DynamicForm export
* Move UserShow to users folder
* Migrate User profile header and create DynamicForm for basic data
* Update UserShow to use UserProfile prop
* Add API Key input
* Add handler to regenerate API Key button
* Handle user profile save
* Add readOnly prop to DynamicForm and begin disabled user behavior
* Add Change Password Modal
* Remove action buttons for disabled users
* Add send password reset behavior
* Add minLength and password comparison to Password Modal
* Resend Invitation button
* Add Convert User Info
* Fix UserShow test
* Some code updates
* Add enable/disable user button
* Add UserPolicy as an idea
* Remove UserPolicy
* Create Edit Profile spec
* Move User profile screenshot to Edit Profile Spec
* Add tests for saving user and changing password errors
* CC is back :) - Fix trailing spaces
* Add test for succesful password update
* A few improvements from code review
* Remove Toggle User button when seeing your own profile
* Create InputWithCopy
* Fix possible errors when network is off and improve Email not sent alert
* Add default response object for $http possible errors
* Changes in UserEdit
- removed onClick from methods name
- regenerate API Key now uses InputWithCopy
- Password title added
* Update UserEdit render behavior and styling
- Password title changed to h5
- change rendering rules for actions
- Password modal is now closed when password is changed
- change DynamicForm readOnly to the fields and add hideSubmitButton
* Create ChangePasswordDialog and update UserEdit
* Fix possible console error
* Remove password match assertion from spec
* Fix typo
* open port 3000 for remote debugging
* add ptvsd
* use port 5678 to avoid changes in VSCode's default config
* attach to ptvsd
* no need to wait for attach
* actually, --debugger seems to be working
* create a new docker entry point for remote debugging
* alternative method to switch to debugging
* replace raven with sentry-sdk
* use sentry-sdk in celery
* use sentry-sdk with flask
* unify Flask and Celery initializations for Sentry
* extract sentry stuff to own module
* it's time for Sentry 0.7.2
* stop testing `collect_query_parameters`, it's an implementation detail
* add tests for `missing_query_params`
* rename SQLQuery -> ParameterizedSqlQuery
* rename sql_query.py to parameterized_query.py
* split to parameterized queries and parameterized SQL queries, where
parameterized queries only do templating and parameterized SQL queries
add tree validation on top of it
* move missing parameter detection to ParameterizedQuery
* get rid of some old code
* fix tests
* set syntax to `custom`
* revert the max-age-related refactoring
* 👋 tree validations 😢
* BaseQueryRunner is no longer a factory for ParameterizedQuery, for now
* add an endpoint for running a query by its id and (optional) parameters
without having to provide the query text
* adds parameter schema to ParameterizedQuery
* adds parameter schema validation (currently for strings)
* validate number parameters
* validate date parameters
* validate parameters on POST /api/queries/<id>/results
* validate enum parameters
* validate date range parameters
* validate query-based dropdowns by preprocessing them at the handler
level and converting them to a populated enum
* change _is_date_range to be a tad more succinct
* a single assignment with a `map` is sufficiently explanatory
* Update redash/utils/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/utils/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/utils/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/utils/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/handlers/query_results.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/utils/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* build error message inside the error
* support all types of numbers as number parameters
* check for permissions when populating query-based dropdowns
* check for access to query before running it
* check for empty rows when populating query-based enums
* don't bother loading query results if user doesn't have access
* 💥 on unexpected parameter types
* parameter schema default is a list, not a dictionary
* fix a totally unrelated typo
* remove redundant null guards
* introduce /dropdown.json endpoint with dummy data
* wire frontend to /dropdown.json
* always return name/value combos from /dropdown.json
* load actual data into /dropdown.json
* pluck correct values for `name` and `value`
* reuse dropdwon plucking logic in QueryResultResource
* simplify _get_dropdown_values
* when doing parameter validation, we only care about the value and not
the display name
* rename dropdown to dropdownOptions
* move dropdown_values to utils/parameterized_query.py
* stop converting queries to enums and encapsulate the work inside
ParameterizedQuery (almost - /dropdown.json would still access the
dropdown_values method)
* re-order arguments by importance
* test query parameter validation
* tests for dropdown_values logic
* remove `.json` suffix to the dropdown endpoint
* allow `BaseResource` to handle JSON stuff
* move _pluck_name_and_value outside its containing method
* case-insensitive lookup when plucking name and value
* separate concerns and simplify test isolation for `dropdown_values`
* pick the default column according to the order specified in the query
result columns attribute
* use `current_org` instead of passing `org`
* test that user has access to the query when calling the /dropdown
endpoint
* Fix: cohorts get stuck when passing strings instead of numbers.
Parse the value to get a number.
* Use parseInt for stage
* Remove redundant parseInt
* Refine existing implementation of dashboards/queries/users lists and a common base controller
* Migrate common list page controller to React and refactor it's logic
* Migrate Dashboard list page to React
* Migrate Queries list page to React
* Migrate Users list page to React
* Remove react-timeago dependency
* Use composition instead of inheritance
* Refine implementation
* Merge sidebar into single component
* Refine column definitions
* Use simple controller instead of React context
* Refine implementation
* Restore changes from getredash/redash#2888
* Tweak Users list page
* Ability to render dynamically defined components
* Tweak users list page
* User list page for non-admins
* Fix: ItemsTable ignores isAvailable field
* Refine implementation
* Refine implementation
* Implement LiveItemsList as higher order component
* Some fixes
* Move some definitions to a better place
* Some fixes
* Refine components
* Refine UsersList page
* More comments for a god of comments
* Fix wrong tables size on smaller screens
* Tweak tables
* [Widget Params] Split title and mapping editing
* [Widget Params] Restyled source editing
* [Widget Params] Switched parameter list to table style
* Displaying different labels and help phrases when changing type
Added link to knowledge base
Fixed issue with existing param default select value
* use the textless endpoint (/api/queries/:id/results) for pristine
queriest
* Revert "use the textless endpoint (/api/queries/:id/results) for pristine"
This reverts commit cd2cee7738.
* move execution preparation to a different function, which will be soon
reused
* go to textless /api/queries/:id/results by default
* let the query view decide if text or textless endpoint is needed
* lint
* stop testing `collect_query_parameters`, it's an implementation detail
* add tests for `missing_query_params`
* rename SQLQuery -> ParameterizedSqlQuery
* rename sql_query.py to parameterized_query.py
* split to parameterized queries and parameterized SQL queries, where
parameterized queries only do templating and parameterized SQL queries
add tree validation on top of it
* move missing parameter detection to ParameterizedQuery
* get rid of some old code
* fix tests
* set syntax to `custom`
* revert the max-age-related refactoring
* 👋 tree validations 😢
* BaseQueryRunner is no longer a factory for ParameterizedQuery, for now
* add an endpoint for running a query by its id and (optional) parameters
without having to provide the query text
* adds parameter schema to ParameterizedQuery
* adds parameter schema validation (currently for strings)
* validate number parameters
* validate date parameters
* validate parameters on POST /api/queries/<id>/results
* validate enum parameters
* validate date range parameters
* validate query-based dropdowns by preprocessing them at the handler
level and converting them to a populated enum
* change _is_date_range to be a tad more succinct
* a single assignment with a `map` is sufficiently explanatory
* Update redash/utils/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/utils/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/utils/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/utils/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/handlers/query_results.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/utils/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* build error message inside the error
* support all types of numbers as number parameters
* check for permissions when populating query-based dropdowns
* check for access to query before running it
* check for empty rows when populating query-based enums
* don't bother loading query results if user doesn't have access
* 💥 on unexpected parameter types
* parameter schema default is a list, not a dictionary
* remove redundant null guards
* Refine Auth service: remove dead code and fix race condition
* Export services in CommonJS style
* Refine Users, Events and OfflineListener services
* Refactor Notifications service - rewrite to CommonJS
* Replace Angular service injection with imports in React components
* Fix Footer tests
* Events service -> recordEvent function
* CR1
* adding uptycs query_runner in redash
* as per comment from Arik comment fixed the code
* fixed function_name
* fixed some indentation issues
* fixed the indentation issue and taken out customer_id from secret
* fixed the dependency of urllib3
* fixed the indententaton issue
* remved the urllib3 from requirements
* fixed the indentation issues
* added the new square image for Uptycs. Removed unnecessary variable and made ssl as an option
* fixed indentation issue
* Renamed SSL to verify_ssl and also added verify_ssl validate in verify in missing places
* add an endpoint for running a query by its id and (optional) parameters
without having to provide the query text
* check for access to query before running it
* allow execution of selected subquery
* fix query save while highlighted
* don't modify queryText and update UI when running selected
* code style and transition
* Fix query selection execution background color
* make naming consistent
* Create ant variables and update form colors
* Remove less extension from imports in ant.less
* Update font-weight for labels
* Add percy snapshot for create data source page
* Remove bold in labels only for checkboxes and radio buttons
* Use null as the default scheduled value.
* Don't serialize None to json, so we can use SQL is not null predicate.
* Fix warning about unicode in tests
* Handling empty query.schedule in UI (#3283)
* Add migration to convert empty schedules to null and drop the not null contraint.
* explicitly look for a False under details['is_invitation_pending'] and
not any falsey result, to avoid locking out invitations which were
created before the Pending Invitation feature was introduced. Solves https://github.com/getredash/redash/issues/3297
* test that old invites (that do not have any is_invitation_pending flag set in their details object) are still acceptable
* create DynamicForm React component
* Render fields based on target in DynamicForm
* Add missing title property to fields
* Fix style properties in DynamicForm
* Render File fields in DynamicForm
* Use React for middle component instead of Angular
* Functional save button
* Update label style
* Render functional actions
* Handle file inputs
* Update render methods to fix code climate issues
* Fix ant input number showing duplicate arrows
* Update DynamicForm style to be vertical
* Separate imports from antd in DynamicForm
* Add Feedback Icons to DynamicForm
* Change Action props on DynamicForm
- use type and pullRight instead of class prop
- update data sources and destinations pages accordingly
* Remove setDefaults method from DynamicForm fields
* Update antd version
* Remove unnecessary class selectors
* Remove another unnecessary class selector
* Separate Ant less dependency tree
* Change order between variables and ant imports
* Remove inc/variables from ant.less
* Update input-height-base for antd
* Set same input-color for Ant and Bootstrap
* add an bit
* prompt on homepage when user's email hasn't been verified
* set e-mail as verified for new setups and invited users
* 👋 copy & paste invite links, it's time for verified e-mails!
* default `is_invitation_pending` to false and actively set it to true
when inviting users, so that existing users won't show "Invitation
Pending"
* fix tests that broke due to default is_invitation_pending value
* treat admin's e-mail address as verified
* add verification endpoint
* send verification e-mail
* Update client/app/components/empty-state/empty-state.html
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/authentication/account.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/handlers/authentication.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/templates/emails/verify.html
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/authentication/account.py
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/templates/verify.html
Co-Authored-By: rauchy <omer@rauchy.net>
* Update redash/templates/emails/verify.txt
Co-Authored-By: rauchy <omer@rauchy.net>
* add link in case redirects are disabled
* POSTing to /email_verification makes more sense than GETting /send_verification
* avoid sending invitations when no_invite is passed along
* Update client/app/pages/users/new.html
Co-Authored-By: rauchy <omer@rauchy.net>
* move e-mail verification prompt to home-page
* get rid of redundant $scope
* return JSON
* flip is_email_verified's default value so that existing users do not
show as not-verified
* e-mail verification propmt isn't dangerous, it just wants to warn you
* Fix: Refresh schedule interval count doesn't adhere to permission rules
* Fix “4.28 weeks” to “30 days”
* Merge interval type and count into one <Select>
* Added support for Apache Drill datasource
* Improvements in `Drill` query runner and minor refactoring
1. Drill query runner now inherits from `BaseHTTPQueryRunner`, because they both have a lot of common code.
2. `BaseHTTPQueryRunner.get_response` method now accepts `http_method` argument (original implementation was only capable of sending `GET` HTTP requests).
3. Added `order` to `BaseHTTPRequestRunner` configuration schema to fix order of UI elements based on the schema.
4. Eliminated duplicate method `_guess_type` in `GoogleSpreadsheet`, `Results` and `Drill` query runners, moved `guess_type` to `redash.query_runner`.
5. Removed tests for `_guess_type` in `GoogleSpreadsheet`, `Results` and `Drill` query runners, merged them into single test case and moved to `tests.query_runner.test_utils`.
6. Various minor changes (code style, imports, etc).
* default `is_invitation_pending` to false and actively set it to true
when inviting users, so that existing users won't show "Invitation
Pending"
* fix tests that broke due to default is_invitation_pending value
* update Flask-OAuthLib
* determine invitation_pending according to empty password. This commit will be reverted, I'm just deferring the implementation
* show '(Invitation Pending)' to users who haven't accepted their invitation yet
* allow resending invitations
* allow deletion of pending users from user list
* set invitation as not pending when following invite link
* prevent deleting activated users
* test that users who follow invitation links are set as non-pending invitations
* prevent re-using invitations
* invitees who use SSO will now also be marked as "non-pending"
* lint
* add last_active_at to users page
* Use our JSON encoder as the SQLAlchemy JSON serializer.
* Fixed some inconsistencies in the user query class methods.
* Minor cosmetic fixes.
* Add some make tasks for easier development.
* Add user detail sync system based on Redis backend.
There is a periodic Celery task that updates a new “details” JSONB column in the “user” table with the data from Redis.
Currently this is only used for tracking the date of last activity of a user but can be extended with other user information later.
Updates a few dependencies.
* Normalize a few Flask extension API names.
* Reduce implementation complexity of JSONEncoder.
* Use request_started signal to make sure we have a request context.
Otherwise loading the user based on the request won’t work.
* Fix test that checks if disabled users can login.
This correctly uses a URL path that includes the current organization and checks for the error message.
The previous test seems to have been a red herring.
* Minor cosmetic fixes.
* Remove needs_sync in favor of just deleting things.
* Misc review fixes.
* Ignore line length.
* Split redash.models import several modules.
* Move walrus UTC DateTimeField into redash.models.types.
* Restore distinctly loading dashboards.
* Simplify default values for user details.
* Define __repr__ methods generically.
* Consistently have underscore methods at the top of model methods.
* Fix tests.
* Split redash.models import several modules.
* Update to latest walrus and redis-py.
* Update kombu to 4.2.2 for redis-py 3.x compatibility.
* Remove redis-cli container after running Make task.
* Move buffer condition after datetime/time conditions.
* Update walrus to 0.7.1.
* Refactor some query APIs.
This uses the flask-sqlalchemy helpers consistently and makes more use of mixins.
* Post rebase fixes.
* Use correct kombu version
* Fix migration down revision
By default Celery will use a file celerybeat-schedule in the current directory.
This is an issue in a Kubernetes/Openshift environment as the file may be lost or even impossible to write.
* Update Cypress element selectors
* Add seed data function to Cypress
* Change Cypress setup to be part of db-seed
* Add DatabaseSource selector to Create Data Source spec
* Add getElement command to Cypress
* Fix eslint issues
* Change Cypress getElement to getByTestId
* Add Percy and test it with the CI
* Change Percy dependency for CI to Cypress' Dockerfile
* Change Percy's execution to the docker container
- add --no-save to avoid errors on Dockerfile.cypress
- pass PERCY_TOKEN from the CI to docker container
* Fix missed char on CircleCI config file
* Move Percy execution back to host on the CI
* Test adding PERCY_TOKEN to frontend-e2e-tests on CI config
* Undo add PERCY_TOKEN to config.yml
* Add Percy token and .git folder to Cypress
* Remove Percy install from config.yml
* Ignore .git folder again and use Percy env vars instead
* Update PERCY_PULL_REQUEST to be CIRCLE_PR_NUMBER
* Update cypress-server.js to handle other cypress commands
- cypress-server.js -> cypress.js
- new commands added to cypress.js
- CircleCI config updated accordingly
- added a Homepage screenshot
* Remove trailing spaces
* Add Create Query spec
* Disable Cypress videos
* Update run browser to Chrome
* Add missing --browser chrome
* display correct error message when attempting to disable yourself
* 403 (Forbidden) feels like a better status code than 400 (Bad Request)
* fix broken test
* remove redundant error title
* take the first 8 characters for frontend version, not backend version
* run `npm run build` after version has been updated in CI
* `pack` should run last
* add SQLQuery class with tests for safe queries and non-safe tautology attacks
* add test for union query injections
* split .apply calls to newline
* add tests for comment attacks
* remove double underscore
* extract complex children check to variable
* inherit from object because I'm not a lamer
Co-Authored-By: rauchy <omer@rauchy.net>
* simplify cognitive complexity
* check that additional columns are not injected
* detect appended queries
* inline .apply calls
* move SQLQuery to it's own module
* move SQLQuery tests to their own module
* serialize SQLQuery instances
* raise an exception when attempting to serialize an unsafe query
* queries without parameters are safe
* remove redundant parentheses
* use cached properties
* rename SQLInjectionException to SQLInjectionError
* support multiple word params and param negations
* refactor out methods that don't involve any state
* don't cache text()
* reduce cognitive complexity
* getredash/redash#3015 When editing a dashboard title results in the visualizations being replaced by the loading markers
* CR1
Co-Authored-By: kravets-levko <levko.ne@gmail.com>
* add git-revision-webpack-plugin
* configure git-revision-webpack-plugin
* add commit to footer
* rename version and commit to backendVersion and frontendVersion
* rename version and commit to backendVersion and frontendVersion
* disable lint error due to use of globals
* fix snapshot test
* read frontend version from VERSION file instead latest git revision
* directly require from version.json file instead of going through WebPack's DefinePlugin
* run snapshots
* Hive: fix issues in building options.
* Hive: add missing import.
* Split Hive into two query runners: one for http and a regular one.
* Upgrade pyhive to support http.
* Specific implementation for databricks:
* Different schema loader, because column names are different (or Hive's schema loader is broken).
* Simpler configuration.
* Simplify Databricks setup even more by removing username.
* Fix tag counts for dashboards to be distinct.
This also makes use of the Dashboard.all base query.
Fix#3108.
* Use Query.all_queries as the base query for Query.all_tags.
* Add test case for Dashboard.all_tags.
* Add Rockset query runner
Per REST API documented here: https://docs.rockset.com/rest/
* Update rockset.py
* Add Rockset logo
* Refactor Rockset qury runner:
* More idomatic names for configuration.
* Move API code to separate class to make it easier removing it when we
switch to official library.
* Make Test Connection work.
* apply autopep8 to rockset.py
* keep query text in local state for now (#3107)
This will be unnecessary once the queryText prop isn't managed by Angular.
* Fix: make formatQuery work
* Autocomplete toggle improvements:
* Refactor to its own component.
* Show state in tooltip (enabled/disabled).
* Disable the toggle if autocomplete is not possible (no schema/too many tokens).
* Remove unsued code.
* Custom icons font (currently has only two icons).
Generated with Icomoon. If we extend its use, we should probably automate this and move to its own package.
* Don't disable live autocomplete for data sources without schema.
It can still be useful to autocomplete from local keywords.
* Differentiate between autocomplete toggle states with an icon.
Also added explicit message for the disabled state.
* Remember thes state of autocomplete.
* Only auto register init functions.
* avoid Query's updated_at from changing when it is linked to new query results
* move comment to previous line
* move QueryResult tests to their own module
* add test which verifies that updated_at is not changed on query data
updates
* tests were false positives - they compared HH:MM:SS, but that never
changed because the original time was 1 week behind.
* remove redundant constructor
* remove hack and use a proper event to prevent updated_at from changing
* use self.assertEqual instead of assert
* Update docker-compose.yml
jut updating docker-compose dev to version 3, any special reason to keep redis on 3? and pg on 9.5? I could also add a volume to pg, any reason not to do so?
* rollback to redis 3 and pg 9.5 due to consistency in project defaults
* Configure volume directly in worker service.
The SVV_COLUMNS table used to determine the tables of a
Redshift database includes all tables, even those the
current user cannot access, by default. These tables clutter
the schema browser and make it harder for users to determine
which tables they should have access to.
These changes modify the Redshift query runner so that
tables the configured user cannot access are filtered
out from the database schema. The checks are two-fold:
* First, the query uses HAS_SCHEMA_PRIVILEGE to check if the
current user has USAGE rights on a schema the given table
belongs to. This privilege is required to access any of
the tables in a schema.
* Second, the query tries to determine if the current user
has SELECT access to the given table. Two cases need to
be considered here:
* First, we need to check if the table is part of an
external schema. Access to tables in external schema
is controlled at schema level - you cannot grant or
revoke access to specific external tables. Additionally,
the HAS_TABLE_PRIVILEGE returns an error if it is asked
to give a verdict for an external table.
Hence, the query checks if the schema a specific table
belongs to is an external schema and if it is, the table
is included in the list (if we got here the user already
had USAGE on the given schema). This check short-circuits
the table-level access check for external tables which
means they are never passed to HAS_TABLE_PRIVILEGE().
* Then, if the table was not part of an external schema,
the HAS_TABLE_PRIVILEGE() function is used to determine
if the user has SELECT access to the given table. The
table is included in the schema if this check passes.
Together this condition ensures that tables the user definitely
cannot access are not included in the schema browser.
These changes have been tested to work in an environment that
includes normal and external schemas, normal and late-binding
views, and normal and external tables.
* Revert "remove pytest_watch (#3048)"
This reverts commit 096eba3876.
* Revert "address tag display on query list page (#2803)"
This reverts commit 99115a12e6.
Add page size settings.
| name | default | description |
| :---- | :------ | :---------- |
| `REDASH_PAGE_SIZE` | 20 | How many items are displayed in one page as default. |
| `REDASH_PAGE_SIZE_OPTIONS` | 5,10,20,50,100 | How many steps as page_size. |
This feature has requested at the meetup in Japan.
https://redash-meetup.connpass.com/event/101420/
* Add Jest packages
* Add first test
* Install eslint rules for jest & move deps to dev
* Configure cirlce to run jest
* package.json: Remove dev command
* package.json: clean command
* Don't autoload test files.
* Fix: webpack-dev-server was recompiling all the files on every change
* Update CircleCI step names
* Prevent open redirection attack
* Add redirection url after logging in test
* Sanitize url just before redirecting it
* Consider when next parameter is None
* Add DB2 as a data source using ibm-db python package
* fix some codeclimate issue
* fix long line and missing white space
* Manage case of bad import
* Add DB2 query_runner as default query runner
* Fixed minor PEP8 rules
* Fix search size on smaller tablet size
* Less prominent tag counter
* Fix hiding logo
* Add missing space between icon and button text
* Different embed icon
* Revert embed icon to its original
* Better edit source icon + markup cleanup
* feat: add support for expanding dashboard visualizations
These changes implement support for expanding a dashboard visualization
into a larger modal dialog.
This is useful if you have a dashboard with lots of small widgets and
want to inspect one of the widgets more closely. In the past, this
would've required you to navigate to the query page to see a larger
version of the visualization. With these changes, visualizations can
be expanded right from the dashboard.
The implementation is simple as it just renders the visualization into
a modal dialog. Other parts of the widget (e.g. parameters) are not
included in this dialog.
* chore(widget-dialog): use query-link widget to render title
This reduces code duplication a bit. The link is made read-only
as navigation doesn't close the modal dialog.
* fix: make ui-select dropdown z-index > modal dialog z-index in dashboard page
Otherwise the dropdown renders behind the modal dialog if filter value
is changed from the modal view of a widget.
* Speed up builds by skipping requirements_all_ds.txt
* Update docker compose file version
* Start services before running commands
* Add boto and Athena dependencies to requirements_dev.txt
* Consistently use simplejson to loading and dumping JSON.
This introduces the new functions redash.utils.json_dumps and redash.utils.json_loads and simplifies the custom encoder setup.
UUIDs are now handled by the default encoder, too.
Fixes#2807.
Use string comparison in parse_boolean instead of the (simple)json module.
* Use tag list ordering from backend.
This basically stops resorting the tag list on the client side and use the already existing (and correct) descending sort order bu tag usage count from the backend.
* Show count of tagged items in tag list.
The integration tests are currently not failing and providing false positives when running them. I'm disabling this until the test suite is more stable.
@jrbenny35 @jezdez
This reduces the code duplication between the dashboard, user and queries list pages and normalizes many of the APIs between them.
This also:
- allows sorting query favorites
- adds a pagination size select to the dashboard list
- fixes a bunch of UI inconsistencies between the queries and dashboards list (e.g. margins)
The new ListCtrl class is subclassed in the various specific page controllers and extended as needed. New list pages can make use of the same pattern in the future.
This also adds some missing event recordings from 34e39eda4a.
* Moved configuration to `.circleci/config.yml` as the documentation for V2 suggests.
* Created a dedicated Docker Compose configuration (in `.circleci/docker-compose.yml`) to remove volumes configuration as this is not supported with CircleCI's Docker executer.
* Fix the Docker image build and tarball packing jobs to work and use correct version.
* Remove references to Gitter
* Move integration tests to their own workflow and run only on master branch
* Fix error in integration tests configuration
* Update names of stages
Add error message when non owner tries to add a user to a dashboard
Fixes#2592
REDASH_FEATURE_SHOW_PERMISSIONS_CONTROL has to be true for this to be applicable.
Operation threw a 403 Forbidden error before this change but it is in the console not shown to the user.
The following error occurs when a query is selected in the
alert page:
> TypeError: "item.getQueryResultPromise is not a function"
This is because .onQueryHandler() is given a plain object instead of a
Query when a query is selected from the dropdown.
These changes fix the issue by wrapping the plain object into a Query
object before calling .getQueryResultPromise() on it.
Previously, exceptions from pysaml2 would've caused
a 500 Internal Server Error response to be sent back
to the user. Now, the user is redirected back to login
page which includes an user friendly error message.
MongoReplicaSetClient is deprecated and will be removed in future.
See http://api.mongodb.com/python/current/api/pymongo/mongo_replica_set_client.html
```
MongoReplicaSetClient will be removed in a future version of PyMongo.
Changed in version 3.0: MongoClient is now the one and only client class for a standalone server, mongos, or replica set. It includes the functionality that had been split into MongoReplicaSetClient: it can connect to a replica set, discover all its members, and monitor the set for stepdowns, elections, and reconfigs.
```
* Fix: line chart with category x-axis: when some values missing, wrong hints displayed on hover
* Fix: second Y-axis not displayed when stacking enabled
* Set of dashboard improvements and bug fixes
- set minimal height of widgets to 1 (was 4)
- bug: for some widgets auto-height wasn't calculated properly (sometimess too small, sometimes too large)
- bug: for small widgets, top-right menu was cut to widgets bounds
- bug: with opened top-right menu widgets with auto-height started "dancing"
- bug: at some point auto-height feature was disabling by itself (in fact - it depends on `angular-grindter`s internal processes)
- fix: widget with empty contents had extra 40px of white space (paddings of container)
* Add scrollbars to pivot table widgets
* Fix: 100% CPU loading caused page lags
- cache GeoJSON to avoid multiple HTTP requests;
- allow to edit map bounds;
- optimize update map calls (do not re-render it every time);
- UI/X imporvements.
This is along the lines of what Flask does for CLI plugins:
http://flask.pocoo.org/docs/0.12/cli/#cli-plugins
The API allows specifying Python callbacks that receive the
Redash Flask app as the only argument and allow extending
the Redash process with the usual Flask API as needed. This
does not cover front-end specific extensions (yet).
`restart: always` will cause containers to restart when the machine is rebooted or the docker process restarts. Setting this to `unless-stopped` will mean the container will restart only if the docker process/computer was rebooted without stopping the container first. Otherwise, running `docker-compose stop` and rebooting will cause the docker containers all to start again (and goodbye laptop battery!)
We might also consider `restart: on-failures` which would mean that rebooting would _not_ cause containers to start again unless requested via `docker-compose up`.
The previous implementation of remote header login did not
support the multiorg mode of Re:Dash. These changes modify
the trusted header authentication to expose a per-organization
login endpoint that logs users in the specified organization.
The feature itself is not that interesting as multiorg is pretty
much impossible to use in a standalone Re:Dash installation. What's
more interesting is that all tests are executed in multiorg mode. It's
not possible to write tests for the trusted header authentication if
the method does not support multiorg mode.
To make benefits of these changes more concrete, some tests were
written to test the basic functionality of trusted header
authentication.
The entity_id and nameid_format form fields didn't work as they
tried to save a setting with an incorrect name. This has now
been fixed and the form can be used to configure these values.
A short debounce delay was also added to the form to avoid
saving the value after every keystroke (if values are inputted
manually).
This makes it possible to configure Redash to require TLS
and verify the certificate Redshift presents.
The setting works similarly to that of the PostgreSQL source
except that Redshift explicitly passes 'prefer' to psycopg2
if sslmode is not configured (for compatibility).
- Use $onChanges instead of $watch (fixes an issue where the query
results was constantly reloading).
- Choose the first value when first loading the options.
While RFC 5321 alows for the username/mailbox art of the address to be
case-sensitive, it is hightly discouraged.
The local-part of a mailbox MUST BE treated as case
sensitive. Therefore, SMTP implementations MUST take care to
preserve the case of mailbox local-parts. In particular, for
some hosts, the user "smith" is different from the user
"Smith". However, exploiting the case sensitivity of mailbox
local-parts impedes interoperability and is discouraged.
Mailbox domains follow normal DNS rules and are hence not
case sensitive.
-- https://tools.ietf.org/rfc/rfc5321.txt
Signed-off-by: Mike Fiedler <miketheman@gmail.com>
By setting TEST_ARGS environment variable with -e option of docker-compose
we can set nosetests options to run the tests as we want, like so;
`docker-compose run --rm -e TEST_ARGS="--with-coverage tests/handlers/test_dashboards.py" server tests`
The issue is, if you try to revoke the permission of a user from an
object, all the permissions on this object get removed. The fix is
assigning filtered query object to it's own reference.
According to SQLAlchemy documentation, `filter` method applies to
the **copy** of the query object which means calling filter doesn't
affect the object receiving filter call. For more information;
http://docs.sqlalchemy.org/en/latest/orm/query.html#sqlalchemy.orm.query.Query.filter
Before this change, pressing enter in Firefox 55 would insert a line
break into the description field and then save it.
This change prevents the line break from being inserted before saving.
There's no change to Chrome's behaviour from this change.
The multi filter option is useful but lacking in an easy easy way to
select all values. I have added in a psudo option '*' that when selected
automatically fills out the mutli-select with all possible filters.
I have also added in a second psudo option '-' which becomes available
_if_ the multi-filter has all possible values selected.
This makes it easy to clear the multi-filter.
* Feature: Query based parameter (drop-down)
* Restrict to string column for query parameter
* Fix lint errors
* Fix html in paramters.html
* Addressed comments from @arikfr
Combines mozilla/redash PR’s 86 and 95.
There was a bug that saved textbox content on a dashboard when you
tried to close without saving. This fixes it.
When we have a long0running query, it is useful to look up what that
query definition is. Having a link directly from the admin page cuts
down on the manual step of opying the query id and pasting that into the
address bar.
Signed-off-by: Mike Fiedler <miketheman@gmail.com>
by default when using python's `logging` module and a `StreamHandler`,
the stream is directed to `sys.stderr`
(https://docs.python.org/2/library/logging.handlers.html)
By setting the `REDASH_LOG_STDOUT` environment variable to `true` we
enable the option to stream the logs to `sys.stdout`. Setting this
configuration option to `false` (or leaving it as default) will
initialize the `StreamHandler` to `sys.stderr` - meaning the original
behavior remains unchanged.
Allow $REDASH_WEB_WORKERS to be set in the environment to change the
number of Gunicorn workers that are started (currently hardcoded to
four). If not set, the default is four, so this will not affect
existing users at all.
Documentated by example in docker-compose example manifests.
When the extra app_context was popped from the stack, it was
triggering flask-sqlalchemy's teardown handler, which was removing
the session causing objects to become detached before they should
be.
In some cases showing recent queries/dashboards based on events becomes
too "expensive" in terms of database resources. This is a fallback option
to show recently updated queries/dashboards instead.
It's possible the Celery metadata object was expired, but the task
is still running (which will result in PENDING status when querying
the AsyncResult object).
Do you need help or have a question? Checkout the Support category in our discussion forum: https://discuss.redash.io/c/support.
Got an idea for a new feature? Check if it isn't on the roadmap already: http://bit.ly/redash-roadmap and start a new discussion in the features category: https://discuss.redash.io/c/feature-requests 🌟.
Found a bug? Please fill out the sections below... thank you 👍
### Issue Summary
A summary of the issue and the browser/OS environment in which it occurs.
### Steps to Reproduce
1. This is the first step
2. This is the second step, etc.
Any other info e.g. Why do you consider this to be a bug? What did you expect to happen instead?
about: Report reproducible software issues so we can improve
---
<!--
We use GitHub only for bug reports 🐛
Anything else should be a discussion: https://github.com/getredash/redash/discussions/ 👫
🚨For support, help & questions use https://github.com/getredash/redash/discussions/categories/q-a
💡For feature requests & ideas use https://github.com/getredash/redash/discussions/categories/ideas
**Found a security vulnerability?** Please email security@redash.io to report any security vulnerabilities. We will acknowledge receipt of your vulnerability and strive to send you regular updates about our progress. If you're curious about the status of your disclosure please feel free to email us again. If you want to encrypt your disclosure email, you can use this PGP key.
-->
### Issue Summary
A summary of the issue and the browser/OS environment in which it occurs.
### Steps to Reproduce
1. This is the first step
2. This is the second step, etc.
Any other info e.g. Why do you consider this to be a bug? What did you expect to happen instead?
@@ -4,21 +4,7 @@ Thank you for taking the time to contribute! :tada::+1:
The following is a set of guidelines for contributing to Redash. These are guidelines, not rules, please use your best judgement and feel free to propose changes to this document in a pull request.
@@ -41,37 +34,68 @@ The following is a set of guidelines for contributing to Redash. These are guide
When creating a new bug report, please make sure to:
- Search for existing issues first. If you find a previous report of your issue, please update the existing issue with additional information instead of creating a new one.
- If you are not sure if your issue is really a bug or just some configuration/setup problem, please start a discussion in [the support forum](https://discuss.redash.io/c/support) first. Unless you can provide clear steps to reproduce, it's probably better to start with a thread in the forum and later to open an issue.
- If you are not sure if your issue is really a bug or just some configuration/setup problem, please start a [Q&A discussion](https://github.com/getredash/redash/discussions/new?category=q-a) first. Unless you can provide clear steps to reproduce, it's probably better to start with a discussion and later to open an issue.
- If you still decide to open an issue, please review the template and guidelines and include as much details as possible.
### Suggesting Enhancements / Feature Requests
If you would like to suggest an enchancement or ask for a new feature:
If you would like to suggest an enhancement or ask for a new feature:
- Please check [the roadmap](https://trello.com/b/b2LUHU7A/re-dash-roadmap) for existing Trello card for what you want to suggest/ask. If there is, feel free to upvote it to signal interest or add your comments.
- If there is no existing card, open a thread in [the forum](https://discuss.redash.io/c/feature-requests) to start a discussion about what you want to suggest. Try to provide as much details and context as possible and include information about *the problem you want to solve* rather only *your proposed solution*.
- Please check [the Ideas discussions](https://github.com/getredash/redash/discussions/categories/ideas) for existing threads about what you want to suggest/ask. If there is, feel free to upvote it to signal interest or add your comments.
- If there is no open thread, you're welcome to start one to have a discussion about what you want to suggest. Try to provide as much details and context as possible and include information about *the problem you want to solve* rather only *your proposed solution*.
### Pull Requests
-**Code contributions are welcomed**. For big changes or significant features, it's usually better to reach out first and discuss what you want to implement and how (we recommend reading: [Pull Request First](https://medium.com/practical-blend/pull-request-first-f6bb667a9b6#.ozlqxvj36)). This to make sure that what you want to implement is aligned with our goals for the project and that no one else is already working on it.
- Include screenshots and animated GIFs in your pull request whenever possible.
**Code contributions are welcomed**. For big changes or significant features, it's usually better to reach out first and discuss what you want to implement and how (we recommend reading: [Pull Request First](https://medium.com/practical-blend/pull-request-first-f6bb667a9b6#.ozlqxvj36)). This is to make sure that what you want to implement is aligned with our goals for the project and that no one else is already working on it.
#### Criteria for Review / Merging
When you open your pull request, please follow this repository’s PR template carefully:
- Indicate the type of change
- If you implement multiple unrelated features, bug fixes, or refactors please split them into individual pull requests.
- Describe the change
- If fixing a bug, please describe the bug or link to an existing github issue / forum discussion
- Include UI screenshots / GIFs whenever possible
- Please add [documentation](#documentation) for new features or changes in functionality along with the code.
- Please follow existing code style. We use PEP8 for Python and sensible style for Javascript.
- Please follow existing code style:
- Python: we use [Black](https://github.com/psf/black) to auto format the code.
- Javascript: we use [Prettier](https://github.com/prettier/prettier) to auto-format the code.
#### Initial Review (1 week)
During this phase, a team member will apply the “Team Review” label if a pull request meets our criteria or a “Needs More Information” label if not. If more information is required, the team member will comment which criteria have not been met.
If your pull request receives the “Needs More Information” label, please make the requested changes and then remove the label. This resets the 1 week timer for an initial review.
Stale pull requests that remain untouched in “Needs More Information” for more than 4 weeks will be closed.
If a team member closes your pull request, you may reopen it after you have made the changes requested during initial review. After you make these changes, remove the “Needs More Information” label. This again resets the timer for another initial review.
#### Full Review (2 weeks)
After the “Team Review” label is applied, a member of the core team will review the PR within 2 weeks.
Reviews will approve, request changes, or ask questions to discuss areas of uncertainty. After you’ve responded, a member of the team will re-review within one week.
#### Merging (1 week)
After your pull request has been approved, a member of the core team will merge the pull request within a week.
### Documentation
The project's documentation can be found at [https://redash.io/help/](https://redash.io/help/). The [documentation sources](https://github.com/getredash/website/tree/master/user-guide) are hosted on GitHub. To contribute edits / new pages, you can use GitHub's interface. Click the "Edit on GitHub" link on the documentation page to quickly open the edit interface.
The project's documentation can be found at [https://redash.io/help/](https://redash.io/help/). The [documentation sources](https://github.com/getredash/website/tree/master/src/pages/kb) are hosted on GitHub. To contribute edits / new pages, you can use GitHub's interface. Click the "Edit on GitHub" link on the documentation page to quickly open the edit interface.
## Additional Notes
### Release Method
We publish a stable release every ~2 months, although the goal is to get to a stable release every month. You can see the change log on [GitHub releases page](http://github.com/getredash/redash/releases).
We publish a stable release every ~3-4 months, although the goal is to get to a stable release every month.
Every build of the master branch updates the latest *RC release*. These releases are usually stable, but might contain regressions and therefore recommended for "advanced users" only.
Every build of the master branch updates the *redash/redash:preview* Docker Image. These releases are usually stable, but might contain regressions and therefore recommended for "advanced users" only.
When we release a new stable release, we also update the *latest* Docker image tag, the EC2 AMIs and GCE images.
## Code of Conduct
This project adheres to the Contributor Covenant [code of conduct](http://redash.io/community/code_of_conduct). By participating, you are expected to uphold this code. Please report unacceptable behavior to team@redash.io.
This project adheres to the Contributor Covenant [code of conduct](https://redash.io/community/code_of_conduct). By participating, you are expected to uphold this code. Please report unacceptable behavior to team@redash.io.
[](https://gitter.im/getredash/redash?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
**_Redash_** is our take on freeing the data within our company in a way that will better fit our culture and usage patterns.
Redash is designed to enable anyone, regardless of the level of technical sophistication, to harness the power of data big and small. SQL users leverage Redash to explore, query, visualize, and share data fromany data sources. Their work in turn enables anybody in their organization to use the data. Every day, millions of users at thousands of organizations around the world use Redash to develop insights and make data-driven decisions.
Prior to **_Redash_**, we tried to use traditional BI suites and discovered a set of bloated, technically challenged and slow tools/flows. What we were looking for was a more hacker'ish way to look at data, so we built one.
Redash features:
**_Redash_** was built to allow fast and easy access to billions of records, that we process and collect using Amazon Redshift ("petabyte scale data warehouse" that "speaks" PostgreSQL).
Today **_Redash_** has support for querying multiple databases, including: Redshift, Google BigQuery, PostgreSQL, MySQL, Graphite,
Presto, Google Spreadsheets, Cloudera Impala, Hive and custom scripts.
1.**Browser-based**: Everything in your browser, with a shareable URL.
2.**Ease-of-use**: Become immediately productive with data without the need to master complex software.
3.**Query editor**: Quickly compose SQL and NoSQL queries with a schema browser and auto-complete.
4.**Visualization and dashboards**: Create [beautiful visualizations](https://redash.io/help/user-guide/visualizations/visualization-types) with drag and drop, and combine them into a single dashboard.
5.**Sharing**: Collaborate easily by sharing visualizations and their associated queries, enabling peer review of reports and queries.
6.**Schedule refreshes**: Automatically update your charts and dashboards at regular intervals you define.
7.**Alerts**: Define conditions and be alerted instantly when your data changes.
8.**REST API**: Everything that can be done in the UI is also available through REST API.
9.**Broad support for data sources**: Extensible data source API with native support for a long list of common databases and platforms.
**_Redash_** consists of two parts:
1.**Query Editor**: think of [JS Fiddle](http://jsfiddle.net) for SQL queries. It's your way to share data in the organization in an open way, by sharing both the dataset and the query that generated it. This way everyone can peer review not only the resulting dataset but also the process that generated it. Also it's possible to fork it and generate new datasets and reach new insights.
2.**Dashboards/Visualizations**: once you have a dataset, you can create different visualizations out of it, and then combine several visualizations into a single dashboard. Currently it supports charts, pivot table and cohorts.
* [Setting up Redash instance](https://redash.io/help-onpremise/setup/setting-up-redash-instance.html) (includes links to readymade AWS/GCE images).
* [Setting up Redash instance](https://redash.io/help/open-source/setup) (includes links to ready-made AWS/GCE images).
* [Documentation](https://redash.io/help/).
## Supported Data Sources
Redash supports more than 35 SQL and NoSQL [data sources](https://redash.io/help/data-sources/supported-data-sources). It can also be extended to support more. Below is a list of built-in sources:
* Want to report a bug or request a feature? Please open [an issue](https://github.com/getredash/redash/issues/new).
* Want to help us build **_Redash_**? Fork the project, edit in a [dev environment](https://redash.io/help-onpremise/setup/setting-up-development-environment-using-vagrant.html), and make a pull request. We need all the help we can get!
* Want to help us build **_Redash_**? Fork the project, edit in a [dev environment](https://github.com/getredash/redash/wiki/Local-development-setup) and make a pull request. We need all the help we can get!
## Security
Please email security@redash.io to report any security vulnerabilities. We will acknowledge receipt of your vulnerability and strive to send you regular updates about our progress. If you're curious about the status of your disclosure please feel free to email us again. If you want to encrypt your disclosure email, you can use [this PGP key](https://keybase.io/arikfr/key.asc).
Please email security@redash.io to report any security vulnerabilities. We will acknowledge receipt of your vulnerability and strive to send you regular updates about our progress. If you're curious about the status of your disclosure please feel free to email us again. If you want to encrypt your disclosure email, you can use [this PGP key](https://keybase.io/arikfr/key.asc).
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.