583 Commits

Author SHA1 Message Date
Vladislav Denisov
42108089ed Metrica: retry query if quota exceeded (#6459)
* metrica: added retries

* updated poetry.lock

* use poetry v1.6.1

* added simple test

* convert unittest to pytest

* add 429 test

* fix 429 status code response?

---------

Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-10-16 21:53:21 -05:00
Masayuki Takahashi
011f9ef311 Add column type to a query result of InfluxDB (#6505)
* Add column type to a query result of InfluxDB

* Remove an unused import

* Migrate to pytest

* Rename test names to descriptive

---------

Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-10-08 14:39:51 +00:00
Vladislav Denisov
a19b17b844 Fixed embedded queries (#6497)
* refactored users models

* added tests
2023-10-04 15:12:17 +10:00
Eric Radman
3d32c55531 Avoid updating query result for archived queries (#6488) 2023-09-29 13:46:02 +10:00
Genki Sugawara
4a36abc628 Add Datadog alert destination (#6476)
* Add Datadog alert destination

* Fix Datadog Event API response code: 200 -> 202

* Add datadog alert dest test

* Sort test_destinations.py imports

* Fix test_datadog_notify_calls_requests_post

* Fix datadog alert dest: Add aggregation_key
2023-09-27 20:22:00 -04:00
Eric Radman
ff85a36f50 Avoid file locking issue when running CLI test (#6449)
__file__ will resolve to tests/test_cli.py, which is already opened by Python.
When tests are run on a network file system the test runner may deadlock while
waiting for an advisory lock to be released on this file.
2023-09-12 18:44:24 -04:00
Daniel Stevenson
f109af9f30 Check all Trino catalogs if no catalog configured (#5860)
* Check all Trino catalogs if no catalog configured

- If catalog is not set, query 'SHOW CATALOGS' to get catalogs
- For each catalog, discover schema, table, columns
- If catalog has a `.`, then quote the catalog name
- New table name returned includes catalog name
- If catalog is set, same result as before

* Use self._handle_run_query_error(error)

* Fix lint problem & format w/ black

* Fix condition so catalog_prefixes are [""] when catalog is set

* Fix for loop / Remove default values for catalog & schema

* Add untested tests for Trino's get_schema

* Black formatting

* Add test_get_schema_catalog_set test & fix tests

* Parameterize the side_effect fn thoroughly

* Get catalogs (not catalog_prefixes) bc confusing

* Trino._get_catalogs added for testability
2023-09-11 21:16:07 -04:00
Vlad Gramuzov
cb4af6dd57 Add Tinybird query runner (#5616)
"

Co-authored-by: Thomas Rausch <thomas@thrau.at>
2023-09-06 12:59:08 +00:00
Anirudh Bagri
c2e7df098d Add webex as destination (#5574)
* Add webex as destination

* import from destinations explicitly

* make format

* remove unattributed image

* make webex bot token required

* don't use magic string

* add metadata kwarg

* simplify link creation

* simplify alert description

* simplify alert subject

* split attachments template into method

* DRY message posting

* use api endpoint method

* add missing param to post_message

* static method

* static method attachments template

* log exception if send fails

* simplify destination handling

* transparent image / right size webex logo

* remove unused organization param

* rename api endpoint and make it a property

* add test

---------

Co-authored-by: Justin Clift <justin@postgresql.org>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-09-05 00:26:21 +00:00
Omer Lachish
f1d5ac0d58 Convert unexpected parameter validation errors to InvalidParameterError (#4084)
* convert unexpected validation errors to InvalidParameterError

* get rid of local except blocks - any failure to validate is caught in _valid()

* avoid bare exception

* don't assign exception if we're not going to use it

* preferentially raise querydetachedfromdatasource error if it is present

---------

Co-authored-by: Omer Lachish <omer@rauchy.net>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-09-04 21:08:41 +00:00
Tsuneo Yoshioka
abe70ab3ca Use multi_byte_search_enabled option for My Queries search and Favorite List search (#5761)
* Use multi_byte_search_enabled option for My Queries search and Favorite List search

* make format

* add search by user tests

---------

Co-authored-by: Justin Clift <justin@postgresql.org>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-08-31 21:43:18 -05:00
Omer Lachish
fcbe726eb2 Expire schemas after 7 days (#5112)
* expire schemas after 7 days

* expire schemas 7 days after the last refresh_schemas scheduled time

* format files

* add expire schema test

* patch schema away

* return nothing on schema get

* fix redis key name

* finally run tests locally and fix tests

---------

Co-authored-by: Omer Lachish <omer@rauchy.net>
Co-authored-by: Justin Clift <justin@postgresql.org>
Co-authored-by: konnectr <1konnectrl@gmail.com>
Co-authored-by: Konstantin Smirnov <46676677+konnectr@users.noreply.github.com>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-09-01 01:20:45 +00:00
yutopp
28db934698 Google Spreadsheet Data Source: support to find a worksheet by title (#5334)
* Support to find a worksheet by the title for google_spreadsheets

* Add tests for parse_query

* Fix types

* Add tests when finding a worksheet by a title is failed

* Proxy by a wrapper instead of using spreadsheet directly. Do not use gspread in tests

* Add tests. Fix format of quoted strings for titles

* Fix an error format

* Add a newline

* Fix formatting

---------

Co-authored-by: Justin Clift <justin@postgresql.org>
2023-08-31 16:28:12 +10:00
Eric Chang
182d84226b New alert destination: Asana (#5753)
* Add Asana alert destination

* change icon

* fix format

* add except in log

* fix flake8

* add metadata kwarg to notify

* update asana image

* make pat and project id required

* rename asana destination icon

* reuse Alert

* extract api base url to property method

* use required options

* add asana destination test

* make format

* define metadata

* fix test

* remove json.dumps from asana test

---------

Co-authored-by: Konstantin Smirnov <46676677+konnectr@users.noreply.github.com>
Co-authored-by: konnectr <1konnectrl@gmail.com>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-08-31 04:12:21 +00:00
ehearty
0258dca82a Enable manipulating parameterized queries with Query Results query runner (#5723)
* feat: pass query parameters to query results queries

* prefixing parameterized queries with param_query_<query_id>

* added tests

* Fix formatting, and also fix accidental paste.

---------

Co-authored-by: Elena Hearty <ehearty@whyhotel.com>
Co-authored-by: Justin Clift <justin@postgresql.org>
2023-08-24 20:47:27 -04:00
Omer Lachish
2d6f5b091c Base duplicate index on column names (#4600)
* increase duplicate column names based on the original column name and not the number of duplicates detected

* add fetch columns test for base query runner

---------

Co-authored-by: Omer Lachish <omer@rauchy.net>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-08-24 04:18:05 +00:00
Ivan Torgashov
e18cd8f248 Passing Request metadata to Alert destinations (#5230) 2023-08-23 23:08:52 -04:00
htamakos
63140260eb Add useQueryAnnotation option to BigQuery Runner (#5675)
* Add useQueryAnnotation option to BigQuery Runner

* Fix formatting

---------

Co-authored-by: Justin Clift <justin@postgresql.org>
2023-08-22 21:43:07 -04:00
Stephen Darlington
9751678c44 Add Apache ignite Query Runner (#5767) 2023-08-23 03:47:10 +10:00
Evgeny
4a5c9c2630 Scheduled queries should include origin Username (#6368) 2023-08-16 13:20:19 +10:00
Harry C
d333660473 Fix python query runner crashing the GUI due to missing 'rows' and/or… (#5749)
* Fix python query runner crashing the GUI due to missing 'rows' and/or 'columns' in the JSON returned data.

* Fix typo of previous commit.

* Throw exception when python query runner has invalid result.

* Update test_python.py
---------

Co-authored-by: YuhengChen <yuheng.chen@imaygou.com>
2023-08-07 21:03:12 +05:00
Konstantin Smirnov
196bfece30 Fix warning test query (#6335)
* replace assertDictContainsSubset -> asserLessEqual

* refactor definition set
2023-08-01 18:57:22 +08:00
Eric Radman
0d699328b8 Allow RSA key used for JWT to be specified as a file path (#6271)
- auth_jwt_auth_public_certs_url may file:// in addition to http/https
- Log an error if payload does not contain an email address
2023-07-29 19:20:18 +10:00
Bryan Yang
8f71e14887 Fixed python query runner and add unittest for python query runner (#4731)
* add test for python runner

* fixed CustomPrint

* remove print

* Reformatted to pass our backend lint tests

Reformatted using:

  $ make format

---------

Co-authored-by: Justin Clift <justin@postgresql.org>
2023-07-28 20:40:16 -05:00
Tadayuki Onishi
f2e31a602e [enhancement] Add a CLI command in order to create organization (#5869)
* Add a cli command in order to create organization

* Update redash/cli/organization.py

Co-authored-by: Jun <junnplus@gmail.com>


* add test

---------

Co-authored-by: Konstantin Smirnov <46676677+konnectr@users.noreply.github.com>
Co-authored-by: Jun <junnplus@gmail.com>
Co-authored-by: konnectr <1konnectrl@gmail.com>
2023-07-22 19:20:17 +05:00
Fernando Salcido
79ef3e4eb0 Handle error when data_source is None (#4936)
* add try except for data_source

* add try except for data_source

* fix errir handle

* add test for no data source

* changed error handle

* fix format

* fix test

---------

Co-authored-by: Konstantin Smirnov <46676677+konnectr@users.noreply.github.com>
Co-authored-by: konnectr <1konnectrl@gmail.com>
2023-07-22 19:17:49 +05:00
Vitaly Vakhteev
7f4ade5f1f Feature/Add Discord Alert Destination (#6106)
* Add discord webhook

* Fix icon

* Boto3 dependency

* Add unit test for Discord webhook

* Add suggestions

* Apply suggestions from code review

Co-authored-by: Jun <junnplus@gmail.com>

* Misunderstood suggestion )

* Add suggestions

* Apply suggestions from code review

Co-authored-by: Jun <junnplus@gmail.com>

* Fix test

* Fix variables in strings

* Fix formatting using our pre-commit hook

---------

Co-authored-by: Jun <junnplus@gmail.com>
Co-authored-by: Justin Clift <justin@postgresql.org>
2023-07-21 12:14:42 +00:00
Peter Lee
be306e9284 fix pubic dashboard not update result #6203 (#6214) 2023-07-21 12:32:59 +05:00
Jun
77a2c24d47 fix test warning (#6217)
Signed-off-by: Ye Sijun <junnplus@gmail.com>
2023-07-21 14:52:46 +09:00
Jun
20dbb461e9 Use assertEqual instead of assertEquals (#6210)
Signed-off-by: Ye Sijun <junnplus@gmail.com>
2023-07-21 10:52:18 +09:00
Justin Clift
ff6377b6e2 Bump SQLAlchemy-Searchable to 1.2.0 (#6173)
* Drop pyparsing, update SQLAlchemy-Searchable to 1.2.0

* fix make_searchable (#6180)

* fix test

Signed-off-by: Ye Sijun <junnplus@gmail.com>

---------

Signed-off-by: Ye Sijun <junnplus@gmail.com>
Co-authored-by: Spencer Weeks <Spince@users.noreply.github.com>
Co-authored-by: Ye Sijun <junnplus@gmail.com>
2023-07-20 23:27:58 +09:00
Eric Radman
0bdd3bd826 Add support for a default alert template (#5996)
- Escape all variables by default since Mustache only has a syntax for raw values
- Generate a generic HTML result table to allow an alert template to display any query
- Optionally allow alert template to be defined to be set using REDASH_ALERTS_DEFAULT_MAIL_BODY_TEMPLATE
- Formatting updated by black
2023-07-18 06:51:42 +10:00
Vinod Pandey
095ac2ecf0 Added query runner for Google Analytics Data API (GA4) and Google Search Console API (#5868)
* quick and dirty GA4 integration to support limited queries

request pattern:
{
   "propertyId": 123456789,
   "dateRanges": [{ "startDate": "yesterday", "endDate": "yesterday" }],
   "dimensions": [{ "name": "date" }],
   "metrics": [{ "name": "activeUsers" }]
}

* Update Dockerfile

* Update requirements_bundles.txt

related issue: 
https://github.com/getredash/redash/pull/5851
https://stackoverflow.com/questions/73929564/entrypoints-object-has-no-attribute-get-digital-ocean

* updated integration code to support all usecases for runReport endpoint (does not have support for runPivotReport yet)

* added google search console as query runner

* removed info logger

* removed files that were causing merge conflicts

* fixed failing testcases and pre-commit formatting changes

* added testcases

* fixed linting errors in test files

* code optimization
2023-07-16 19:45:14 -05:00
Jun
9b2f635692 format code by black and isort (#6167)
Signed-off-by: Ye Sijun <junnplus@gmail.com>
2023-07-11 19:13:54 +10:00
Jun
73f49cbf0c upgrade flask (#6138)
* upgrade flask

Signed-off-by: Ye Sijun <junnplus@gmail.com>

* fix test

Signed-off-by: Ye Sijun <junnplus@gmail.com>

* override value_proc for click.prompt

Signed-off-by: Ye Sijun <junnplus@gmail.com>

---------

Signed-off-by: Ye Sijun <junnplus@gmail.com>
2023-07-08 06:05:27 +09:00
Arik Fraimovich
28b0a2379d Remove extensions mechanism (#5895)
* Remove extensions mechanism.

* Missing change.
2023-03-20 07:39:21 -07:00
Jesse
4186f8303e New ElasticSearch Query Runner (#5794)
- A runner supporting the newest versions of ES,
  aggregation, nested aggregations and nested fields.
- A runner for the SQL OpenDistro flavor
- A runner for the SQL X-Pack flavor

Co-authored-by: Nicolas Le Manchet <nicolas@lemanchet.fr>
Co-authored-by: wwl717195673 <717195673@qq.com>
2022-07-20 07:47:44 -05:00
Jesse
9abc4f5f1e Clickhouse: Multi-statements support (#5792)
ClickHouse query runner splits query into several and execute each query in turn. The result of the last execution is returned. Implementation uses ClickHouse sessions in the HTTP protocol. `session_id` is generated for the first query and then it is used with the subsequent queries (together with the `session_check` parameter).

If query runner gets a success response with empty body from ClickHouse (for example, in case of temporary table creation request) query runner returns empty response.

authored-by: Liubov Ulitina <ulitinalm@vl.ru>
2022-07-12 12:27:20 -05:00
Jesse
65f7b6c5af Sort Python safe built-ins (#5781)
Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-07-06 08:16:36 -05:00
Jesse
12c4750684 Fix: don't accept password login requests if password auth is disabled (#5693) 2022-01-28 08:52:31 -06:00
Tin C
f77f1b5ca1 Fix: auto limit breaks for Oracle queries (#5181)
Moves auto limit primitives to the base SQL query runner
2022-01-20 12:03:04 -06:00
Jesse
61bbb5aa7a Merge pull request from GHSA-fcpv-hgq6-87h7 2021-11-23 14:58:20 -08:00
Jesse
3c248acf21 Fix: pagination is broken on the dashboard list page (#5612)
* Fix: pagination is broken on the dashboard list page (#5516)
* Add test that reproduces issue #5466
* Fix: Duplicate dashboard rows were returned by Dashboard.all() (#5466)
* Update changelog for V10
* Update changelog for #5516
2021-10-21 11:25:30 -05:00
adamzwakk
e4e567bbb9 Fixing failure report rendering (#5492) 2021-05-14 06:25:52 -07:00
Jesse
9c8c1bfa9a Adds rate limit to /forgot. (#5425)
Security vulnerability was disclosed by Sohail Ahmed <https://www.linkedin.com/in/sohail-ahmed-755776184/>
2021-04-26 12:02:47 -05:00
iwakiriK
f21f7e211f Athena: skip tables with no StorageDescriptor (#5447) 2021-04-21 15:01:57 -05:00
Jesse
67263e1b0f Fixes issue #5445: Scheduled query not working (#5448)
* use 'query_id' everywhere instead of 'Query ID'
* some black while we're at it

Co-authored-by: Omer Lachish <omer@rauchy.net>
2021-04-08 13:32:34 -05:00
Omer Lachish
9fdf1f341d Reset failure counter on adhoc success (#5394)
* reset failure counter when query completes successfully via adhoc

* Use "query_id" in metadata, but still allow "Query ID" for transition/legacy support
2021-03-12 12:02:29 -08:00
Justin Talbot
0e1587a068 Add My Dashboards filter option to the Dashboards list (#5375)
* Add My Dashboards filter option to the Dashboards list. Added API endpoint to get the list of a user's dashboards, similar to the My Queries feature.

* Update empty dashboard list state to show an invite to create a new dashboard, like My Queries

* Update to Levko's suggested approach. Clean up some of the formatting for consistency. Put the 'My Queries/Dashboards' item before the Favorites since that organization seems cleaner to me.

* Address Levko's comments
2021-02-02 12:37:48 +02:00
Arik Fraimovich
e71ccf5de5 Fix: add a merge migration to solve multi head issue (#5364)
* Add unit test to test for multi-head migrations issue

* Add merge migration
2021-01-21 10:55:52 -08:00