Commit Graph

543 Commits

Author SHA1 Message Date
Andrii Chubatiuk
cbde237b12 removed explicit object inheritance (#6686)
* removed explicit object inheritance

* minor fix

* pr comments

---------

Co-authored-by: Andrew Chubatiuk <andrew.chubatiuk@motional.com>
2024-01-05 19:52:02 +09:00
fabrei
58bf96c298 Adds ssl support for prometheus query runner. (#6657)
* Adds ssl support for prometheus query runner.

- Adds possibilty to upload and use of ssl cert, key and ca file in redash ui

* Extends test cases for prometheus query runner.

- Adds secret attribute to configuration schema.

* Fixes wrong timestamps in different timezones in prometheus' testcases.

- Dynamically calculates timestamps in testcases to be robust in
  different timezones.
- Adds now datetime function to make it more testable.

* Fixes timestamp in prometheus' testcases which can be wrong depending on timezone.

---------

Co-authored-by: Masayuki Takahashi <masayuki038@gmail.com>
2023-12-17 21:58:16 +09:00
fabrei
66ef942572 Adds influxdb v2 query runner. (#6646)
* Adds influxdb v2 query runner.

- Adds test cases
- Adds influxdb v2 icon
- Updates python dependencies

* Fixes import order.

* Fixes code formatting for black tool.

* Adds influxdb version 2 support in readme.

---------

Co-authored-by: Fabian Reiber <reiber@dfn-cert.de>
Co-authored-by: Masayuki Takahashi <masayuki038@gmail.com>
2023-12-12 23:01:50 +09:00
Masayuki Takahashi
9bbdb4b765 Show an error message on "Test Connection" failure for Google Spreadsheet Query Runner (#6652) 2023-12-10 00:42:46 +09:00
anushae6
09addaadc3 add e6data query runner with test cases (#6604)
* add e6data query runner with test cases

* added more test cases
2023-11-22 20:59:21 -05:00
Vladislav Denisov
a07b8a6bd3 Yandex.Disk Query runner (#6598)
* Snapshot: 23.11.0-dev

* dataframe_to_result function moved outside python query runner

* added yandex disk query runner

* moved file_extension check

* skip unsupported extensions in schema

* removed unused variable

* added support for xlsx with multiple sheets

* moved pandas-converters to utils file

* added tests

* fixed backend tests

* fixed pandas to redash type conversion

* added more tests

* added tests for pandas

* added tests for pandas converter and yandex disk

* added tests for read_file and multiple sheets

* pandas: do not load if lib is not installed

* added test for yaml read

* fixed test for yaml read

---------

Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-11-21 21:16:29 -05:00
Peter Lee
2d879510e4 add fork dashboard function (#6588)
* add fork dashboard function

* add test

* fix

---------

Co-authored-by: guyu <guyu@fordeal.com>
2023-11-11 20:56:47 +00:00
Masayuki Takahashi
198b422eaf Display catalog information on the schema pane when connecting to Trino (#6578) 2023-11-06 20:58:12 -05:00
Guido Petri
39e4ea155c Support multi column results with using mongo query runner (#6558)
* Support multi column results with using mongo query runner

* format

Signed-off-by: del-zhenwu <lizhenxiang@pjlab.org.cn>

* Update test_mongodb.py

update unit test case: nested dict for mongodb runner

* Update test_mongodb.py

* fix formatting

---------

Signed-off-by: del-zhenwu <lizhenxiang@pjlab.org.cn>
Co-authored-by: del-zhenwu <dele.zhenwu@gmail.com>
Co-authored-by: del-zhenwu <lizhenxiang@pjlab.org.cn>
2023-10-28 21:13:17 -04:00
Ken Michalak
7b03e60f9d add pagination and base_url to JSON query_runner (#6499) 2023-10-18 08:08:37 -05:00
Vladislav Denisov
42108089ed Metrica: retry query if quota exceeded (#6459)
* metrica: added retries

* updated poetry.lock

* use poetry v1.6.1

* added simple test

* convert unittest to pytest

* add 429 test

* fix 429 status code response?

---------

Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-10-16 21:53:21 -05:00
Masayuki Takahashi
011f9ef311 Add column type to a query result of InfluxDB (#6505)
* Add column type to a query result of InfluxDB

* Remove an unused import

* Migrate to pytest

* Rename test names to descriptive

---------

Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-10-08 14:39:51 +00:00
Vladislav Denisov
a19b17b844 Fixed embedded queries (#6497)
* refactored users models

* added tests
2023-10-04 15:12:17 +10:00
Eric Radman
3d32c55531 Avoid updating query result for archived queries (#6488) 2023-09-29 13:46:02 +10:00
Genki Sugawara
4a36abc628 Add Datadog alert destination (#6476)
* Add Datadog alert destination

* Fix Datadog Event API response code: 200 -> 202

* Add datadog alert dest test

* Sort test_destinations.py imports

* Fix test_datadog_notify_calls_requests_post

* Fix datadog alert dest: Add aggregation_key
2023-09-27 20:22:00 -04:00
Eric Radman
ff85a36f50 Avoid file locking issue when running CLI test (#6449)
__file__ will resolve to tests/test_cli.py, which is already opened by Python.
When tests are run on a network file system the test runner may deadlock while
waiting for an advisory lock to be released on this file.
2023-09-12 18:44:24 -04:00
Daniel Stevenson
f109af9f30 Check all Trino catalogs if no catalog configured (#5860)
* Check all Trino catalogs if no catalog configured

- If catalog is not set, query 'SHOW CATALOGS' to get catalogs
- For each catalog, discover schema, table, columns
- If catalog has a `.`, then quote the catalog name
- New table name returned includes catalog name
- If catalog is set, same result as before

* Use self._handle_run_query_error(error)

* Fix lint problem & format w/ black

* Fix condition so catalog_prefixes are [""] when catalog is set

* Fix for loop / Remove default values for catalog & schema

* Add untested tests for Trino's get_schema

* Black formatting

* Add test_get_schema_catalog_set test & fix tests

* Parameterize the side_effect fn thoroughly

* Get catalogs (not catalog_prefixes) bc confusing

* Trino._get_catalogs added for testability
2023-09-11 21:16:07 -04:00
Vlad Gramuzov
cb4af6dd57 Add Tinybird query runner (#5616)
"

Co-authored-by: Thomas Rausch <thomas@thrau.at>
2023-09-06 12:59:08 +00:00
Anirudh Bagri
c2e7df098d Add webex as destination (#5574)
* Add webex as destination

* import from destinations explicitly

* make format

* remove unattributed image

* make webex bot token required

* don't use magic string

* add metadata kwarg

* simplify link creation

* simplify alert description

* simplify alert subject

* split attachments template into method

* DRY message posting

* use api endpoint method

* add missing param to post_message

* static method

* static method attachments template

* log exception if send fails

* simplify destination handling

* transparent image / right size webex logo

* remove unused organization param

* rename api endpoint and make it a property

* add test

---------

Co-authored-by: Justin Clift <justin@postgresql.org>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-09-05 00:26:21 +00:00
Omer Lachish
f1d5ac0d58 Convert unexpected parameter validation errors to InvalidParameterError (#4084)
* convert unexpected validation errors to InvalidParameterError

* get rid of local except blocks - any failure to validate is caught in _valid()

* avoid bare exception

* don't assign exception if we're not going to use it

* preferentially raise querydetachedfromdatasource error if it is present

---------

Co-authored-by: Omer Lachish <omer@rauchy.net>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-09-04 21:08:41 +00:00
Tsuneo Yoshioka
abe70ab3ca Use multi_byte_search_enabled option for My Queries search and Favorite List search (#5761)
* Use multi_byte_search_enabled option for My Queries search and Favorite List search

* make format

* add search by user tests

---------

Co-authored-by: Justin Clift <justin@postgresql.org>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-08-31 21:43:18 -05:00
Omer Lachish
fcbe726eb2 Expire schemas after 7 days (#5112)
* expire schemas after 7 days

* expire schemas 7 days after the last refresh_schemas scheduled time

* format files

* add expire schema test

* patch schema away

* return nothing on schema get

* fix redis key name

* finally run tests locally and fix tests

---------

Co-authored-by: Omer Lachish <omer@rauchy.net>
Co-authored-by: Justin Clift <justin@postgresql.org>
Co-authored-by: konnectr <1konnectrl@gmail.com>
Co-authored-by: Konstantin Smirnov <46676677+konnectr@users.noreply.github.com>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-09-01 01:20:45 +00:00
yutopp
28db934698 Google Spreadsheet Data Source: support to find a worksheet by title (#5334)
* Support to find a worksheet by the title for google_spreadsheets

* Add tests for parse_query

* Fix types

* Add tests when finding a worksheet by a title is failed

* Proxy by a wrapper instead of using spreadsheet directly. Do not use gspread in tests

* Add tests. Fix format of quoted strings for titles

* Fix an error format

* Add a newline

* Fix formatting

---------

Co-authored-by: Justin Clift <justin@postgresql.org>
2023-08-31 16:28:12 +10:00
Eric Chang
182d84226b New alert destination: Asana (#5753)
* Add Asana alert destination

* change icon

* fix format

* add except in log

* fix flake8

* add metadata kwarg to notify

* update asana image

* make pat and project id required

* rename asana destination icon

* reuse Alert

* extract api base url to property method

* use required options

* add asana destination test

* make format

* define metadata

* fix test

* remove json.dumps from asana test

---------

Co-authored-by: Konstantin Smirnov <46676677+konnectr@users.noreply.github.com>
Co-authored-by: konnectr <1konnectrl@gmail.com>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-08-31 04:12:21 +00:00
ehearty
0258dca82a Enable manipulating parameterized queries with Query Results query runner (#5723)
* feat: pass query parameters to query results queries

* prefixing parameterized queries with param_query_<query_id>

* added tests

* Fix formatting, and also fix accidental paste.

---------

Co-authored-by: Elena Hearty <ehearty@whyhotel.com>
Co-authored-by: Justin Clift <justin@postgresql.org>
2023-08-24 20:47:27 -04:00
Omer Lachish
2d6f5b091c Base duplicate index on column names (#4600)
* increase duplicate column names based on the original column name and not the number of duplicates detected

* add fetch columns test for base query runner

---------

Co-authored-by: Omer Lachish <omer@rauchy.net>
Co-authored-by: Guido Petri <18634426+guidopetri@users.noreply.github.com>
2023-08-24 04:18:05 +00:00
Ivan Torgashov
e18cd8f248 Passing Request metadata to Alert destinations (#5230) 2023-08-23 23:08:52 -04:00
htamakos
63140260eb Add useQueryAnnotation option to BigQuery Runner (#5675)
* Add useQueryAnnotation option to BigQuery Runner

* Fix formatting

---------

Co-authored-by: Justin Clift <justin@postgresql.org>
2023-08-22 21:43:07 -04:00
Stephen Darlington
9751678c44 Add Apache ignite Query Runner (#5767) 2023-08-23 03:47:10 +10:00
Evgeny
4a5c9c2630 Scheduled queries should include origin Username (#6368) 2023-08-16 13:20:19 +10:00
Harry C
d333660473 Fix python query runner crashing the GUI due to missing 'rows' and/or… (#5749)
* Fix python query runner crashing the GUI due to missing 'rows' and/or 'columns' in the JSON returned data.

* Fix typo of previous commit.

* Throw exception when python query runner has invalid result.

* Update test_python.py
---------

Co-authored-by: YuhengChen <yuheng.chen@imaygou.com>
2023-08-07 21:03:12 +05:00
Konstantin Smirnov
196bfece30 Fix warning test query (#6335)
* replace assertDictContainsSubset -> asserLessEqual

* refactor definition set
2023-08-01 18:57:22 +08:00
Eric Radman
0d699328b8 Allow RSA key used for JWT to be specified as a file path (#6271)
- auth_jwt_auth_public_certs_url may file:// in addition to http/https
- Log an error if payload does not contain an email address
2023-07-29 19:20:18 +10:00
Bryan Yang
8f71e14887 Fixed python query runner and add unittest for python query runner (#4731)
* add test for python runner

* fixed CustomPrint

* remove print

* Reformatted to pass our backend lint tests

Reformatted using:

  $ make format

---------

Co-authored-by: Justin Clift <justin@postgresql.org>
2023-07-28 20:40:16 -05:00
Tadayuki Onishi
f2e31a602e [enhancement] Add a CLI command in order to create organization (#5869)
* Add a cli command in order to create organization

* Update redash/cli/organization.py

Co-authored-by: Jun <junnplus@gmail.com>


* add test

---------

Co-authored-by: Konstantin Smirnov <46676677+konnectr@users.noreply.github.com>
Co-authored-by: Jun <junnplus@gmail.com>
Co-authored-by: konnectr <1konnectrl@gmail.com>
2023-07-22 19:20:17 +05:00
Fernando Salcido
79ef3e4eb0 Handle error when data_source is None (#4936)
* add try except for data_source

* add try except for data_source

* fix errir handle

* add test for no data source

* changed error handle

* fix format

* fix test

---------

Co-authored-by: Konstantin Smirnov <46676677+konnectr@users.noreply.github.com>
Co-authored-by: konnectr <1konnectrl@gmail.com>
2023-07-22 19:17:49 +05:00
Vitaly Vakhteev
7f4ade5f1f Feature/Add Discord Alert Destination (#6106)
* Add discord webhook

* Fix icon

* Boto3 dependency

* Add unit test for Discord webhook

* Add suggestions

* Apply suggestions from code review

Co-authored-by: Jun <junnplus@gmail.com>

* Misunderstood suggestion )

* Add suggestions

* Apply suggestions from code review

Co-authored-by: Jun <junnplus@gmail.com>

* Fix test

* Fix variables in strings

* Fix formatting using our pre-commit hook

---------

Co-authored-by: Jun <junnplus@gmail.com>
Co-authored-by: Justin Clift <justin@postgresql.org>
2023-07-21 12:14:42 +00:00
Peter Lee
be306e9284 fix pubic dashboard not update result #6203 (#6214) 2023-07-21 12:32:59 +05:00
Jun
77a2c24d47 fix test warning (#6217)
Signed-off-by: Ye Sijun <junnplus@gmail.com>
2023-07-21 14:52:46 +09:00
Jun
20dbb461e9 Use assertEqual instead of assertEquals (#6210)
Signed-off-by: Ye Sijun <junnplus@gmail.com>
2023-07-21 10:52:18 +09:00
Justin Clift
ff6377b6e2 Bump SQLAlchemy-Searchable to 1.2.0 (#6173)
* Drop pyparsing, update SQLAlchemy-Searchable to 1.2.0

* fix make_searchable (#6180)

* fix test

Signed-off-by: Ye Sijun <junnplus@gmail.com>

---------

Signed-off-by: Ye Sijun <junnplus@gmail.com>
Co-authored-by: Spencer Weeks <Spince@users.noreply.github.com>
Co-authored-by: Ye Sijun <junnplus@gmail.com>
2023-07-20 23:27:58 +09:00
Eric Radman
0bdd3bd826 Add support for a default alert template (#5996)
- Escape all variables by default since Mustache only has a syntax for raw values
- Generate a generic HTML result table to allow an alert template to display any query
- Optionally allow alert template to be defined to be set using REDASH_ALERTS_DEFAULT_MAIL_BODY_TEMPLATE
- Formatting updated by black
2023-07-18 06:51:42 +10:00
Vinod Pandey
095ac2ecf0 Added query runner for Google Analytics Data API (GA4) and Google Search Console API (#5868)
* quick and dirty GA4 integration to support limited queries

request pattern:
{
   "propertyId": 123456789,
   "dateRanges": [{ "startDate": "yesterday", "endDate": "yesterday" }],
   "dimensions": [{ "name": "date" }],
   "metrics": [{ "name": "activeUsers" }]
}

* Update Dockerfile

* Update requirements_bundles.txt

related issue: 
https://github.com/getredash/redash/pull/5851
https://stackoverflow.com/questions/73929564/entrypoints-object-has-no-attribute-get-digital-ocean

* updated integration code to support all usecases for runReport endpoint (does not have support for runPivotReport yet)

* added google search console as query runner

* removed info logger

* removed files that were causing merge conflicts

* fixed failing testcases and pre-commit formatting changes

* added testcases

* fixed linting errors in test files

* code optimization
2023-07-16 19:45:14 -05:00
Jun
9b2f635692 format code by black and isort (#6167)
Signed-off-by: Ye Sijun <junnplus@gmail.com>
2023-07-11 19:13:54 +10:00
Jun
73f49cbf0c upgrade flask (#6138)
* upgrade flask

Signed-off-by: Ye Sijun <junnplus@gmail.com>

* fix test

Signed-off-by: Ye Sijun <junnplus@gmail.com>

* override value_proc for click.prompt

Signed-off-by: Ye Sijun <junnplus@gmail.com>

---------

Signed-off-by: Ye Sijun <junnplus@gmail.com>
2023-07-08 06:05:27 +09:00
Arik Fraimovich
28b0a2379d Remove extensions mechanism (#5895)
* Remove extensions mechanism.

* Missing change.
2023-03-20 07:39:21 -07:00
Jesse
4186f8303e New ElasticSearch Query Runner (#5794)
- A runner supporting the newest versions of ES,
  aggregation, nested aggregations and nested fields.
- A runner for the SQL OpenDistro flavor
- A runner for the SQL X-Pack flavor

Co-authored-by: Nicolas Le Manchet <nicolas@lemanchet.fr>
Co-authored-by: wwl717195673 <717195673@qq.com>
2022-07-20 07:47:44 -05:00
Jesse
9abc4f5f1e Clickhouse: Multi-statements support (#5792)
ClickHouse query runner splits query into several and execute each query in turn. The result of the last execution is returned. Implementation uses ClickHouse sessions in the HTTP protocol. `session_id` is generated for the first query and then it is used with the subsequent queries (together with the `session_check` parameter).

If query runner gets a success response with empty body from ClickHouse (for example, in case of temporary table creation request) query runner returns empty response.

authored-by: Liubov Ulitina <ulitinalm@vl.ru>
2022-07-12 12:27:20 -05:00
Jesse
65f7b6c5af Sort Python safe built-ins (#5781)
Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
2022-07-06 08:16:36 -05:00
Jesse
12c4750684 Fix: don't accept password login requests if password auth is disabled (#5693) 2022-01-28 08:52:31 -06:00