Commit Graph

1920 Commits

Author SHA1 Message Date
Omer Lachish
570e8d9f23 set invitation as not-pending in render_token_login_page only when (#3500)
handling invites (as opposed to password reset links)
2019-02-27 09:06:41 +02:00
Omer Lachish
0d76c036cb Be more permissive when parameters are safe (#3383)
* use the textless endpoint (/api/queries/:id/results) for pristine
queriest

* reverse conditional. not not is making me the headaches.

* add ParameterizedQuery#is_safe with an inital naive implementation which
treats any query with a text parameter as not safe. This will be
remedied later when DB drivers will handle these parameters.

* allow getting new query results even if user has only view permissions
to the data source (given that the query is safe)

* fix lint error - getDerivedStateFromProps should be placed after state

* Revert "use the textless endpoint (/api/queries/:id/results) for pristine"

This reverts commit cd2cee7738.

* move execution preparation to a different function, which will be soon
reused

* go to textless /api/queries/:id/results by default

* let the query view decide if text or textless endpoint is needed

* allow safe queries to be executed in the UI even if the user has no
permission to execute and create new query results

* change `run_query`'s signature to accept a ParameterizedQuery instead of
constructing it inside

* use dict#get instead of a None guard

* use ParameterizedQuery in queries handler as well

* test that /queries/:id/results allows execution of safe queries even if
user has view_only permissions

* lint

* raise HTTP 400 when receiving invalid parameter values. Fixes #3394

* remove unused methods

* avoid cyclic imports by importing only when needed

* verify that a ParameterizedQuery without any parameters is considered
safe

* introduce query.parameter_schema

* encapsulate ParameterizedQuery creation inside Query
2019-02-26 20:55:01 +02:00
Arik Fraimovich
5c5bfbdbbe Make sure Flask app created in Celery's worker process (#3465)
* Make sure Flask app created in worker process

* Add reference to GitHub issue
2019-02-25 19:11:03 +02:00
Sergei Beregov
75c34bf18d Add custom JSON encoder for PostgreSQL (#3442)
To handle columns with [range types][1] and display them as a
string custom JSON encoder for PostgreSQL was added.

Merging this PR will fix issue #1764

[1]:https://www.postgresql.org/docs/9.3/rangetypes.html
2019-02-25 17:52:45 +02:00
Wei
1a357df9b3 Fix prometheus query runner: get_schema and query range (#3471) 2019-02-25 11:40:35 +02:00
Subhi Al Hasan
8679b8756e Cleaner approach for _is_collection_a_view (#3113)
* Fix collection fields retreival bug when Views are present in MongoDB

* fixing _is_collection_a_view function

* Update redash/query_runner/mongodb.py

Co-Authored-By: jodevsa <jodevsa@gmail.com>

* using options() instead of try/except on collstats  to detect if a collection  is a view
2019-02-21 23:39:36 +02:00
Omer Lachish
2a37cb31d9 Fix max-age issues on textless endpoint (#3477)
* max_age should default to -1

* pass maxAge along to `execute`
2019-02-21 16:02:06 +02:00
Arik Fraimovich
4999ab5de7 Provide default value for .pop to avoid KeyError exception (#3474) 2019-02-21 11:17:07 +02:00
Omer Lachish
a17eb14cdf support e-mail verification for multi org setups (#3468) 2019-02-20 13:45:09 +02:00
Omer Lachish
1ad0fa6a9b avoid Flask debug error message about strict slashes (#3469) 2019-02-20 13:19:06 +02:00
Arik Fraimovich
77dcc80522 Fix: ParameterizedQuery: support for upper cases column names. (#3461)
* Fix: ParameterizedQuery: support for upper cases column names.

* Fix test name
2019-02-20 12:08:38 +02:00
Arik Fraimovich
e35f2b8f51 Fix: when max_age is None the handler fails (#3462) 2019-02-19 20:47:29 +02:00
Omer Lachish
0bca2d8920 when working with a schema, fail in the case that there are parameters (#3452)
which are not part of the schema
2019-02-19 20:46:30 +02:00
Arik Fraimovich
f421119f9d Remove options NullPool does not support (#3460) 2019-02-19 16:04:00 +02:00
Arik Fraimovich
ebef0efe06 Fix: timedelta values were not JSON serialized properly (#3463)
.
2019-02-19 14:27:37 +01:00
Ran Byron
3147a0bd98 Version data - footer -> header menu (#3458) 2019-02-18 22:15:51 +02:00
Arik Fraimovich
60472e2fe0 Add support for Amazon ES service with IAM authentication (#3446)
* Add support for Amazon ES service with IAM authentication

* Add required dependency.
2019-02-17 15:11:40 +02:00
Arik Fraimovich
a34269cc7d Change: encrypt data source options. 🔓 (#2970)
* Change: encrypt data source options
* Implement migration
2019-02-17 13:54:19 +02:00
Omer Lachish
a8f74a1078 remove the word "type" from error message, as enum out-of-range errors (#3449)
have nothing to do with types.
2019-02-17 13:42:44 +02:00
Arik Fraimovich
58a53e3470 Fix: remove widgets when deleting a visualization. (#3423)
Closes #3257.
2019-02-17 10:30:23 +02:00
swfz
afaedb9062 [Feature] Table visualization: Raise the upper limit of MAX_JSON_SIZE (#3310)
* move constant value to clientConfig
* change name maxJsonSize to tableCellMaxJSONSize
* value from environment. default is 50000
2019-02-13 11:42:40 +02:00
Arik Fraimovich
d6827e3601 Version update time (#3429)
* Version update time 

* Need more 
2019-02-13 09:51:07 +02:00
Omer Lachish
c028e49bfd send ip and user id to sentry (#3430) 2019-02-13 09:41:41 +02:00
Omer Lachish
330c5a85f1 Enable remote debugging with ptvsd (#3419)
* open port 3000 for remote debugging

* add ptvsd

* use port 5678 to avoid changes in VSCode's default config

* attach to ptvsd

* no need to wait for attach

* actually, --debugger seems to be working

* create a new docker entry point for remote debugging

* alternative method to switch to debugging
2019-02-12 09:10:18 +02:00
Arik Fraimovich
cb22764d68 Bug fix] Saving a new query removes reference to last query result (#3421)
* Correctly test if the current query result is for the current query.
* Serialize a new query with its visualizations.
2019-02-11 14:13:01 +02:00
Omer Lachish
71fb1442f1 Upgrade Sentry SDK (#3418)
* replace raven with sentry-sdk

* use sentry-sdk in celery

* use sentry-sdk with flask

* unify Flask and Celery initializations for Sentry

* extract sentry stuff to own module

* it's time for Sentry 0.7.2
2019-02-10 21:56:16 +02:00
Jannis Leidel
23908edc28 Handle InterruptException in Athena query runnner like in the Presto query runner. (#3403) 2019-02-10 13:41:16 +02:00
Omer Lachish
03f040da0e Unify query based dropdown population (#3337)
* stop testing `collect_query_parameters`, it's an implementation detail

* add tests for `missing_query_params`

* rename SQLQuery -> ParameterizedSqlQuery

* rename sql_query.py to parameterized_query.py

* split to parameterized queries and parameterized SQL queries, where
parameterized queries only do templating and parameterized SQL queries
add tree validation on top of it

* move missing parameter detection to ParameterizedQuery

* get rid of some old code

* fix tests

* set syntax to `custom`

* revert the max-age-related refactoring

* 👋 tree validations 😢

* BaseQueryRunner is no longer a factory for ParameterizedQuery, for now

* add an endpoint for running a query by its id and (optional) parameters
without having to provide the query text

* adds parameter schema to ParameterizedQuery

* adds parameter schema validation (currently for strings)

* validate number parameters

* validate date parameters

* validate parameters on POST /api/queries/<id>/results

* validate enum parameters

* validate date range parameters

* validate query-based dropdowns by preprocessing them at the handler
level and converting them to a populated enum

* change _is_date_range to be a tad more succinct

* a single assignment with a `map` is sufficiently explanatory

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/handlers/query_results.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* build error message inside the error

* support all types of numbers as number parameters

* check for permissions when populating query-based dropdowns

* check for access to query before running it

* check for empty rows when populating query-based enums

* don't bother loading query results if user doesn't have access

* 💥 on unexpected parameter types

* parameter schema default is a list, not a dictionary

* fix a totally unrelated typo

* remove redundant null guards

* introduce /dropdown.json endpoint with dummy data

* wire frontend to /dropdown.json

* always return name/value combos from /dropdown.json

* load actual data into /dropdown.json

* pluck correct values for `name` and `value`

* reuse dropdwon plucking logic in QueryResultResource

* simplify _get_dropdown_values

* when doing parameter validation, we only care about the value and not
the display name

* rename dropdown to dropdownOptions

* move dropdown_values to utils/parameterized_query.py

* stop converting queries to enums and encapsulate the work inside
ParameterizedQuery (almost - /dropdown.json would still access the
dropdown_values method)

* re-order arguments by importance

* test query parameter validation

* tests for dropdown_values logic

* remove `.json` suffix to the dropdown endpoint

* allow `BaseResource` to handle JSON stuff

* move _pluck_name_and_value outside its containing method

* case-insensitive lookup when plucking name and value

* separate concerns and simplify test isolation for `dropdown_values`

* pick the default column according to the order specified in the query
result columns attribute

* use `current_org` instead of passing `org`

* test that user has access to the query when calling the /dropdown
endpoint
2019-02-10 13:10:39 +02:00
Levko Kravets
11507c5e5e Show active and pending users separately (for admins) (#3400) 2019-02-07 20:30:55 +02:00
Arik Fraimovich
7cfea8a6a0 Fix: only login user when it's the current user. (#3396)
Otherwise it would login the admin as the user...
2019-02-05 15:53:51 +02:00
Arik Fraimovich
933dd753a8 Make the logic around schedule['until'] easier to read (#3376) 2019-02-03 21:13:27 +02:00
Jannis Leidel
69e34f048a Add archived queries section to queries list. (#2888)
* Add archived queries section to queries list.

* Refactor route building for list based controllers.

This also fixes the dashboard empty state page.
2019-02-03 14:35:25 +02:00
Marina Samuel
feab2a040b BigQuery should correctly handle tmp tables that do not have a schema field. (#3382) 2019-01-31 23:20:13 +02:00
Aidarbek Suleimenov
0b9f575dab Fix: make ClickHouse password and username truly optional (#3362)
* clickhouse optional password

* clickhouse URL and user made optional
2019-01-30 11:17:43 +02:00
Aidarbek Suleimenov
61e7cdaa81 Filename set when /results called directly (#3359)
* filename set when /results called directly

* /results filename changed from query name to id

* Long line shortened
2019-01-29 19:47:42 +02:00
Vladislav Denisov
53aecdc607 yandex_metrica: changed auth from params to headers (#3360) 2019-01-29 18:17:13 +02:00
Omer Lachish
371b319e92 Server-side parameter validation (#3315)
* stop testing `collect_query_parameters`, it's an implementation detail

* add tests for `missing_query_params`

* rename SQLQuery -> ParameterizedSqlQuery

* rename sql_query.py to parameterized_query.py

* split to parameterized queries and parameterized SQL queries, where
parameterized queries only do templating and parameterized SQL queries
add tree validation on top of it

* move missing parameter detection to ParameterizedQuery

* get rid of some old code

* fix tests

* set syntax to `custom`

* revert the max-age-related refactoring

* 👋 tree validations 😢

* BaseQueryRunner is no longer a factory for ParameterizedQuery, for now

* add an endpoint for running a query by its id and (optional) parameters
without having to provide the query text

* adds parameter schema to ParameterizedQuery

* adds parameter schema validation (currently for strings)

* validate number parameters

* validate date parameters

* validate parameters on POST /api/queries/<id>/results

* validate enum parameters

* validate date range parameters

* validate query-based dropdowns by preprocessing them at the handler
level and converting them to a populated enum

* change _is_date_range to be a tad more succinct

* a single assignment with a `map` is sufficiently explanatory

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/handlers/query_results.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* build error message inside the error

* support all types of numbers as number parameters

* check for permissions when populating query-based dropdowns

* check for access to query before running it

* check for empty rows when populating query-based enums

* don't bother loading query results if user doesn't have access

* 💥 on unexpected parameter types

* parameter schema default is a list, not a dictionary

* remove redundant null guards
2019-01-29 09:18:07 +02:00
Arik Fraimovich
c31cb01065 Move BigQueryGCE to its own file (#3356)
* Move BigQueryGCE to its own file

* Add missing import
2019-01-28 09:37:47 +02:00
Vibhor Kumar
1a61ee3ec0 Add: Uptycs query runner (#3319)
* adding uptycs query_runner in redash

* as per comment from Arik comment fixed the code

* fixed function_name

* fixed some indentation issues

* fixed the indentation issue and taken out customer_id from secret

* fixed the dependency of urllib3

* fixed the indententaton issue

* remved the urllib3 from requirements

* fixed the indentation issues

* added the new square image for Uptycs. Removed unnecessary variable and made ssl as an option

* fixed indentation issue

* Renamed SSL to verify_ssl and also added verify_ssl validate in verify in missing places
2019-01-23 20:07:40 +02:00
Omer Lachish
a9c514aaf7 Textless query result endpoint (#3311)
* add an endpoint for running a query by its id and (optional) parameters
without having to provide the query text

* check for access to query before running it
2019-01-23 11:10:04 +02:00
Miles Maddox
ff6b20b69c support for fetching all JQL results by way of pagination (#3304) 2019-01-22 15:52:13 +02:00
YOSHIDA Katsuhiko
c4bf44677a Fix an error of exporting dict value as Excel (#3323) 2019-01-22 14:43:52 +02:00
YOSHIDA Katsuhiko
b3643ffbb7 Add regenerate function of user's API key (#3224)
* Add regenerate function of user's API Key

* Update client/app/pages/users/show.js

Co-Authored-By: kyoshidajp <claddvd@gmail.com>

* Remove unused error message

* Refactoring: Inline temp

* Update client/app/pages/users/show.js

Co-Authored-By: kyoshidajp <claddvd@gmail.com>

* Change action event of regenerate user API key
2019-01-20 13:38:20 +02:00
Eric Chang
b91d4bdcaf override default integer/float formatting with environment variables (#3307) 2019-01-20 10:10:36 +02:00
Arik Fraimovich
e8120c5f79 Use None as "not scheduled" default value of a query (#3277)
* Use null as the default scheduled value.

* Don't serialize None to json, so we can use SQL is not null predicate.

* Fix warning about unicode in tests

* Handling empty query.schedule in UI (#3283)

* Add migration to convert empty schedules to null and drop the not null contraint.
2019-01-18 11:30:45 +02:00
Omer Lachish
7847cf7d63 Fix invitation pending for older invitations (#3298)
* explicitly look for a False under details['is_invitation_pending'] and
not any falsey result, to avoid locking out invitations which were
created before the Pending Invitation feature was introduced. Solves https://github.com/getredash/redash/issues/3297

* test that old invites (that do not have any is_invitation_pending flag set in their details object) are still acceptable
2019-01-17 11:56:16 +02:00
Omer Lachish
121a44ef15 Remove tree validations and introduce ParameterizedQuery (#3230) 2019-01-17 10:26:00 +02:00
koooge
4c9326a9da handlers: Fix post users (#3273) 2019-01-13 15:52:56 +02:00
Omer Lachish
26252be75a Verify admin email address (#3267)
* add an  bit

* prompt on homepage when user's email hasn't been verified

* set e-mail as verified for new setups and invited users

* 👋 copy & paste invite links, it's time for verified e-mails!

* default `is_invitation_pending` to false and actively set it to true
when inviting users, so that existing users won't show "Invitation
Pending"

* fix tests that broke due to default is_invitation_pending value

* treat admin's e-mail address as verified

* add verification endpoint

* send verification e-mail

* Update client/app/components/empty-state/empty-state.html

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/authentication/account.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/handlers/authentication.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/templates/emails/verify.html

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/authentication/account.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/templates/verify.html

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/templates/emails/verify.txt

Co-Authored-By: rauchy <omer@rauchy.net>

* add link in case redirects are disabled

* POSTing to /email_verification makes more sense than GETting /send_verification

* avoid sending invitations when no_invite is passed along

* Update client/app/pages/users/new.html

Co-Authored-By: rauchy <omer@rauchy.net>

* move e-mail verification prompt to home-page

* get rid of redundant $scope

* return JSON

* flip is_email_verified's default value so that existing users do not
show as not-verified

* e-mail verification propmt isn't dangerous, it just wants to warn you
2019-01-13 13:57:12 +02:00
Ilya Ruzakov
0b6f1fc21b [Data Sources] Implement Apache Drill (#3188)
* Added support for Apache Drill datasource

* Improvements in `Drill` query runner and minor refactoring

1. Drill query runner now inherits from `BaseHTTPQueryRunner`, because they both have a lot of common code.
2. `BaseHTTPQueryRunner.get_response` method now accepts `http_method` argument (original implementation was only capable of sending `GET` HTTP requests).
3. Added `order` to `BaseHTTPRequestRunner` configuration schema to fix order of UI elements based on the schema.
4. Eliminated duplicate method `_guess_type` in `GoogleSpreadsheet`, `Results` and `Drill` query runners, moved `guess_type` to `redash.query_runner`.
5. Removed tests for `_guess_type` in `GoogleSpreadsheet`, `Results` and `Drill` query runners, merged them into single test case and moved to `tests.query_runner.test_utils`.
6. Various minor changes (code style, imports, etc).
2019-01-10 09:12:35 +02:00