Commit Graph

398 Commits

Author SHA1 Message Date
Jannis Leidel
21082fbe0e Make the "celery" queue the default instead of "schema" to stay backward-compatible. (#3534)
Fix #3325.
2019-03-06 08:37:54 +02:00
Gabriel Dutra
6a75ac4a57 Migrate User Pages to React (#3506)
* Create React version for the EmailSettingsWarning

* Migrate the Create User Page

* Migrate UserProfile to React

* Add /users/me to the routes (Percy ftw)

* Fix UserShow test spec

* Remove Error Messages component

* Show invitation link if email server not setup (#3519)

* return invite link to client if e-mail server is not set up

* add a couple of tests to make sure invite links are only returned when neccessary

* show invite link when e-mail is not configured

* remove "an e-mail has been sent" when there's no e-mail configured

* return invite_url in re-invites as well. Also refactor to reuse the code.

* Use CreateUserDialog instead of Page

* Render invite link on Resend Invitation click

* Add email validation to DynamicForm

* Fix EmailWarning position + update user list with user creation success

* Fix console error on UserProfile

* Redirect from /users/new  + rename createUser -> showCreateUserDialog

* Use alert instead of toastr for user creation errors

* Remove logic from CreateUserDialog

* CR

* Use Promise.reject instead of throw to avoid console error
2019-03-04 18:26:51 -03:00
Omer Lachish
e9c88ea176 Verify address when users change their e-mail (#3504)
* re-verify e-mail address on change

* send verification e-mail to the new address
2019-02-27 12:17:20 +02:00
Omer Lachish
0d76c036cb Be more permissive when parameters are safe (#3383)
* use the textless endpoint (/api/queries/:id/results) for pristine
queriest

* reverse conditional. not not is making me the headaches.

* add ParameterizedQuery#is_safe with an inital naive implementation which
treats any query with a text parameter as not safe. This will be
remedied later when DB drivers will handle these parameters.

* allow getting new query results even if user has only view permissions
to the data source (given that the query is safe)

* fix lint error - getDerivedStateFromProps should be placed after state

* Revert "use the textless endpoint (/api/queries/:id/results) for pristine"

This reverts commit cd2cee7738.

* move execution preparation to a different function, which will be soon
reused

* go to textless /api/queries/:id/results by default

* let the query view decide if text or textless endpoint is needed

* allow safe queries to be executed in the UI even if the user has no
permission to execute and create new query results

* change `run_query`'s signature to accept a ParameterizedQuery instead of
constructing it inside

* use dict#get instead of a None guard

* use ParameterizedQuery in queries handler as well

* test that /queries/:id/results allows execution of safe queries even if
user has view_only permissions

* lint

* raise HTTP 400 when receiving invalid parameter values. Fixes #3394

* remove unused methods

* avoid cyclic imports by importing only when needed

* verify that a ParameterizedQuery without any parameters is considered
safe

* introduce query.parameter_schema

* encapsulate ParameterizedQuery creation inside Query
2019-02-26 20:55:01 +02:00
Arik Fraimovich
77dcc80522 Fix: ParameterizedQuery: support for upper cases column names. (#3461)
* Fix: ParameterizedQuery: support for upper cases column names.

* Fix test name
2019-02-20 12:08:38 +02:00
Omer Lachish
0bca2d8920 when working with a schema, fail in the case that there are parameters (#3452)
which are not part of the schema
2019-02-19 20:46:30 +02:00
Arik Fraimovich
58a53e3470 Fix: remove widgets when deleting a visualization. (#3423)
Closes #3257.
2019-02-17 10:30:23 +02:00
Omer Lachish
03f040da0e Unify query based dropdown population (#3337)
* stop testing `collect_query_parameters`, it's an implementation detail

* add tests for `missing_query_params`

* rename SQLQuery -> ParameterizedSqlQuery

* rename sql_query.py to parameterized_query.py

* split to parameterized queries and parameterized SQL queries, where
parameterized queries only do templating and parameterized SQL queries
add tree validation on top of it

* move missing parameter detection to ParameterizedQuery

* get rid of some old code

* fix tests

* set syntax to `custom`

* revert the max-age-related refactoring

* 👋 tree validations 😢

* BaseQueryRunner is no longer a factory for ParameterizedQuery, for now

* add an endpoint for running a query by its id and (optional) parameters
without having to provide the query text

* adds parameter schema to ParameterizedQuery

* adds parameter schema validation (currently for strings)

* validate number parameters

* validate date parameters

* validate parameters on POST /api/queries/<id>/results

* validate enum parameters

* validate date range parameters

* validate query-based dropdowns by preprocessing them at the handler
level and converting them to a populated enum

* change _is_date_range to be a tad more succinct

* a single assignment with a `map` is sufficiently explanatory

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/handlers/query_results.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* build error message inside the error

* support all types of numbers as number parameters

* check for permissions when populating query-based dropdowns

* check for access to query before running it

* check for empty rows when populating query-based enums

* don't bother loading query results if user doesn't have access

* 💥 on unexpected parameter types

* parameter schema default is a list, not a dictionary

* fix a totally unrelated typo

* remove redundant null guards

* introduce /dropdown.json endpoint with dummy data

* wire frontend to /dropdown.json

* always return name/value combos from /dropdown.json

* load actual data into /dropdown.json

* pluck correct values for `name` and `value`

* reuse dropdwon plucking logic in QueryResultResource

* simplify _get_dropdown_values

* when doing parameter validation, we only care about the value and not
the display name

* rename dropdown to dropdownOptions

* move dropdown_values to utils/parameterized_query.py

* stop converting queries to enums and encapsulate the work inside
ParameterizedQuery (almost - /dropdown.json would still access the
dropdown_values method)

* re-order arguments by importance

* test query parameter validation

* tests for dropdown_values logic

* remove `.json` suffix to the dropdown endpoint

* allow `BaseResource` to handle JSON stuff

* move _pluck_name_and_value outside its containing method

* case-insensitive lookup when plucking name and value

* separate concerns and simplify test isolation for `dropdown_values`

* pick the default column according to the order specified in the query
result columns attribute

* use `current_org` instead of passing `org`

* test that user has access to the query when calling the /dropdown
endpoint
2019-02-10 13:10:39 +02:00
Levko Kravets
11507c5e5e Show active and pending users separately (for admins) (#3400) 2019-02-07 20:30:55 +02:00
Jannis Leidel
69e34f048a Add archived queries section to queries list. (#2888)
* Add archived queries section to queries list.

* Refactor route building for list based controllers.

This also fixes the dashboard empty state page.
2019-02-03 14:35:25 +02:00
Omer Lachish
371b319e92 Server-side parameter validation (#3315)
* stop testing `collect_query_parameters`, it's an implementation detail

* add tests for `missing_query_params`

* rename SQLQuery -> ParameterizedSqlQuery

* rename sql_query.py to parameterized_query.py

* split to parameterized queries and parameterized SQL queries, where
parameterized queries only do templating and parameterized SQL queries
add tree validation on top of it

* move missing parameter detection to ParameterizedQuery

* get rid of some old code

* fix tests

* set syntax to `custom`

* revert the max-age-related refactoring

* 👋 tree validations 😢

* BaseQueryRunner is no longer a factory for ParameterizedQuery, for now

* add an endpoint for running a query by its id and (optional) parameters
without having to provide the query text

* adds parameter schema to ParameterizedQuery

* adds parameter schema validation (currently for strings)

* validate number parameters

* validate date parameters

* validate parameters on POST /api/queries/<id>/results

* validate enum parameters

* validate date range parameters

* validate query-based dropdowns by preprocessing them at the handler
level and converting them to a populated enum

* change _is_date_range to be a tad more succinct

* a single assignment with a `map` is sufficiently explanatory

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/handlers/query_results.py

Co-Authored-By: rauchy <omer@rauchy.net>

* Update redash/utils/parameterized_query.py

Co-Authored-By: rauchy <omer@rauchy.net>

* build error message inside the error

* support all types of numbers as number parameters

* check for permissions when populating query-based dropdowns

* check for access to query before running it

* check for empty rows when populating query-based enums

* don't bother loading query results if user doesn't have access

* 💥 on unexpected parameter types

* parameter schema default is a list, not a dictionary

* remove redundant null guards
2019-01-29 09:18:07 +02:00
Omer Lachish
a9c514aaf7 Textless query result endpoint (#3311)
* add an endpoint for running a query by its id and (optional) parameters
without having to provide the query text

* check for access to query before running it
2019-01-23 11:10:04 +02:00
YOSHIDA Katsuhiko
b3643ffbb7 Add regenerate function of user's API key (#3224)
* Add regenerate function of user's API Key

* Update client/app/pages/users/show.js

Co-Authored-By: kyoshidajp <claddvd@gmail.com>

* Remove unused error message

* Refactoring: Inline temp

* Update client/app/pages/users/show.js

Co-Authored-By: kyoshidajp <claddvd@gmail.com>

* Change action event of regenerate user API key
2019-01-20 13:38:20 +02:00
Arik Fraimovich
e8120c5f79 Use None as "not scheduled" default value of a query (#3277)
* Use null as the default scheduled value.

* Don't serialize None to json, so we can use SQL is not null predicate.

* Fix warning about unicode in tests

* Handling empty query.schedule in UI (#3283)

* Add migration to convert empty schedules to null and drop the not null contraint.
2019-01-18 11:30:45 +02:00
Omer Lachish
7847cf7d63 Fix invitation pending for older invitations (#3298)
* explicitly look for a False under details['is_invitation_pending'] and
not any falsey result, to avoid locking out invitations which were
created before the Pending Invitation feature was introduced. Solves https://github.com/getredash/redash/issues/3297

* test that old invites (that do not have any is_invitation_pending flag set in their details object) are still acceptable
2019-01-17 11:56:16 +02:00
Omer Lachish
121a44ef15 Remove tree validations and introduce ParameterizedQuery (#3230) 2019-01-17 10:26:00 +02:00
koooge
4c9326a9da handlers: Fix post users (#3273) 2019-01-13 15:52:56 +02:00
Ilya Ruzakov
0b6f1fc21b [Data Sources] Implement Apache Drill (#3188)
* Added support for Apache Drill datasource

* Improvements in `Drill` query runner and minor refactoring

1. Drill query runner now inherits from `BaseHTTPQueryRunner`, because they both have a lot of common code.
2. `BaseHTTPQueryRunner.get_response` method now accepts `http_method` argument (original implementation was only capable of sending `GET` HTTP requests).
3. Added `order` to `BaseHTTPRequestRunner` configuration schema to fix order of UI elements based on the schema.
4. Eliminated duplicate method `_guess_type` in `GoogleSpreadsheet`, `Results` and `Drill` query runners, moved `guess_type` to `redash.query_runner`.
5. Removed tests for `_guess_type` in `GoogleSpreadsheet`, `Results` and `Drill` query runners, merged them into single test case and moved to `tests.query_runner.test_utils`.
6. Various minor changes (code style, imports, etc).
2019-01-10 09:12:35 +02:00
Omer Lachish
445f8e5c36 Fix invitation pending for existing users (#3261)
* default `is_invitation_pending` to false and actively set it to true
when inviting users, so that existing users won't show "Invitation
Pending"

* fix tests that broke due to default is_invitation_pending value

* update Flask-OAuthLib
2019-01-09 13:48:47 +02:00
Arik Fraimovich
08953cc919 Redis based implementation of user active_at timestamp update (#3256)
* Switch to simpler implementation
* Fix active_at update code
* Fix sync test
2019-01-08 14:03:49 +02:00
Omer Lachish
823f172a9f Invitation Pending changes (#3229)
* determine invitation_pending according to empty password. This commit will be reverted, I'm just deferring the implementation

* show '(Invitation Pending)' to users who haven't accepted their invitation yet

* allow resending invitations

* allow deletion of pending users from user list

* set invitation as not pending when following invite link

* prevent deleting activated users

* test that users who follow invitation links are set as non-pending invitations

* prevent re-using invitations

* invitees who use SSO will now also be marked as "non-pending"

* lint
2019-01-08 08:52:48 +02:00
Jannis Leidel
44dff83046 Add "Active at" column to user list. (#3026)
* add last_active_at to users page

* Use our JSON encoder as the SQLAlchemy JSON serializer.

* Fixed some inconsistencies in the user query class methods.

* Minor cosmetic fixes.

* Add some make tasks for easier development.

* Add user detail sync system based on Redis backend.

There is a periodic Celery task that updates a new “details” JSONB column in the “user” table with the data from Redis.

Currently this is only used for tracking the date of last activity of a user but can be extended with other user information later.

Updates a few dependencies.

* Normalize a few Flask extension API names.

* Reduce implementation complexity of JSONEncoder.

* Use request_started signal to make sure we have a request context.

Otherwise loading the user based on the request won’t work.

* Fix test that checks if disabled users can login.

This correctly uses a URL path that includes the current organization and checks for the error message.

The previous test seems to have been a red herring.

* Minor cosmetic fixes.

* Remove needs_sync in favor of just deleting things.

* Misc review fixes.

* Ignore line length.

* Split redash.models import several modules.

* Move walrus UTC DateTimeField into redash.models.types.

* Restore distinctly loading dashboards.

* Simplify default values for user details.

* Define __repr__ methods generically.

* Consistently have underscore methods at the top of model methods.

* Fix tests.

* Split redash.models import several modules.

* Update to latest walrus and redis-py.

* Update kombu to 4.2.2 for redis-py 3.x compatibility.

* Remove redis-cli container after running Make task.

* Move buffer condition after datetime/time conditions.

* Update walrus to 0.7.1.

* Refactor some query APIs.

This uses the flask-sqlalchemy helpers consistently and makes more use of mixins.

* Post rebase fixes.

* Use correct kombu version

* Fix migration down revision
2019-01-07 10:30:42 +02:00
Marina Samuel
cdd2259d08 Closes #2396: Add finer-grained scheduling. (#2426)
* Closes #187: Add finer-grained scheduling - backend.

* Closes #2396 - Add finer-grained scheduling - frontend.

* Fix linting issues

* Rename ScheduleDialgo to .jsx
2019-01-06 10:59:50 +02:00
Omer Lachish
d7edaa3ba2 Tests for find_missing_params (#3225)
* stop testing `collect_query_parameters`, it's an implementation detail

* add tests for `missing_query_params`
2018-12-31 12:34:57 +02:00
Arik Fraimovich
f62d0e1300 Use lower cased names for groups: (#3221)
Otherwise sorting depends on the Postgres collation and causes tests
to fail on different envoirnments.
2018-12-26 17:16:13 +02:00
Takuya Arita
8ed872756c Add test case for redash.utils.generate_token (#3211) 2018-12-23 15:13:00 +02:00
Arik Fraimovich
83ea472d37 Merge pull request #3093 from getredash/backend-parameter-templating
Remove Mustache templating from frontend
2018-12-20 22:24:11 +02:00
Omer Lachish
479247b60c Merge branch 'master' into backend-parameter-templating 2018-12-16 10:22:29 +02:00
Omer Lachish
94905a287a tests for legacy session user identifiers 2018-12-12 13:03:50 +02:00
Omer Lachish
2312db46f2 test that other sessions are invalidated when changing an e-mail. I had
to resort to comments in code in order to explain this. I'm a failure
today. 😭
2018-12-11 15:14:43 +02:00
Omer Lachish
b3495b8c00 test that user does not get logged out when changing email or password 2018-12-11 12:25:51 +02:00
Omer Lachish
dec790a9f3 Merge branch 'master' into invalidate-sessions-after-email-or-password-change 2018-12-11 10:41:27 +02:00
Omer Lachish
944bee6101 update identity only after succesfully updating user information 2018-12-10 14:25:04 +02:00
Omer Lachish
38ed046c9f Fix disable error message (#3175)
* display correct error message when attempting to disable yourself
* 403 (Forbidden) feels like a better status code than 400 (Bad Request)
* fix broken test
* remove redundant error title
2018-12-10 14:02:50 +02:00
Omer Lachish
c426c826f7 fix tests that rely on sessions 2018-12-10 12:27:39 +02:00
Omer Lachish
76321937d7 Remove API permissions for users who have been disabled (#3162) 2018-12-05 12:21:18 +02:00
Arik Fraimovich
c9ca2b99f6 Fix: Alert.evaluate failing when the column is missing. (#3167) 2018-12-05 11:28:05 +02:00
Omer Lachish
d769afab6f Merge branch 'master' into backend-parameter-templating 2018-12-03 13:22:44 +02:00
Omer Lachish
cf86509a0b remove leftovers of SQLQuery from utils 2018-12-03 13:17:40 +02:00
Arik Fraimovich
9c12b04578 json_dumps: add support for serializing buffer objects. (#3156) 2018-12-03 10:57:36 +02:00
Omer Lachish
0a62bee3a1 Merge branch 'master' into backend-parameter-templating 2018-12-03 09:44:52 +02:00
Omer Lachish
9579f12a83 Protect against SQL injections by using tree comparisons (#3109)
* add SQLQuery class with tests for safe queries and non-safe tautology attacks

* add test for union query injections

* split .apply calls to newline

* add tests for comment attacks

* remove double underscore

* extract complex children check to variable

* inherit from object because I'm not a lamer

Co-Authored-By: rauchy <omer@rauchy.net>

* simplify cognitive complexity

* check that additional columns are not injected

* detect appended queries

* inline .apply calls

* move SQLQuery to it's own module

* move SQLQuery tests to their own module

* serialize SQLQuery instances

* raise an exception when attempting to serialize an unsafe query

* queries without parameters are safe

* remove redundant parentheses

* use cached properties

* rename SQLInjectionException to SQLInjectionError

* support multiple word params and param negations

* refactor out methods that don't involve any state

* don't cache text()

* reduce cognitive complexity
2018-12-02 21:51:06 +02:00
Jannis Leidel
a16170e701 Fix tag counts for dashboards and queries. (#3120)
* Fix tag counts for dashboards to be distinct.

This also makes use of the Dashboard.all base query.

Fix #3108.

* Use Query.all_queries as the base query for Query.all_tags.

* Add test case for Dashboard.all_tags.
2018-11-28 14:06:15 +02:00
Arik Fraimovich
bd20ce12ac Don't allow updating user's email to blacklisted domain. (#3127) 2018-11-26 21:22:14 +02:00
Udomomo
8257d9d037 Add permissions to the result of "manage.py groups list" command (#3007)
* Add permissions to the result of "groups list" command

* added permissions to test case

* removed setting for debug
2018-11-25 13:47:25 +02:00
Omer Lachish
ef8839aafa add tests for comment attacks 2018-11-21 13:52:08 +02:00
Omer Lachish
14860f6a8b split .apply calls to newline 2018-11-21 13:51:19 +02:00
Omer Lachish
a52c783857 add test for union query injections 2018-11-21 13:45:52 +02:00
Omer Lachish
5e7c785891 add SQLQuery class with tests for safe queries and non-safe tautology attacks 2018-11-21 13:45:29 +02:00
Omer Lachish
b242cefaa0 Merge branch 'master' into backend-parameter-templating 2018-11-21 10:51:44 +02:00