* Enable graceful shutdown of rq workers
* Use `exec` in the `worker` command of the entrypoint to propagate
the `TERM` signal
* Allow rq processes managed by supervisor to exit without restart on
expected status codes
* Allow supervisorctl to contact the running supervisor
* Add a `shutdown_worker` command that will send `TERM` to all running
worker processes and then sleep. This allows orchestration systems
to initiate a graceful shutdown before sending `SIGTERM` to
supervisord
* Use Heroku worker as the BaseWorker
This implements a graceful shutdown on SIGTERM, which simplifies
external shutdown procedures.
* Fix imports based upon review
* Remove supervisorctl config
* Change front-end and data model for SAML2 auth - static configuration
* Add changes to use inline metadata.
* add switch for static and dynamic SAML configurations
* Fixed config of backend static/dynamic to match UI
* add ability to encrypt/decrypt SAML assertions with pem and crt files. Upgraded to pysaml2 6.1.0 to mitigate signature mismatch during decryption
* remove print debug statement
* Use utility to find xmlsec binary for encryption, formatting saml_auth module
* format SAML Javascript, revert want_signed_response to pre-PR value
* pysaml2's entityid should point to the sp, not the idp
* add logging for entityid for validation
* use mustache_render instead of string formatting. put all static logic into static branch
* move mustache template for inline saml metadata to the global level
* Incorporate SAML type with Enabled setting
* Update client/app/pages/settings/components/AuthSettings/SAMLSettings.jsx
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Chad Chen <chad.chen@databricks.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* added bar chart boilerplate
* added x/y manipulation
* replaced x/y management to inner series preparer
* added tests
* moved axis inversion to all charts series
* removed line and area
* inverted labels ui
* removed normalizer check, simplified inverted axes check
* finished working hbar
* minor review
* added conditional title to YAxis
* generalized horizontal chart for line charts, resetted state on globalSeriesType change
* fixed updates
* fixed updates to layout
* fixed minor issues
* removed right Y axis when axes inverted
* ran prettier
* fixed updater function conflict and misuse of getOptions
* renamed inverted to swapped
* created mappingtypes for swapped columns
* removed unused import
* minor polishing
* improved series behaviour in h-bar
* minor fix
* added basic filter to ChartTypeSelect
* final setup of filtered chart types
* Update viz-lib/src/components/visualizations/editor/createTabbedEditor.jsx
* added proptypes and renamed ChartTypeSelect props
* Add missing import
* fixed import, moved result array to global scope
* merged import
* clearer naming in ChartTypeSelect
* better lodash map syntax
* fixed global modification
* moved result inside useMemo
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
* Extra actions for Query View and Query Source pages
* Convert Queries List page to functional component
* Convert Dashboards List page to functional component
* Extra actions for Query List page
* Extra actions for Dashboard List page
* Extra actions for Dashboard page
* Pass some extra data to Dashboard.HeaderExtra component
* CR1
* Test Cypress on package list
* Skip Puppeteer Chromium as well
* Put back missing npm install on netlify.toml
* Netlify: move env vars to build.environment
* Remove cypress:install script
* Update Cypress dockerfile
* Copy package-lock.json to Cypress dockerfile
* Directly map query results column to GeoJSON property
* Use cache for geoJson requests
* Don't handle bounds changes while loading geoJson data
* Choropleth: fix map "jumping" on load; don't save bounds if user didn't edit them; refine code a bit
* Improve cache
* Optimize Japan Perfectures map (remove irrelevant GeoJson properties)
* Improve getOptions for Choropleth; remove unused code
* Fix test
* Add US states map
* Convert USA map to Albers projection
* Allow to specify user-friendly field names for maps
* Ask user to log in when session expires
* Update implementation
* Update implementation
* Minor fix
* Update modal
* Do not intercept calls to api/session as Auth.requireSession() relies on it
* Refine code; adjust popup size and position
* expire CSRF tokens after 6 hours
* use axios' built-in cookie to header copy mechanism
* add axios-auth-refresh
* retry CSRF-related 400 errors by refreshing the cookie
* export the auth refresh interceptor to support ejecting it if neccessary
* reject the original request if it's unrelated to CSRF
* Convert TagsList to functional component
* Convert TagsList to typescript
* Allow to unselect all tags
* Add title to Tags block and explicit "clear filter" button
* Some tweaks
* add default limit 1000
* Add frontend changes and connect to backend
* Fix query hash because of default limit
* fix CircleCI test
* adjust for comment
* Refactor CardsList - pass a suffix for list item
Adding :id to an item to be used as a key suffix is redundant and the same
can be accomplished by using :index from the map function.
* Move CardsList to typescript
* Convert CardsList component to functional component
* CR1
* CR2
* Support multiple queries in a single query box
* Implement statement splitting function and add tests for it
* Add a test for databricks-specific syntax
* Split statements before running query
* Introduce Link component
* Use Link component for external links as well
* Remove unused file (I hope it's really not needed)
* Use Link component in visualizations library
* Simplify Link component implementation
* CR1
* Trigger build
* CR2
* Set corejs version in .babelrc so Jest doesn't complain.
* Rewrite services/routes in TypeScript.
* Add TypeScript definitions for DialogComponent.
* Make image paths more portable
* Add current route context and hook.
* Make EmptyState more flexible by being able to pass in getSteps function.
* Rewrite ItemsList in TypeScript.
* Introduce the possibility to add custom sorters for a column.
* Rearrange props to be friendly to TypeScript.
* Type definitions for NotificationApi.
* Use Databricks query editor components for databricks_internal type of query runner.
* URL Escape password in Alembic configuration.
* Compare types in migrations.
* allow non-sequential IDs for DataSources in Cypress tests
* refactor redash-api to a set of Cypress commands
* support mounting Redash endpoints in Cypress routes
* fix some parameter specs by waiting for schema to load
* extract baseUrl from cypress.json
* Restyled by prettier (#5110)
Co-authored-by: Restyled.io <commits@restyled.io>
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <commits@restyled.io>
* add lock table header
* Move styling to a new class
* Update renderer.less
* Move class to table and fix top border
* Update renderer.less
* Update viz-lib/src/visualizations/table/renderer.less
Thanks, this change is good to me.
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* add flask-wtf
* add CSRF tokens to all static forms
* add CSRF tokens to all axios requests
* disable CSRF validation in unit tests
* support CSRF-protected requests in *most* cypress tests
* don't enfroce CSRF checks by default
* avoid CSRF enforcement in unit tests
* remove redundant spread
* some camel casing hiccups
* always yield the CSRF cookie, but avoid enforcing it if CSRF toggle is off
* Restyled by prettier (#5056)
Co-authored-by: Restyled.io <commits@restyled.io>
* set a CSRF header only if cookie is present
* enforce CSRF in CI
* install lodash directly for Cypress
* install request-cookies directly for Cypress. We should probably start loading package.json deps
* enable CSRF support when logout and login happen within the same spec
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <commits@restyled.io>
When using some of the customized login flows such as `REMOTE_USER` the deployed site breaks due to not finding template files. This change updated the app default to use the existing Flask templates directory rather than the compiled static assets directory which only contains an index.html file.
* Add refresh button in the bottom
* Add caching
* Drop allSettled
* Simplify refresh button
* Update error to return 500
* Load tables before loading columns
* Don't mutate schema
* Reset db name and schemas when changing data source
* Load both tables and columns
* Return error with code 200
* Code review updates
* Add expiration time to the cache Keys
* Back with RQ
* getredash/redash#5026 Fix wrong Y-axis range for stacked bar chart
* Update tests
* Use Plotly's built-in algorinthm to compute Y-axis range
* Update tests
* Revert previous solution (yRange-related code)
* Revert other unrelated changes
* Revert other unrelated changes
* Move chart rendering to own file and ensure that rendering steps will occur in necessary order
* Reduce amount of plot updates by mergin separate updates into a sigle cumulative update
* Give better names for several functions
* TASK Add typescript dependencies to package.json
* TASK Add typescript to build process and npm scripts and TASK Move example components to typescript and add an example definition file.
* TASK Move back to ts-loader instead of babel typescript preset
* FIX Remove unnecessary changes
* FIX Explicitly mention tsconfig file in webpack.config.js to avoid `error while parsing tsconfig.json, The 'files' list in config file 'tsconfig.json' is empty`
See (https://github.com/TypeStrong/ts-loader/issues/405#issuecomment-330108362)
* FIX Move tsconfig to client subdirectory to make it accessible in docker container (only webpack.config.js is copied over from root folder in Dockerfile)
* TASK Move from ts-loader to babel to reduce compatibility issues between ES6/7 and typescript compilation.
* TASK Add types for classnames, hoist-non-react-statics and lodash. Fix default export of DashboardList and run prettier on eslintrc
* Run npm install
* Trigger tests
* Run npm install 2
* Trigger tests
* on dashboard api calls - take the id from the beginning of the slug, unless there is no number in it - in that case, take the entire slug as id
* add dashboard id when showing links to dashboards
* change path to include new name when renaming dashboards
* move slug generation to backend
* redirect to new name after changing (this time with a proper promise)
* oh right, we already have a slug function
* add spec that makes sure that renamed dashboards are redirected to the
url which contains their new name
* use id-slug in all Cypress specs
* move dashboards from /dashboard/:slug to /dashboards/:id-:name_as_slug
* Update dashboard url as its name changes
* Update separator to be "/"
* Update missing dashboard urls
* Update api not to depend on int id
* Use '-' instead of '/' as separator and update Dashboard.get calls
* slug -> name_as_slug
* Keep slug urls on cypress
* Update route path
* Use legacy attr for GET
* Use getter for urlForDashboard
* Update dashboard url when loaded by slug
* Update Dashboard routes to use id instead of slug
* Update Dashboard handler tests
* Update Cypress tests
* Fix create new dashboard spec
* Use axios { params }
* Drop Ternary operator
* Send updated slug directly in 'slug' attr
* Update multiple urls Dashboard test name
* Update route names
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
* allow overriding the type of key used for primary/foreign keys of the different models
* rename key_types to singular key_type
* add some documentation for `database_key_definitions`
* Textbox: confirm close if text was changed
* Update texting (with @gabrieldutra)
* Update texting
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* Move components specific to UserProfile page to corresponding folder
* Split UserProfile page into components
* Rename components, refine code a bit
* Add some extension points
* Fix margin
* Split OrganizationSettings page into components
* Update change handling: use objects instead of string keys, move some logic to more appropriate place
* Convert OrganizationSettings page to functional component and refine code a bit
* Add some extension points
* Improve onChange handler
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* Fix CLI command for "status"
CLI command "status" can fail due to incorrect connection information to RQ.
This change matches the behavior from line 65 and solves the connection error.
* Move connection up to CLI entrypoint
* ErrorMessage is not centered
* Adjust ErrorMessage size on large screens
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* Vertical navbar
* Update vertical menu look and add create menu.
* Make query editor work with vertical nav.
* Dark mode
* Fix create menu & make sidebar fixed.
* Update Alert pages layout
* Update System status pages
* Update Queries and Dashboards list pages
* Update Query Source and Query View pages
* Use dark theme for mobile navbar
* Update Dashboard page: fix Add widget/textbox panel positioning
* Dashboard page: fix layout issues when container changes its size (fixes known issues: navbar expand/collapse, scrollbar appears/hides)
* Fix dashboard page sticky header (there was a 15px space above it)
* Fix embeds
* Extract desktop navbar component; move mobile navbar and its styles to ApplicationLayout folder
* Remove old app header
* Fix tests
* Restore version info block
* Make Percy capture entire page
* Make vertical navbar expand/collapse animation smoother (as it's currently impossible to disable it :-( )
* Fix misc UI issues (show Create label on expanded menu; fix some CSS; don't select items on click)
* Allow to override navbars with DynamicComponent
* Fix misc UI issues: expand/collapse button animation, menu items styles, menu expand/collapse animation
* Hide submenu arrow; show username when menu is expanded
* Refine CSS and make it more isolated; adjust colors
* Update tests
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* V9 Changelog: Initial Draft from Jesse
* V9 Changelog: Add later updates
* Adjust title spacing
* Apply Jesse's suggestions
Co-authored-by: Jesse <jesse@whitehouse.dev>
* provide an explanation on how to switch from Celery to RQ when upgrading to v9
* Update CHANGELOG.md
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* Add contributor names
* Update version.
* Update CHANGELOG
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Jesse <jesse@whitehouse.dev>
Co-authored-by: Omer Lachish <omer@rauchy.net>
* Plotly Charts: use .bg to determine legends size
* Test: remove hack for legend below plotly
* Revert "Test: remove hack for legend below plotly"
This reverts commit d8efb0c032.
* Use .legend to calculate bounds
* Also update plots without legend
* Sankey: Make sure last stage has "Exit"
* Sankey: Use 2 as min stage width size to render
* Use null instead of "Exit"
* Add comment about corresponding exit node
* Add multiple stages on Cypress test
* don't join underlying exception message with commas when invalid parameter errors happen
* Revert "don't join underlying exception message with commas when invalid parameter errors happen"
This reverts commit 71a21b7ce6.
* when a problem occurs during refresh_queries, report it as a RefreshQueriesError
* Fix: table viz crashing when search is enabled
* Replace that weird hack with more controlled code
* Don't clear search input, apply search when data changes
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
* Add build arg to Dockerfile to control if we should build frontend assets
* Move more env settings into the shared one.
* Use build arg in docker-compose to skip frontend build.
* CirlceCI: Skip building frontend assets in backend tests
* Create dummy template files
* Expand file names manually.
* Add build arg to skip dev dependencies.
* Update Dockerfile
* Reverse logic of skip_dev_deps to what it should be.
* run queries through adhoc SSH tunnels
* reduce indent by losing try/else clause
* document host/port getters and setters
* handle forceful schema refreshes in RQ and poll for their results using the /jobs endpoint
* set schema refresh timeout to 5 minutes
* Restyled by prettier (#4847)
Co-authored-by: Restyled.io <commits@restyled.io>
* send schema refresh errors as part of API response
* Use correct get_schema call.
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <commits@restyled.io>
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* Add option to explicitly set chart legend position
* Revert some chanes in order to fix tests
* Leave only "auto" and "below the plot" legend placement options
* Move Show legend checkbox to select; fix spelling
* Add visualizations project settings
* Move visualizations to redash-visualizations
* Delete shared components
* Remove antd from deps
* Remove p-r-5 from table utils
* Remove visualization deps from package.json
* Rename package and change its version
* Test preinstall script
* Update Dockerfile build for frontend
* Test adding dockerignore
* Update jest tests
* Add step for jest tests
* Include viz-lib on dev commands
* User prettier v1 for now
* Delete unused libs on the app
* Add readme draft (to be finished)
* Add getOptions to Editor
* Add required libraries and finish basic example
* Bump version
* A new intro paragraph to explain what Redash is
We have been using Redash at Databricks and really love it. I took the time to work with Arik to create a better, more up-to-date description of the project.
* Add data sources
* Fix: showing current settings tab broken in MULTI_ORG.
* Revert "Fix: showing current settings tab broken in MULTI_ORG."
This reverts commit a88defd0b5.
* Add test for SettingsMenu#isActive
* Use stripBase to remove slug from url
* ODBC Based Databricks connector.
* Install Databricks' ODBC driver in Docker image
* Add useragent string.
* Add Types enum to redash.query_runner to replace the seprate constants.
* Databricks connector:
1. Parse types.
2. Send additional connection options.
3. Correctly parse errors.
* Switch to TYPE constants to use code with Python 2.
* Add note about the Databricks driver terms and conditions.
* Show message about Databricks driver terms and conditions.
* Handle cases when the query doesn't return any results.
* Update redash/query_runner/databricks.py
Co-Authored-By: Jesse <jesse@whitehouse.dev>
* Use new Databricks logo
* Fix connection string options
Co-authored-by: Jesse <jesse@whitehouse.dev>
* Move Dashboard off `subqueryload()` loader in all() method due to inconsistent results bug in SQLAlchemy when leveraging distinct within a subqueryload call through paginate.
* Added source reference to Presto Query Runner connection through the pyhive client to announce to presto that the query is coming from `redash` instead of `pyhive`.
* Removing source line from presto query runner to refactor based on feedback.
Let the notifications go into browser/OS notification trays so users can click on them from there if they miss the initial notification. Modern browsers generally use OS notifications so the user is in control of the notification at the OS level.
* Fix comparison error when scale is None
Prevents `'>' not supported between instances of 'NoneType' and 'int'` error when scale is `None`
* Update oracle.py
* Fix scale logic.
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* add pyexasol datasource, ensure that integer dont overflow in javascript
* support setting encryption for Exasol connections
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* feature: add ability to make the restriction of api calls to private addresses optional
* chore: fix typo
* Update redash/settings/__init__.py
Co-authored-by: lprice92 <lprice92@iastate.edu>
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* Limit filters to 40% of query fixed layout space
* Add check for height to determine fixed layout
* Add maxTagCount of 5 to Filters
* Update maxTagCount settings to be similar to Parameters
* feat: provide ssl options for Cassandra data source
* remove Log and prints
* Refactor to create module methods and unit tests
* Switch to using Enumerator and temp file
* Fix temporary file lifecycle for cert
* Align with changes on master
* Fix non certificate but ssl enabled usecase
* Fix query based param with no results crashing page
* Add message for empty dropdown parameters
* Handle 500 no results case with empty result set
* Cypress: Make sure it shows the message
* Use .ant-select-selection to open dropdown
Redash's docker-compose file will no longer bring up an environment from
a cold start due to recent upstream changes to the postgres image that
force the user to either set a password for the default superuser or
opt-in to allowing all connections without a password via environment
variable.
Upstream PR: https://github.com/docker-library/postgres/pull/658
Related Discussion: https://github.com/docker-library/postgres/issues/681
* Cohort: add settings for tooltips, value formats and placeholder
* Cohort: add settings for colors
* Cohort: change all settings tabs to use horizontal inputs
* Cohort: show color labels in editor
* Trigger lint error on warnings on CI
* Test removing pip3 command from frontend unit
* Test eslint warning
* Revert "Test eslint warning"
This reverts commit 89d407345a.
* Revert "Test removing pip3 command from frontend unit"
This reverts commit 424c900200.
* Run apt update before installing pip3
* Stop using route.resolve feature (pages should load all the data themselves)
* Remove route.resolve feature
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* Allow touch action on dashboard grid
* Deactivate touch when resizing widgets
* Disable touch interactions on Plotly
* Update Plotly and use dragmode: false
* Remove autoFocus from ItemsList search
* Fix spacing for queries and dashboard favorites
* Make sure admin pages don't go over 100% width
'postgres' is a default database name in the Docker image, but if an
external database server is used, than Redash database can have
a different name (specified in REDASH_DATABASE_URL).
* Fix: when default value is false make sure it's still stringified.
* Fix: when extra field is of type boolean make sure it's different from default value to decide if it's open.
* Use isNil to check the default value
* Restyled by prettier (#4704)
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
* Snowflake: use different method of showing columns if no schema specified in db name
* Update redash/query_runner/snowflake.py
Co-Authored-By: Omer Lachish <omer@rauchy.net>
* Update redash/query_runner/snowflake.py
Co-authored-by: Omer Lachish <omer@rauchy.net>
* move filtering of invalid schedules to the query
* simplify retrieved_at assignment and wrap in a try/except block to avoid one query blowing up the rest
* refactor refresh_queries to use simpler functions with a single responsibility and add try/except blocks to avoid one query blowing up the rest
* avoid blowing up when job locks point to expired Job objects. Enqueue them again instead
* there's no need to check for the existence of interval - all schedules have intervals
* disable faulty schedules
* reduce FP style in refresh_queries
* report refresh_queries errors to Sentry (if it is configured)
* avoid using exists+fetch and use exceptions instead
* Update oracle.py
The reason I propose this change is to fix an issue when oracle password has an @
example of connection string: user/p@ssword@host
* Update oracle.py
Fixing init after comments
* Remove empty constructor.
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* Realign Data Source and Refresh Schedule
* Adjust execution status height
* Rewrite Query Page Header flexibility
* Remove margin from QuerySource parameters
* Cypress: Visit visualization instead of click in tab
* Fix wrong css class name in dashboard-grid
Because we already call USE DATABASE before running SHOW COLUMNS adding IN DATABASE is redundant, but causes an error if the user specifies a schema along with database name.
* Adds logic to sort column names returned by the query runner. If `sorted`
raises an Exception it returns the column names unaltered from the query
runner.
* Moves table name sorting from model code into schema handler.
* Moves token sorting into the model code.
* Replaces single-quotes with double-quotes for consistency.
* Applies black formatting to changes.
* Moves schema sort into separate method. Adds test.
* Fixes output schema variable name. Without this the sorted cache is never returned!
____ ____ ____ _____
/ __ \/ __ \/ __ \/ ___/
/ /_/ / /_/ / /_/ (__ )
\____/\____/ .___/____/
/_/
* Adds test case guaranteeing that the model actually _uses_ the schema sorter.
Related to a31f90178c
* add meta information to executing queries
* add a table for running queries
* add pagination to queues table
* sort the queues table
* add pagination to all tables
* remove legacy session identifier support
* remove redundant test
* redirect to login to support any invalid session identifiers
* be more specific with caught errors
* reject empty values in DynamicForm
* don't submit form values if they are empty (unless they are
intentionally set to empty string)
* set empty values to null to clear out data source option in the model
* check explicitly for null
* - Added support to specify read preference when query a replicaset database (for example, secondaryPreferred - to try and read data from secondary before primary).
- Removed old code that used MongoClientReplicaSet as it is now just a reference to MongoClient
- Fixed a documentation type :-)
* Moving to PyMongo 3.3.1 which also supports MongoDB 3.2
* Changed the readPreference config to use an enum
* Pass readPreference to MongoClient
* primaryPreferred is now the default
* Move each hook to own file; move hooks and components to own folders
* Update URL and timer only when refresh rate changes
* Skip dashboard refresh if previous refresh is still running
* Fix test
* Add redshift role use option
* Update requirements for SSL socket wrap issue fixes
* Split Redshift class into User and IAM logins
* Update incorrect register
* Change type names
* Correct class name to inherit
* Render IAM redshift image and field order correct
* Update redash/query_runner/pg.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* Update redash/query_runner/pg.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* Remove need for specified urllib - specify pyopenssl is enough
* Pyopenssl back down to 19.0.0
Co-authored-by: Arik Fraimovich <arik@arikfr.com>
* Migrate router and <app-view> to React: skeleton
* Update layout on route change
* Start moving page routes from angular to react
* Move page routes to react except of public dashboard and visualization embed)
* Move public dashboard and visualization embed routes to React
* Replace $route/$routeParams usages
* Some cleanup
* Replace AngularJS $location service with implementation based on history library
* Minor fix to how ApplicationView handles route change
* Explicitly use global layout for each page instead of handling related stuff in ApplicationArea component
* Error handling
* Remove AngularJS and related dependencies
* Move Parameter factory method to a separate file
* Fix CSS (replace custom components with classes)
* Fix: keep other url parts when updating location partially; refine code
* Fix tests
* Make router work in multi-org mode (respect <base> tag)
* Optimzation: don't resolve route if path didn't change
* Fix search input in header; error handling improvement (handle more errors in pages; global error handler for unhandled errors; dialog dismiss 'unhandled rejection' errors)
* Fix page keys; fix navigateTo calls (third parameter not available)
* Use relative links
* Router: ignore location REPLACE events, resolve only on PUSH/POP
* Fix tests
* Remove unused jQuery reference
* Show error from backend when creating Destination
* Remove route.resolve where not necessary (used constant values)
* New Query page: keep state on saving, reload when creating another new query
* Use currentRoute.key instead of hard-coded keys for page components
* Tidy up Router
* Tidy up location service
* Fix tests
* Don't add parameters changes to browser's history
* Fix test (improved fix)
Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* enforce hard limits on non-responsive work horses by workers
* move differences from Worker to helper methods to help make the specialization clearer
* move HardLimitingWorker to redash/tasks
* move schedule.py to /tasks
* explain the motivation for HardLimitingWorker
* pleasing CodeClimate
* pleasing CodeClimate
* port query execution to RQ
* get rid of argsrepr
* avoid star imports
* allow queries to be cancelled in RQ
* return QueryExecutionErrors as job results
* fix TestTaskEnqueue and QueryExecutorTests
* remove Celery monitoring
* get rid of QueryTask and use RQ jobs directly (with a job serializer)
* Revert "remove Celery monitoring"
This reverts commit 37a74ea403.
* reduce occurences of the word 'task'
* use Worker, Queue and Job instead of spreading names that share behavior details
* remove locks for failed jobs as well
* did I not commit that colon? oh my
* push the redis connection to RQ's stack on every request to avoid verbose connection setting
* use a connection context for tests
* remove Celery monitoring
* 👋 Celery
* remove Celery from Cypress
* black it up
* some more black
* return all started/queued job ids (for future monitoring
* Restyled by prettier (#4522)
* remove celery.py
* remove some frontend residuals that reappeared after a merge
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
* launch and monitor multiple workers using supervisor
* run supervisord in non-daemon mode
* redirect all output to stdout/stderr
* no need to log supervisord's output because it is redirected to stdout anyway
* updated and less brittle healthcheck
* add supervisor healthchecks
* remove redundant supervisor installation as it is installed by pip
* add a 5 minute check gate
* enforce hard limits on non-responsive work horses by workers
* move differences from Worker to helper methods to help make the specialization clearer
* move HardLimitingWorker to redash/tasks
* move schedule.py to /tasks
* explain the motivation for HardLimitingWorker
* pleasing CodeClimate
* pleasing CodeClimate
* port query execution to RQ
* get rid of argsrepr
* avoid star imports
* allow queries to be cancelled in RQ
* return QueryExecutionErrors as job results
* fix TestTaskEnqueue and QueryExecutorTests
* remove Celery monitoring
* get rid of QueryTask and use RQ jobs directly (with a job serializer)
* Revert "remove Celery monitoring"
This reverts commit 37a74ea403.
* reduce occurences of the word 'task'
* use Worker, Queue and Job instead of spreading names that share behavior details
* remove locks for failed jobs as well
* did I not commit that colon? oh my
* push the redis connection to RQ's stack on every request to avoid verbose connection setting
* use a connection context for tests
* black it up
* run RQ on all queues when running in Cypress
* Allow executing query with either view_query or execute_query permissions.
* Render AuthHeader according to permissions.
* Don't return dashboards where you only have access to textbox widget.
Closes#4099.
* Prettier all the JS files
* Add GitHub Action to autoformat code pushed to master
* Fix eslint violation due to formatting.
* Remove GitHub actions for styling
* Add restyled.io config
* Remove app/service/query-string (unused) and its dependency.
* Fix usage of mixed operators.
* eslint --fix fixes for missing dependencies for react hooks
* Fix: useCallback dependency passed to $http's .catch.
* Satisfy react/no-direct-mutation-state.
* Fix no-mixed-operators violations.
* Move the decision of whether to render Custom chart one level up to make sure hooks are called in the same order.
* Fix: name was undefined. It wasn't detected before because there is such global.
* Simplify eslint config and switch to creat-react-app's eslint base.
* Add prettier config.
* Make sure eslint doesn't conflict with prettier
* A few updates post eslint (#4425)
* Prettier command in package.json
* crude unit tests for counter visualisation utils
* improve type safety with default param values for getCounterData()
* fix count rows never shows zero
* remove default values for getCounterData() params
* Remove --max-old-space-size=4096 from build
Looks like it's no longer needed.
* Update to node v12.
* Add build:old-node-version for those who have Node < 12.
* Add support for configuring a Username/Password for the connection to Druid
* Bump pydruid version for username/password support
* Deal with missing/empty configuration parameters
* refreshing snowflake schema w/o waking cluster
Have also added a new internal method to not select a
warehouse while executing query
Using 'show columns' to fetch database schema instead of
executing a select query in information schema
show columns does not require a warehouse to run
* modularising snowflake code to avoid repetitions
fixing internal function syntax and avoiding
code repetition
* removing user object in snowflake schema query
* Migrate Funnel visualization to React: Editor
* Migrate Funnel visualization to React: Renderer
* Replace Auto sort options with Sort Column + Reverse Order
* Add option for items limit (instead of hard-coded value)
* Add number formatting options
* Replace d3.max with lodash.maxBy; fix bug in prepareData
* Add options for min/max percent values
* Debounce inputs
* Tests
* Refine Renderer: split components, use Ant Table for rendering, fix data handling
* Extract utility function to own file
* Fix tests
* Fix: sometimes after updating options, funnel shows "ghost" rows from previous dataset
* Sort by value column by default
* Migrate Cohort to React: Editor
* Extract prepareData and getOptions to own files
* Refine CohortRenderer Angular component (js, less, prepareData) for easier migration
* Migrate Cohort to React: Renderer
* Migrate Cornelius to React: styles
* Migrate Cohort to React: Cornelius library
* Cornelius: add licence info; remove unused style
* Cornelius: use numeral to format numbers; revisit styles
* Cornelius: use moment to format date labels
* Cornelius: use chroma for cell backgrounds; update options; update proptypes; minor fixes
* Tidy up
* Tests
* add some logging to scheduler
* schedule jobs only if they are not already scheduled
* jobs scheduled with an interval over 24 hours were not repeated
* schedule version_check using standard scheduling
* clean up old jobs that are not part of the definition anymore
* add some tests
* add one more test to verify that reschedules are not done when not neccesary
* no need to check for func existence - all jobs have a func to run
* Migrate settings-screen to React
* Use black instead of blue color for active item
* Revert "Use black instead of blue color for active item"
This reverts commit 0e4ececa6a.
* Add selectable=false to the Menu
* add some logging to scheduler
* clean failed RQ job data from Redis
* move stale job purging to tasks/general.py
* provide better documentation on why we don't reject keys in FailedJobRegistry at the moment
* pleasing the CodeClimate overlords
* simplified clenaup by deleting both job data and registry entry
* use FailedJobRegistry as source of truth for purging
* remove redundant key deletion
* Update redash/settings/__init__.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* CircleCI workflow improvements
- Don't automatically build the Docker image.
- Make the Python lint step requirement for the follow up steps. When it fails it usually means there is a code error which will prevent the next steps anyway.
* Fix YAML syntax error.
* Add separate build Docker image step for master branch
* Start draft for new Parameter structure
* Add the rest of the methods
* EnumParameter
* QueryBasedDropdownParameter
* DateParameter
* DateRangeParameter
* Update Parameter usage on code
* Merge dynamicValue into normalizedValue
* Add updateLocals and omit unwanted props
* Allow null NumberParameter and omit parentQueryId
* Rename parameter getValue to getExecutionValue
* Update $$value to normalizedValue + omit on save
* Add a few comments
* Remove ngModel property from Parameter
* Use value directly in DateRangeParameter
* Use simpler separator for DateRange url param
* Add backward compatibility
* Use normalizeValue null value for isEmpty
* Start creating jest tests
* Add more tests
* Normalize null value for multi mode in Enum
* Use saved value for param isEmpty
* Make core app compatible with Python 3
No backward compatibility with Python 2.7 is kept.
This commit mostly contains changes made with 2to3 and manual
tweaking when necessary.
* Use Python 3.7 as base docker image
Since it is not possible to change redash/base:debian to Python 3
without breaking future relases, its Dockerfile is temporarly
copied here.
* Upgrade some requirements to newest versions
Some of the older versions were not compatible with Python 3.
* Migrate tests to Python 3
* Build frontend on Python 3
* Make the HMAC sign function compatible with Python 3
In Python 3, HMAC only works with bytes so the strings and the
float used in the sign function need to be encoded.
Hopefully this is still backward compatible with already generated
signatures.
* Use assertCountEqual instead of assertItemsEqual
The latter is not available in Python 3.
See https://bugs.python.org/issue17866
* Remove redundant encoding header for Python 3 modules
* Remove redundant string encoding in CLI
* Rename list() functions in CLI
These functions shadow the builtin list function which is
problematic since 2to3 adds a fair amount of calls to the builtin
list when it finds dict.keys() and dict.values().
Only the Python function is renamed, from the perspective of the
CLI nothing changes.
* Replace usage of Exception.message in CLI
`message` is not available anymore, instead use the string
representation of the exception.
* Adapt test handlers to Python 3
* Fix test that relied on dict ordering
* Make sure test results are always uploaded (#4215)
* Support encoding memoryview to JSON
psycopg2 returns `buffer` objects in Python 2.7 and `memoryview`
in Python 3. See #3156
* Fix test relying on object address ordering
* Decode bytes returned from Redis
* Stop using e.message for most exceptions
Exception.message is not available in Python 3 anymore, except
for some exceptions defined by third-party libraries.
* Fix writing XLSX files in Python 3
The buffer for the file should be made of bytes and the actual
content written to it strings.
Note: I do not know why the diff is so large as it's only a two
lines change. Probably a white space or file encoding issue.
* Fix test by comparing strings to strings
* Fix another exception message unavailable in Python 3
* Fix export to CSV in Python 3
The UnicodeWriter is not used anymore. In Python 3, the interface
provided by the CSV module only deals with strings, in and out.
The encoding of the output is left to the user, in our case
it is given to Flask via `make_response`.
* (Python 3) Use Redis' decode_responses=True option (#4232)
* Fix test_outdated_queries_works_scheduled_queries_tracker (use utcnow)
* Make sure Redis connection uses decoded_responses option
* Remove unused imports.
* Use Redis' decode_responses option
* Remove cases of explicit Redis decoding
* Rename helper function and make sure it doesn't apply twice.
* Don't add decode_responses to Celery Redis connection URL
* Fix displaying error while connecting to SQLite
The exception message is always a string in Python 3, so no
need to try to decode things.
* Fix another missing exception message
* Handle JSON encoding for datasources returning bytes
SimpleJSON assumes the bytes it receives contain text data, so it
tries to UTF-8 encode them. It is sometimes not true, for instance
the SQLite datasource returns bytes for BLOB types, which typically
do not contain text but truly binary data.
This commit disables SimpleJSON auto encoding of bytes to str and
instead uses the same method as for memoryviews: generating a
hex representation of the data.
* Fix Python 3 compatibility with RQ
* Revert some changes 2to3 tends to do (#4261)
- Revert some changes 2to3 tends to do when it errs on the side of caution regarding dict view objects.
- Also fixed some naming issues with one character variables in list comprehensions.
- Fix Flask warning.
* Upgrade dependencies
* Remove useless `iter` added by 2to3
* Fix get_next_path tests (#4280)
* Removed setting SERVER_NAME in tests setup to avoid a warning.
* Change get_next_path to not return empty string in case of a domain only value.
* Fix redirect tests:
Since version 0.15 of Werkzeug it uses full path for fixing the location header instead of the root path.
* Remove explicit dependency for Werkzeug
* Switched pytz and certifi to unbinded versions.
* Switch to new library for getting country from IP
`python-geoip-geolite2` is not compatible with Python 3, instead
use `maxminddb-geolite2` which is very similar as it includes
the geolite2 database in the package .
* Python 3 RQ modifications (#4281)
* show current worker job (alongside with minor cosmetic column tweaks)
* avoid loading entire job data for queued jobs
* track general RQ queues (default, periodic and schemas)
* get all active RQ queues
* call get_celery_queues in another place
* merge dicts the Python 3 way
* extend the result_ttl of refresh_queries to 600 seconds to allow it to continue running periodically even after longer executions
* Remove legacy Python flake8 tests
* add rq and an rq_worker service
* add rq_scheduler and an rq_scheduler service
* move beat schedule to periodic_jobs queue
* move version checks to RQ
* move query result cleanup to RQ
* use timedelta and DRY up a bit
* move custom tasks to RQ
* do actual schema refreshes in rq
* rename 'period_jobs' to 'periodic', as it obviously holds jobs
* move send_email to rq
* DRY up enqueues
* ditch and use a partially applied decorator
* move subscribe to rq
* move check_alerts_for_query to rq
* move record_event to rq
* make tests play nicely with rq
* 👋 beat
* rename rq_scheduler to plain scheduler, now that there's no Celery scheduler entrypoint
* add some color to rq-worker's output
* add logging context to rq jobs (while keeping execute_query context via get_task_logger for now)
* move schedule to its own module
* cancel previously scheduled periodic jobs. not sure this is a good idea.
* rename redash.scheduler to redash.schedule
* allow custom dynamic jobs to be added decleratively
* add basic monitoring to rq queues
* add worker monitoring
* pleasing the CodeClimate overlords
* adjust cypress docker-compose.yml to include rq changes
* DRY up Cypress docker-compose
* add rq dependencies to cypress docker-compose service
* an odd attempt at watching docker-compose logs when running with Cypress
* Revert "an odd attempt at watching docker-compose logs when running with Cypress"
This reverts commit 016bd1a93e.
* show docker-compose logs at Cypress shutdown
* Revert "DRY up Cypress docker-compose"
This reverts commit 43abac7084.
* minimal version for binding is 3.2
* remove unneccesary code reloads on cypress
* add a command which errors if any of the workers running inside the current machine haven't been active in the last minute
* SCHEMAS_REFRESH_QUEUE is no longer a required setting
* split tasks/queries.py to execution.py and maintenance.py
* fix tests after query execution split
* pleasing the CodeClimate overlords
* rename worker to celery_worker and rq_worker to worker
* use /rq_status instead of /jobs
* show started jobs' time ago according to UTC
* replace all spaces in column names
* fix query tests after execution split
* exit with an int
* general lint
* add an entrypoint for rq_healthcheck
* fix indentation
* delete all existing periodic jobs before scheduling them
* remove some unrequired requires
* move schedule example to redash.schedule
* add RQ integration to Sentry's setup
* pleasing the CodeClimate overlords
* remove replication settings from docker-compose - a proper way to scale using docker-compose would be the --scale CLI option, which will be described in the knowledge based
* revert to calling a function in dynamic settings to allow periodic jobs to be scheduled after app has been loaded
* don't need to depend on context when templating failure reports
* set the timeout_ttl to double the interval to avoid job results from expiring and having periodic jobs not reschedule
* whoops, bad merge
* describe custom jobs and don't actually schedule them
* fix merge
* don't need to depend on context when templating failure reports
* extract a render_template function with some docs
* CodeClimate has really outdone itself this time. Removed a whitespace character in order to fix 2 CodeClimate errors
* apparently whitespace doesn't count as a character
* pagerduty.py: Change default summary text
Change is made to use alert name in PagerDuty's alert
destination default summary text instead of
query id and name
* Update default description text & field description
* Add interface to implement custom persistence for QueryResult data
Co-authored-by: Omer Lachish <omer@rauchy.net>
* Deserialize query results data in the model
* Change order of mixins.
* Make DBPersistence.data setter in sycn with getter + tests
* The sender email address has to be None for the test of "is email server
configured" to be correct. Moved the dev setting into docker-compose.yml.
* Move the REDASH_MAIL_SERVER setting into docker-compose.yml to revert the default value to its original value in case anyone was using it.
* Make worker dependant on email as it's the one that actually using it.
* DynamicForm support for advanced options
* Randomly select a few options to be advanced
* Merge conditions with the same logic
* Address some comments
* Update styling for the button
* Some style adjustments (#4162)
* Don't set default value to additional settings
* Rename advanced -> extra
* Show extra fields by default when they are filled
* Update hasFilledExtraField logic
* Add example field from destination as extra
* Support for predefined options in data sources
* DynamicForm Select: title -> name
* Make it work with "enum" prop
* Make it work for "extendedEnum" prop
* Not JS
* Deep copy the configuration schema
* Handle non-array in multi-value QueryBasedParameter
* Use state value in QueryBasedParameterInput
* Normalize array in parameter structure
* Add Multi-selection test
* Remove unnecessary not null check
* npm install react-pivottable
* Initiate Pivot Table Migration
* Update renderer with editor options
* Clean up
* Remove old pivottable from package.json
* Test Percy Snapshot with Pivot Table in a Dashboard
* Tmp: use cy.wait to make sure dashboard is loaded
* Clean up Percy snapshot test
* Small improvements
- cy.all with multiple args
- add controls to pivot valid options
* Watch for options in the Renderer
* Improve sizing for Number inputs
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Migrate WidgetDialog
* Start migrating Widget
* Update textbox to use HtmlContent
* QueryLink migration and some updates
* Add visualization rendering
* Render widget
* Add delete button
* Update AutoHeight
* Add widget bottom
* Add Drodpown button
* Split Widget component
* Update with #4056 and trigger netlify
* In progress: use composition
* Add header and footer
* Update widget actions positioning
* Re-render when refreshing from widget
* Add workaround to force DashboardGrid re-render
* VisualizationWidgetFooter component
* VisualizationWidget menu
* Separate RestrictedWidget
* Update tests
* Update margin for Parameters
* Remove widget files
* Revert "Improve sizing for Number inputs"
This reverts commit a02ce8f0aa.
* Some cleanup
* Move refresh logic to the Dashboard
* Add loadingWidgets logic to the public dashboard
* Add onLoadWidget
* Remove parameter from URL when empty
* Recreate widget array instead of loadingWidgets
* Add comment about re-rendering + whitespace missing
* CR changes
* Use plain html instead of string syntax
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Snippets: Don't change url when not needed
* Revert "Snippets: Don't change url when not needed"
This reverts commit 2f346f3bb4.
* Query Snippets: use onClick instead of link
* Initial commit of BeaconConsent component
* Add comment about being able to change setting
* Use <Text> correctly
* Final version of consent screen
* Show beacon consent message on homepage only if it wasn't enabled already.
* Add consent setting to organization settings screen.
* Add support for custom message in OrgSetting.save.
* Implmenet consent saving.
* If consent given, send extra data
* Add HelpTrigger
* Make CodeClimate happy
* Wrap everything with DynamicComponent
* getredash/redash#2629 Refactor Chart visualization, add option for handling NULL values (keep/convert to 0.0)
* Handle null values in line/area stacking code; some cleanup
* Handle edge case: line/area stacking when last value of one of series is missing
* Mjnor update to line/area stacking code
* Fix line/area normalize to percents feature
* Unit tests
* Refine tests; add tests for prepareLayout function
* Tests for prepareData (heatmap) function
* Tests for prepareData (pie) function
* Tests for prepareData (bar, line, area) function
* Tests for prepareData (scatter, bubble) function
* Tests for prepareData (box) function
* Remove unused file
* Code formatting
* Move annotation logic into query runner, so it can be overriden in the query runner.
* Add mixin to __all__
* Switch to flag instead of mixin
* Feature (Redshift): option to set query group for adhoc/scheduled queries (#4114)
* Add scheduled status to query job metadata.
* Add: option to set query group for adhoc/scheduled Redshift queries
* Scheduled might not be set for already enqueued queries.
* refresh only affected queries in dashboard when parameters are changed
* rename pendingParameters to updatedParameters
* select which widgets to update according to their mapping as a dashboard-level parameter
* use lodash's include
* [Qubole] - Adding support to process Quantum query types.
Quantum is a serverless interactive service that offers
direct SQL access to user's data lake. Changes are made
to accept `quantum` query type from user which makes
`Cluster Label` as optional.
* -Making quantum as defult query.
-Dictionary safe access to connection parmeters
* keeping pep8 standards
* Maintainig pep8 std
* Use latest version of qds-sdk
* Use qds-sdk v1.13.0
* Use qds-sdk v1.12.0
* Use qds-sdk v1.13.0
* Updating SDK with verified version
* hive as default query type
* qds-sdk : Locking most recent release version
* qds-sdk : Locking recent release version
* falling back to original version of qds-sdk
* [Data Sources] Add: Azure Data Explorer (Kusto) query runner
* CodeClimate fixes
* Remove TODO
* Fixed configuration properties names for Azure Kusto
* Azure Kusto: get_schema in one query
* azure-kusto-data update to 0.0.32
* Add Kusto to the default query runners list
kombu is a dependency of Celery and usually we let them declare the version, but their version is pinned and the most recent release (4.6.4) has a severe regression where workers stop responding to inspect commands.
* Initial commit for adding Dgraph support
* Made suggestions from https://codeclimate.com/github/getredash/redash/pull/3964
* feedback from @arikfr
* added logo for Dgraph from Twitter
* Better conversion of Dgraph JSON to Redash's internal JSON
* made recommendations from @arikfr
* removed unused function
* Fix loading of periodic tasks and clean up extension loading.
This does a few things:
- add tests for extension loading
- refactor the extension and periodic task loading
- better handle assertions raised by extensions (e.g. when an extension tries to override an already registered view)
- attach exception traceback to error log during loading for improved debugging
* Use site.addsitedir instead of calling pip.
* Use sys.path instead of site.addsitedir and also the setup.py egg_info command.
* allowing to specify a custom work group for AWS Athena queries
* Fixing title + adding correct position in the UI
* Adding assume role configuration to Athena query runner.
* removing extra blank lines
* fixes based on comments to the PR
With the default Docker installed from sources on Ubuntu 19.04 it failed starting the project when asking for Compose version 3.7, but everything worked fine with 3.2.
* Fix W292 no newline at end of file
* Fix extra whitespace
* Fix E305 expected 2 blank lines after class or function definition
* Fix W391 blank line at end of file
* Fix E231 missing whitespace after
* Fix E303 too many blank lines
* Fix E302 expected 2 blank lines
* Fix E128 continuation line under-indented for visual indent
* avoid using 'abort' in parameterized query - raise an exception instead
* when facing invalid parameters or detached dropdown queries - continue to refresh the rest of the outdated queries
* test that dropdown queries detached from data source raise an exception when fetch values is attempted
* test that queries with invalid parameters arent refreshed
* test that queries with dropdown query parameters which are detached from the data source are skipped
* fix stale test double name
* newlines. newlines everywhere.
* pass org into dropdown_values
* pass in org in every ParameterizedQuery usage
* Update redash/tasks/queries.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* reduce refresh_queries log noise
* track failure count for queries that failed to apply parameters, and also notify the failures
* Update redash/tasks/queries.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* newlines. newlines everywhere.
* Allow multiple values for enum parameter
* Allow multi-select for Query dropdown parameters
* CR + make sure list values are allowed
* Add prefix, suffix and separator
* Rename multipleValues and cast options as strings
* Replicate serialization logic on frontend
* Add Quote Option Select
* Make sure it's enum or query before join
* Add a couple of tests
* Add help to quote option
* Add min-width and normalize empty array
* Improve behavior when changing parameter settings
- Set parameter value again to pass through checks
- Add setValue check for multi values
* Validate enum values on setValue + CodeClimate
* Ran wording suggestions
* Updates after Apply Changes
* Fix failing Cypress tests
* Make sure enumOptions exists before split
* Improve propTypes for QueyBasedParameterInput
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* CR
* Cypress: Test for multi-select Enum
* Fix multi-selection Cypress spec
* Update Refresh Schedule
* initial work on e-mail report for failed queries
* send failure report only for scheduled queries and not for adhoc queries
* add setting to determine if to send failure reports
* add setting to determine interval of aggregated e-mail report
* html templating of scheduled query failure report
* break line
* support timeouts for failure reports
* aggregate errors within message and warn if approaching threshold
* handle errors in QueryExecutor.run instead of on_failure
* move failure report to its own module
* indicate that failure count is since last report
* copy changes
* format with <code>
* styling, copy and add a link to the query instead of the query text
* separate reports with <hr>
* switch to UTC
* move <h2> to actual e-mail subject
* add explicit message for SoftTimeLimitExceeded
* fix test to use soft time limits
* default query failure threshold to 100
* use base_url from utils
* newlines. newlines everywhere.
* remove redundant import
* apply new design for failure report
* use jinja to format the failure report
* don't show comment block if no comment is provided
* don't send emails if, for some reason, there are no available errors
* subtract 1 from failure count, because the first one is represented by 'Last failed'
* don't show '+X more failures' if there's only one
* extract subject to variable
* format as text, while we're at it
* allow scrolling for long exception messages
* test that e-mails are scheduled only when beneath limit
* test for indicating when approaching report limits + refactor
* test that failures are aggregated
* test that report counts per query and reason
* test that the latest failure occurence is reported
* force sending reports for testing purposes
* Update redash/templates/emails/failures.html
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Update redash/templates/emails/failures.html
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Update redash/tasks/failure_report.py
* add org setting for email reports
* remove logo from failure report email
* correctly use the organization setting for sending failure reports
* use user id as key for failure reports data structure
* Update redash/tasks/failure_report.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* build comments while creating context for e-mail templates
* figure out the base url when creating the e-mail
* no need to expire pending failure report keys as they are deleted anyway when sent
* a couple of CodeClimate changes
* refactor key creationg to a single location
* refactor tests to send e-mail from a single function
* use beat to schedule a periodic send_aggregated_errors task instead of using countdown per email
* remove pending key as it is no longer required when a periodic task picks up the reports to send
* a really important blank line. REALLY important.
* Revert "a really important blank line. REALLY important."
This reverts commit c7d8ed8972.
* a really important blank line. REALLY important. It is the best blank line.
* don't send failure emails to disabled users
* Draft for Date Dynamic values
* Use value with prefix instead of specific attr
* Fix not possible to select static value
* Update antd version
* Cleanup and DateRangeParameter
* Dynamic DateTimeRange
* Add Dynamic options to Date Parameters
* UI refinements
* Add getDynamicValue function
* Add 'This' options and prevent text clipping
* Make allowClear available
* Update ScheduleDialog snapshot
* Add some protections and separate Date/DateRange
* Accept null values on date or daterange parameters
* Handle undefined values on Moment propType
* Move export to end of files
* Remove Today/Now option
* Update with Apply Changes
* Show name instead of value for dynamic values
* Add comment about supporting useCurrentDateTime
* Cypress Tests: Date Parameters
* Cypress Tests: Date Range Parameters
* Don't put null params in the url
* Add workaround comments to Cypress tests
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Fix Dynamic Value as default for global parameters
* Update Back to Static Value
* Add isValid to value on Date and DateRange inputs
* CR suggestions
* Fix Back to Static Value for Dates
* Update Dynamic Value Styling
* Fix failing Date tests
* Fix selectedDynamicValue
* Parameter spec: Remove date range clickThrough
* Add transition
* Fix failing Cypress tests
* Back with 'width: auto'
* Check value is valid on Back to Static value
* CR
* Update Date Range width
* add a deprecated flag to query runners and show only non-deprecated query runners when adding a new data source
* add a deprecated flag to alert destinations and show only non-deprecated alert destinations when adding a new alert destination
* add a deprecated() decorator for a more succint way to deprecate
* deprecate URL query runner and HipChat alert destination
* use class properties instead of class methods for deprecation
* I <3 newlines
* return message explaining unsafe sharing
* use backend-generated message for public dashboards
* use backend-generated message for embeds
* Update redash/handlers/query_results.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* refactor simple (non-interpolated) query result handler error messages to a single location
* use error_messages to test out unsafe error messages (along with a couple of others)
* Update redash/handlers/query_results.py
Co-Authored-By: Ran Byron <ranbena@gmail.com>
* Update redash/handlers/query_results.py
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* avoid catching errors on text widgets' load(), as they don't have a visualization and therefore do not return any promise
* throw error when failing to load widgets on public dashboards - in case something needs to be done with it at a later time, and it's the right thing to do anyway
* use Promise.resolve instead of checking for undefined
* call serialize_query_result instead of directly calling to_dict
* filter unneeded query result fields for unauthenticated users
* test for serialization filtering
* lint
* use project instead of list comprehension
* pick up *.py file changes and restart scheduler
* only watch /redash in order to avoid reloading on other file changes (e.g. tests)
* add dev_scheduler entrypoint
* use exec
* Update bin/docker-entrypoint
* rename dev_scheduler to dev_worker
* use same defaults as worker
* avoid catching errors on text widgets' load(), as they don't have a visualization and therefore do not return any promise
* throw error when failing to load widgets on public dashboards - in case something needs to be done with it at a later time, and it's the right thing to do anyway
* use Promise.resolve instead of checking for undefined
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* use the textless endpoint (/api/queries/:id/results) for pristine
queriest
* Revert "use the textless endpoint (/api/queries/:id/results) for pristine"
This reverts commit cd2cee7738.
* go to textless /api/queries/:id/results by default
* change `run_query`'s signature to accept a ParameterizedQuery instead of
constructing it inside
* raise HTTP 400 when receiving invalid parameter values. Fixes#3394
* enqueue jobs for ApiUsers
* rename `id` to `user_id`
* support executing queries using Query api_keys by instantiating an ApiUser that would be able to execute the specific query
* show deprecation messages for ALLOW_PARAMETERS_IN_EMBEDS. Also, move
other message (email not verified) to use the same mechanism
* add link to forum message regarding embed deprecation
* change API to /api/queries/:id/dropdowns/:dropdown_id
* split to 2 different dropdown endpoints and implement the second
* add test cases for /api/queries/:id/dropdowns/:id
* use new /dropdowns endpoint in frontend
* first e2e test for sharing embeds
* Pleasing the CodeClimate overlords
* All glory to CodeClimate
* remove residues from bad rebase
* add query id and data source id to serialized public dashboards
* add global parameters directive to public dashboards page
* allow access to a query by the api_key of the dashboard which includes
it
* rename `object` to `obj`
* simplify permission tests once `has_access` accepts groups
* support global parameters for public dashboards
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* rename `object` to `obj`
* simplify permission tests once `has_access` accepts groups
* no need to log `is_api_key`
* send parameters to public dashboard page
* allow access to a query by the api_key of the dashboard which includes it
* disable sharing if dashboard is associated with unsafe queries
* remove cypress test added in the wrong place due to a faulty rebase
* add support for clicking buttons in cy.clickThrough
* Cypress test which verifies that dashboards with safe queries can be shared
* Cypress test which verifies that dashboards with unsafe queries can't be shared
* remove duplicate tests
* use this.enabled and negate when needed
* remove stale comment
* add another Cypress test to verify that unauthenticated users have access to public dashboards with parameters
* obviously, I commit 'only' the first time I use it
* search for query access by query id and not api_key
* no need to fetch latest query data as it is loaded by frontend from the textless endpoint
* test that queries associated with dashboards are accessible when supplying the dashboard api_key
* propagate `isDirty` down to `QueryBasedParameterInput`
* go to /api/:id/dropdown while editing a query, since dropdown queries might still not be associated with the parent. see #3711
* show helpful error message if dropdown values cannot be fetched
* use backticks instead of line concatenation
* remove requirement to have direct access to dropdown query in order validate it. parent query association checks are sufficient
* remove isDirty-based implementation and allow dropdown queries through nested ACL even if they aren't associated yet (given that the user has _direct_ access to the dropdown query)
* fix tests to cover all cases for /api/queries/:id/dropdowns/:id
* fix indentation
* require access to the query, not the data source
* resolve dashboard user by query id
* apply new copy to Cypress tests
* if only something would have prevented me from commiting an 'only' call 🤔
* very important handling of whitespace
* respond to parameter's Apply button
* text widgets are safe for sharing
* remove redundant event
* add a safety check that object has dashboard_api_keys before calling it
* supply a parameter value for text parameters to have it show up
* add parameter values for date and datetime
* use the current year and month to avoid pagination
* use Cypress.moment() instead of preinstalled moment()
* explicitly create parameters
* refresh query data if a querystring parameter is provided
* avoid sending a data_source_id - it's only relevant to unsaved queries, since a saved query's data_source is available in the backend
* remove empty query text workaround
* provide default value to parameter
* add a few more dashboard sharing specs
* lint
* wait for DynamicTable to appear to reveal that actual results are displaying
* override error message for unsafely shared widgets
* Support multi-byte search for query names and descriptions
* add multi_byte_support_enabled option to organization settings
* add `ilike %...%` to query search conditions when the option is enabled
* Improve description for multi_byte_search_enabled option
Co-Authored-By: Arik Fraimovich <arik@arikfr.com>
* Remove tsvector from search when multi_byte_search_enabled
* Add a multi-byte search test case
* Allow calling query results endpoint without parameters.
* Fix: allow serializing empty or bad dates
* Revert "Fix: allow serializing empty or bad dates"
This reverts commit cc49319d9e.
* supply a parameter value for text parameters to have it show up
* add parameter values for date and datetime
* use the current year and month to avoid pagination
* use Cypress.moment() instead of preinstalled moment()
* capture time before clicking on Now
* use now from input
* use now from input for another test
* Add regenerate function of query's API Key
* Add regenerate API Key button
* Add regenerate Query API Key tests
* Fix too long line
* Replace with this
* Return a simple version query
* Update only API Key
* Update API Key via query
* Celery task to clear schedule was added
* fix formating
* empty_schedules task was put in separate task
* worker interval changed, new tests added
* past artifact deleted
* test queries moved to right class, lambda was used to filter data
* unnecessary changes eliminated
* more unnecessary files deleted
* line shortened
* Line shortened more
* codeclimate changes
* Unused test deleted, logs added
* remove legacy session identifier support
* remove redundant test
* redirect to login to support any invalid session identifiers
* be more specific with caught errors
* use authorization according to api_key (if provided) over session
* flatten lists and dicts to json to be used with SQLite's json_extract functions
* add test that verifies that lists and dicts are saved
* add test that verifies that lists and dicts are saved
* remove legacy session identifier support
* remove redundant test
* redirect to login to support any invalid session identifiers
* be more specific with caught errors
* fix refresh for public dashboards
* remove legacy session identifier support
* remove redundant test
* redirect to login to support any invalid session identifiers
* be more specific with caught errors
* Update PromiseRejectionError to show error message from API response
* Update version to 8.0.0-beta.
* Revert "Update version to 8.0.0-beta."
This reverts commit c8fa74967f.
* Decouple extensions from Flask app.
This separates the extension registry from the Flask app and also introduces a separate registry for preriodic tasks.
Fix#3466.
* Address review feedback.
* Update redash/extensions.py
Co-Authored-By: jezdez <jannis@leidel.info>
* Minor comment in requirements.
* Refactoring after getting feedback.
* Uncoupled bin/bundle-extensions from Flas app instance.
* Load bundles in bundle script and don’t rely on Flask.
* Upgraded to importlib-metadata 0.9.
* Add missing requirement.
* Fix TypeError.
* Added requirements for bundle_extension script.
* Install bundles requirement file correctly.
* Decouple bundle loading code from Redash.
* Install bundle requirements from requirements.txt.
* Use circleci/node for build-docker-image step, too.
* Dashboard grid React migration
* Updated tests
* Fixes comments
* One col layout
* Tests unskipped
* Test fixes
* Test fix
* AutoHeight feature
* Kebab-cased
* Get rid of lazyInjector
* Replace react-grid-layout with patched fork to fix performance issues
* Fix issue with initial layout when page has a scrollbar
* Decrease polling interval (500ms is too slow)
* Rename file to match it's contents
* Added some notes and very minor fixes
* Fix Remove widget button (should be visible only in editing mode); fix widget actions menu
* Fixed missing grid markings
* Enhanced resize handle
* Updated placeholder color
* Render DashboardGrid only when dashboard is loaded
* Add touch state to parameters and autoupdate query
* Use values change event instead of $watch
* Remove getQueryResultDebounced
* Add Apply button
* Remove Input Number spinners for Parameters
* Make Apply Button optional
* Update share_embed_spec
* Change debounce to the Parameters component
* Remove unnecessary click on Execute query
* Add apply button to the remaining places
* Update dashboard_spec
* Use onKeyUp for InputNumber
* Simplify onParametersValuesChanged
* Update DateTime onChange function
* Don't apply when modifier key is pressed
* Remove refresh Button from Parameters
* Update apply button styling
* Update apply right distance
* Remove debounce for testing
* Use data-dirty instead of classNames for styling
* Make sure $apply runs before calling onChange
* If MySQL returns multiple resultSets (eg when executing multiple statements, or calling stored procedures) then use the last resultSet that has columns
* Make cancellation of a MySQL query work (the same way the C client does it)
* Address code climate moans
* Migrate Organization Settings to React
* Fix failing spec and replace default values from inputs
* Add HelpTrigger and handleChange to SAML options
* Undo changes to ant-variables.less
* Add time format to OrganizationSettings
* propagate `isDirty` down to `QueryBasedParameterInput`
* go to /api/:id/dropdown while editing a query, since dropdown queries might still not be associated with the parent. see #3711
* show helpful error message if dropdown values cannot be fetched
* use backticks instead of line concatenation
* remove requirement to have direct access to dropdown query in order validate it. parent query association checks are sufficient
* remove isDirty-based implementation and allow dropdown queries through nested ACL even if they aren't associated yet (given that the user has _direct_ access to the dropdown query)
* fix tests to cover all cases for /api/queries/:id/dropdowns/:id
* fix indentation
* require access to the query, not the data source
* use require_access instead of has_access
* provide queryId when fetching query results in order to allow query-based api-key authentication to work properly
* cypress test to verify that embeds without parameters are shared succesfully
* rename Percy snapshot
* add an endpoint for fetching job using a query's api_key
* when unauthenticated, use api_key to get job, and fetch the latest query
result (as opposed to fetching the query result by ID)
* add 'refresh dataset' button to parameters directive
* fix scope error introduced by earlier commit
* show timer when refreshing results
* Show input for missing parameters in embedded visualizations (#3741)
* Redirect to default parameter values when parameters are missing in
embedded visualizations
* Revert "Redirect to default parameter values when parameters are missing in"
This reverts commit 43c65500b7.
* load all data after page is loaded
* return no data only when parameters are missing
* data binding no longer required
* show an error on embeds that fail to load
* data binding no longer required
* present full-page error when dealing with unsafe queries
* don't render the execute button for each parameter
* show 'missing parameter value' error
* Don't reload the whole page when parameter value changes.
* Set API key and load config before rendering.
* Add Query#hasParameters method.
* Don't show download controls for parameterized queries (they won't work).
* Use getUrl to construct a correct query link.
* WIP: have a single way to load results
1. This preloads the query before rendering the page, so we can benefit from using default parameters & make the logic in component simpler.
2. Use a single way to load results, to make sure we do polling when try to load the query results for the first time.
* Show persistent errors and finish loading logic.
* Check if query is safe and show message otherwise.
* Fix test for unsafe parameters embed.
* wait for query results to return before taking snapshot
## What type of PR is this? (check all applicable)
<!-- Please leave only what's applicable -->
- [x] Refactor
- [x] Bug Fix
## Description
This basically makes sure that when import the redash package we don't accidentally trigger import-time side-effects such as requiring Redis.
Refs #3569 and #3466.
## What type of PR is this? (check all applicable)
- [x] Bug Fix
## Description
Without this change the Help Drawer couldn't load content anymore.
## Related Tickets & Documents
#3404
* Add rate limits for user resources.
* Disable rate limiting in tests (except for tests that need it).
* Update strings to unicode to avoid SQLA warnings
* extract time limit decisions to a dynamic settings function
* introduce environment variable for scheduled query time limits
* pass in org_id to query_time_limit
* add an interaction test that verifies that time limits are applied to
jobs
* really important newlines according to CodeClimate
* Don't force an order by created date - any usecases that want
this already request it explicitly and it breaks the search
function that wants to order by best match.
* Fix the logic so that we fall back to a default search order when there's
no search term, rather than when there is one and we should use the
best-match ordering.
* Remove accidentially added blank line
* support date ranges for parameterized embeds
* add qs
* remove hideous implementation and use qs
* get rid of apiKey querystring parameter and introduce query-string
module
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* use the textless endpoint (/api/queries/:id/results) for pristine
queriest
* Revert "use the textless endpoint (/api/queries/:id/results) for pristine"
This reverts commit cd2cee7738.
* go to textless /api/queries/:id/results by default
* change `run_query`'s signature to accept a ParameterizedQuery instead of
constructing it inside
* raise HTTP 400 when receiving invalid parameter values. Fixes#3394
* support querystring params
* extract coercing of numbers to function, along with a friendlier
implementation
* wire embeds to textless endpoint
* allow users with view_only permissions to execute queries on the
textless endpoint, as it only allows safe queries to run
* enqueue jobs for ApiUsers
* add parameters component for embeds
* include existing parameters in embed code
* fetch correct values for json requests
* remove previous embed parameter code
* rename `id` to `user_id`
* support executing queries using Query api_keys by instantiating an ApiUser that would be able to execute the specific query
* bring back ALLOW_PARAMETERS_IN_EMBEDS (with link on deprecation coming up)
* show deprecation messages for ALLOW_PARAMETERS_IN_EMBEDS. Also, move
other message (email not verified) to use the same mechanism
* add link to forum message on setting deprecation
* rephrase deprecation message
* add link to forum message regarding embed deprecation
* change API to /api/queries/:id/dropdowns/:dropdown_id
* split to 2 different dropdown endpoints and implement the second
* add test cases for /api/queries/:id/dropdowns/:id
* use new /dropdowns endpoint in frontend
* first e2e test for sharing embeds
* Pleasing the CodeClimate overlords
* All glory to CodeClimate
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* split has_access between normal users and ApiKey users
* remove residues from bad rebase
* allow access to safe queries via api keys
* rename `object` to `obj`
* support both objects and group dicts in `has_access` and `require_access`
* simplify permission tests once `has_access` accepts groups
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* rename `object` to `obj`
* support both objects and group dicts in `has_access` and `require_access`
* simplify permission tests once `has_access` accepts groups
* fix bad rebase
* send embed parameters through POST data
* no need to log `is_api_key`
* move query fetching by api_key to within the Query model
* fetch user by adding a get_by_id function on the User model
* pass parameters as POST data (fixes test failure introduced by switching
from query string parameters to POST data)
* test the right thing - queries with safe parameters should be embeddable
* introduce cy.clickThrough
* add another Cypress test to make sure unsafe queries cannot be embedded
* serialize Parameters into query string
* set is_api_key as the last parameter to (hopefully) avoid
backward-dependency problems
* Update redash/models/parameterized_query.py
Co-Authored-By: rauchy <omer@rauchy.net>
* attempt to fix empty percy snapshots
* snap percies after DOM is fully loaded
* handle an edge case where dropdown queries are connected to data sources
that no longer exist
* Rethinking it, an empty result set makes no sense and it's better to
throw an error
* remove redundant import
* change has_access and require_access signatures to work with the objects that require access, instead of their groups
* rename `object` to `obj`
* support both objects and group dicts in `has_access` and `require_access`
* simplify permission tests once `has_access` accepts groups
* Migrate TypePicker to React
* Migrate DataSources and Destinations List
* Fixes to DestinationsList
* Add CreateDataSource (testing with Steps)
* Render the form after type selection
* Add HELP_LINKS to CreateDataSource
* Add Done behavior
* Add scrollToTop to CreateDataSource
* TypePicker styling adjusts
* Add CreateDestination
* Update resouce gets to componentDidMount
* Create EditForm components
* Migrate Edit pages
* Remove angular logic from DynamicForm
* Add actions to EditPages
* TypePicker title style adjustments
* Add Empty and Loading state
* UX improvements
* Review changes
* Styling updates on TypePicker, forms background fix
* Add blank line removed by mistaken
* Reorganize TypePicker
* Hide Search on List Pages
* Fix spacing in Forms
* Update Create Data Source and Destination to be a Dialog
* Remove max-height from the form
* Fix DynamicForm import in CreateUserDialog
* Route /new to open CreateSourceDialog
* Add HelpTrigger + refine styling and Edit Pages
* Remove help links from data source resource
* Update Cypress specs
* TypePicker -> CardsList
* Remove old TypePicker styling and change CardsList styling to less
* Test if Percy shows Dialogs
* Personal review cleanup
* CR
* Remove unnecessary query on dialog success
* Handle resource errors in Edit Pages
* Add CreateDestination policy
* Add placeholder and separator to the Name field
* Use cy.click instead of cy.wait
* Revert "Use cy.click instead of cy.wait" (Didn't work)
This reverts commit 77285d9fa3.
* Align help trigger on the right and rename Steps
* Refine behavior for long names
* Update toastr calls to use notification instead
* Redirect to target after creation
* Remove autoFocus on DynamicForm for Edit Pages
* Add eslint-disable for cy.wait
* Fix Celery worker CLI parameter name that was changed in Celery 4.x.
* Set Celery worker --max-memory-per-child to 1/4th of total system memory.
* Review fixes.
* Review fixes.
* check that e-mail server is configured before marking the email address
as not verified and sending out a verification e-mail
* use helper method in `invite_user`
* move email_server_configured helper to settings
* add test to verify that email addresses arent marked as unverified if
there's no e-mail server to verify them
* simplify a couple of tests with patch
* combine conditions into single variable
* Booleans, gotta love 'em
* Browser support config
* Removed some offending code
* Added unsupported html page and redirect for IE
* Typo in regex
* Made html page static
* Added redirect script to multi_org
* Moved static html page to client/app
* Normalize Flask initialization API use.
* Use Flask-Talisman.
* Enable HSTS when HTTPS is enforced.
* More details about how CSP is formatted and write CSP directives as a string.
* Use CSP frame-ancestors directive and not X-Frame-Options for embedable endpoints.
* Add link to flask-talisman docs.
* set remember_token cookie to be HTTP-Only and Secure
* Reorganize secret key configuration to be forward thinking and backward compatible.
* Fix a few more inconsistencies when loading and dumping JSON.
Refs #2807. Original work in: #2817
These change have been added since c2429e92d2.
* Review fixes.
* change API to /api/queries/:id/dropdowns/:dropdown_id
* extract property
* split to 2 different dropdown endpoints and implement the second
* make access control optional for dropdowns (assuming it is verified at a
different level)
* add test cases for /api/queries/:id/dropdowns/:id
* use new /dropdowns endpoint in frontend
* require access to dropdown queries when creating or updating parent
queries
* rename Query resource dropdown endpoints
* check access to dropdown query associations in one fugly query
* move ParameterizedQuery to models folder
* add dropdown association tests to query creation
* move group by query ids query into models.Query
* use bound parameters for groups query
* format groups query
* use new associatedDropdowns endpoint in dashboards
* pass down parameter and let it return dropdown options. Go Levko!
* change API to /api/queries/:id/dropdowns/:dropdown_id
* split to 2 different dropdown endpoints and implement the second
* use new /dropdowns endpoint in frontend
* pass down parameter and let it return dropdown options. Go Levko!
* fix bad rebase
* add comment to clarify the purpose of checking the queryId
This release was long time in the making and has several major changes:
- Our backend code was updated to support Python 3 and we no longer support Python 2. If you're using our Docker images, this should be a transparent change for you.
- We replaced Celery with RQ for background jobs processing. This will require some setup updates -- see instructions below.
- The frontend code is now 100% React and we removed all the Angular dependencies.
This release was made possible by contributions from over 50 people: @ari-e, @ariarijp, @arihantsurana, @arikfr, @atharvai, @cemremengu, @chulucninh09, @citrin, @daniellangnet, @DavidHernandez, @deecay, @dmudro, @erans, @erels, @ezkl, @gabrieldutra, @gstaykov, @ialeinikov, @ikenji, @Jakdaw, @jezdez, @juanvasquezreyes, @koooge, @kravets-levko, @kykrueger, @leibowitz, @leosunmo, @lihan, @loganprice, @mickeey2525, @mnoorenberghe, @monicagangwar, @NicolasLM, @p-yang, @Ralnoc, @ranbena, @randyzwitch, @rauchy, @rxin, @saravananselvamohan, @satyamkrishna, @shinsuke-nara, @stefan-mees, @stevebuckingham, @susodapop, @taminif, @thewarpaint, @tsuyoshizawa, @uncletimmy3, @wengkham.
### Upgrading
Typically, if you are running your own instance of Redash and wish to upgrade, you would simply modify the Docker tag in your `docker-compose.yml` file. Since RQ has replaced Celery in this version, there are a couple extra modifications that need to be done in your `docker-compose.yml`:
1. Under `services/scheduler/environment`, omit `QUEUES` and `WORKERS_COUNT` (and omit `environment` altogether if it is empty).
2. Under `services`, add a new service for general RQ jobs:
```yaml
worker:
<<:*redash-service
command:worker
environment:
QUEUES:"periodic emails default"
WORKERS_COUNT:1
```
Following that, force a recreation of your containers with `docker-compose up --force-recreate --build` and you should be good to go.
### UX
- Redesigned Query Results page:
- Completely new layout is easier to read for non-technical Redash users.
- Empty query results are clearly displayed. User is now prompted to edit or execute the query.
- Mobile Experience Improvements:
- UI element spacing has been redesigned for clarity
- Admin pages now honor max-width. Tables scroll independent of the top menu.
- Large legends no longer shrink the visualization on small screens.
- Fix: it was sometimes impossible to scroll pages with dashboards because the visualizations captured every touch event.
- Fix: Visualizations on small screens would not always show horizontal scroll bars.
- Dashboards can now be un-archived using the API.
- Dashboard UI performance was improved.
- List pages were changed to show a user's name instead of avatar.
- Search-enabled tables now show a prompt for which columns will be searched.
- In the visualization editor, the settings pane now scrolls independent of the visualization preview.
- Tokens in the schema viewer now sort alphabetically.
- Links to settings panes that require Admin privileges are now hidden from non-Admins.
- The Admin page now remembers which tab you were viewing after a page reload.
### Visualizations
- Feature: Allow bubble size control with either coefficient or sizemode.
- Feature: Table visualization now treats Unix timestamps in query results as timestamps.
- Feature: It's now possible to provide a description to each Table column, appearing in UI as a tooltip.
- Feature: Added tooltip and popover templating to the map with markers visualization.
- Feature: Added an organization setting to hide the Plotly mode bar on all visualizations.
- Feature: Cohort visualization now has appearance settings.
- Feature: Add option to explicitly set Chart legend position.
- Change: Deprecated visualizations are now hidden.
- Change: Table settings editor now extends vertically instead of horizontally.
- Change: The maximum table pagination is now 500.
- Change: Pie chart labels maintain contrast against lighter slices.
- Fix: Chart series switched places when picking Y axis.
- Fix: Third column was not selectable for Bubble and Heatmap charts.
- Fix: On the counter visualizations, the “count rows” option showed an empty string instead of 0.
- Fix: Table visualization with column named "children" rendered +/- buttons.
- Fix: Sankey visualization now correctly occupies all available area even with fewer stages.
- Fix: Pie chart ignores series labels.
### Data Sources
- New Data Sources: Amazon Cloudwatch, Amazon CloudWatch Logs Insights, Azure Kusto, Exasol.
- Athena:
- Added the option to specify a base cost in settings, displaying a price for each query when executed.
- BigQuery:
- Fix: large jobs continued running after the user clicked “Cancel” query execution.
- Cassandra:
- Updated driver to 3.21.0 which dramatically reduces Docker build times.
- SSL options are now available.
- Clickhouse:
- You can now choose whether to verify the SSL certificate.
- Databricks:
- Databricks now use an ODBC-based connector.
- Fix: Date column was coerced to DateTime in the front-end.
- Druid:
- Added username and password authentication option.
- Microsoft SQL Server
- Added support for ODBC connections via pyodbc. There are now two MSSQL data source types. One using TDS. The other is using ODBC.
- MongoDB:
- Added support for running queries on secondary in replicaset mode.
- Fix: Connection test always succeeded.
- Oracle:
- Fix: Connection would fail if username or password contained special characters.
- Fix: Comparisons would fail if scale was None.
- RDS:
- Updated rds-combined-ca-bundle.pem to the latest CA.
- Redshift:
- Added the ability to use IAM Roles and Users.
- Fix: Redshift was unable to have its schema refreshed.
- Rockset:
- Fix: Allow Redash to load collections in all workspaces.
- Snowflake:
- You can now refresh the snowflake schema without waking the cluster.
- Added support for all of Snowflake’s datetime types. Otherwise certain timestamps would only appear as strings in the front-end.
- TreasureData:
- Fix: API calls would fail when setting a non-default region.
### Alerts
- Feature: Added ability to mute alerts without deleting them.
- Fix: numerical comparisons failed if value from query was a string.
### Parameters
- Added Last x Days options for date range parameters.
- Fix: Parameters added in empty queries were always added as text parameters
### Bug Fixes
- Fix: Alembic migration schema was preventing v4 users from upgrading. In v5 we started encrypting data source credentials in the database.
- Fix: System admin dashboard would not show correct database size if non-default name was used.
- Fix: refresh_queries job would break if any query had a bad schedule object.
- Fix: Orgs with LDAP enabled couldn’t disable password login.
- Fix: SSL mode was sometimes sent as an empty string to the database instead of omitted entirely.
- Fix: When creating new Map visualization with clustering disabled, map would crash on save.
- Fix: It was possible on the New Query page to click “Save” multiple times, causing multiple new query records to be created.
- Fix: Visualization render errors on a dashboard would crash the entire page.
- Fix: A scheduled execution failure would modify the query’s “updated_at” timestamp.
- Fix: Parameter UI would wrap awkwardly during some drag operations.
- Fix: In dashboard edit mode, users couldn’t modify widgets.
- Fix: Frontend error when parsing a NaN float.
### Other
- Added TSV as a download format (in addition to CSV and Excel).
- Added maildev settings (helps with automated settings).
- Refine permissions usage in Redash to allow for guest users
- The query results API now explicitly handles 404 errors.
- Forked queries now retain the tags of the original query.
- We now allow setting custom Sentry environments.
- Started using Black linter for our Python source code
- Added CLI command to re-encrypt data source details with new secret key.
- Favorites list is now loaded on menu click instead of on page load.
- Administrators can now allow connections to private IP addresses.
## v8.0.0 - 2019-10-27
There were no changes in this release since `v8.0.0-beta.2`. This is just to mark a stable release.
## v8.0.0-beta.2 - 2019-09-16
This is an update to the previous beta release, which includes:
- Add options for users to share anonymous usage information with us (see [docs](https://redash.io/help/open-source/admin-guide/usage-data) for details).
- Visualizations:
- Allow the user to decide how to handle null values in charts.
- Upgrade Sentry-SDK to latest version.
- Make horizontal table scroll visible in dashboard widgets without scrolling.
- Data Sources:
- Add support for Azure Data Explorer (Kusto).
- MySQL: fix connections without SSL configuration failing.
- Amazon Redshift: option to set query group for adhoc/scheduled queries.
- Hive: make error message more friendly.
- Qubole: add support to run Quantum queries.
- Display data source icon in query editor.
- Fix: allow users with view only acces to use the queries in Query Results
- Dashboard: when updating parameters refersh only widgets that use those parameters.
This release had contributions from 12 people: @arikfr, @cclauss, @gabrieldutra, @justinclift, @kravets-levko, @ranbena, @rauchy, @sandeepV2, @shinsuke-nara, @spacentropy, @sphenlee, @swfz.
## v8.0.0-beta - 2019-08-18
After months of being heads down with hard work, it's finally time to wrap up the V8 release 🤩 This release includes many long awaited improvements to parameters, UX improvements, further React migration and other changes, fixes and improvements.
While this version is already running on the hosted platform to make sure it's stable, we're excited to put this in the hands of our Open Source users.
Starting from this release we will no longer build a tarball distribution of the codebase and recommend everyone to switch over to using our Docker images. We're planning on dropping Python 2 support towards its EOL this year and switching over to the Docker image will make this transition much simpler.
This release was made possible by contributions from over 40 people: @aidarbek, @AntonZarutsky, @ariarijp, @arikfr, @combineads, @deecay, @fmy, @gabrieldutra, @guwenqing, @guyco33, @ialeinikov, @Jakdaw, @jezdez, @justinclift, @k-tomoyasu, @katty0324, @koooge, @kravets-levko, @ktmud, @KumanoTanaka, @kyoshidajp, @nason, @oldPadavan, @openjck, @osule, @otsaloma, @ranbena, @rauchy, @rueian, @sekiyama58, @shinsuke-nara, @taminif, @The-Alchemist, @vv-p, @washort, @wudi-ayuan, @ygrishaev, @yoavbls, @yoshiken, @yusukegoto and the support of over 500 organizations who subscribed to our hosted version and by that sponsor the team's work.
### Parameters
- Parameter UI improvements:
- Support for multi-select in dropdown (and query dropdown) parameters.
- Support for dynamic values in date and date-range parameters.
- Search dropdown parameter values.
- New UX for applying parameter changes in queries and dashboards.
- Allow using Safe Parameters in visualization embeds and public dashboards. Safe Parameters are any parameter type except for the a text parameter (dropdowns are safe).
### Data Sources
- New Data Sources: Couchbase, Phoenix and Dgraph.
- New JSON data source (and deprecated old URL data source).
- Snowflake: update connector to latest version.
- PostgreSQL: show only accessible tables in schema.
- BigQuery:
- Correctly handle NaN values.
- Treat repeated fields as rrays.
- [BigQuery] Fix: in some queries there is no mode field
- DynamoDB:
- Support for Unicode in queries.
- Safe loading of schema.
- Rockset: better handling of query errors.
- Google Sheets:
- Support for Team Drive.
- Friendlier error message in case of an API error and more reliable test connection.
- MySQL:
- Support for calling Stored Procedures and better handling of query cancellation.
- Switch to using `mysqlclient` (a maintained fork of `Python-MySQL`).
- MongoDB: Support serializing Decimal128 values.
- Presto: support for passwords in connection settings.
- Amazon Athena: allow to specify custom work group.
- Query Results: querying a column with a dictionary or array fails
- Clickhouse: make sure we don't show password in error messages.
- Enable Cassandra support by default.
### Visualizations
- Charts:
- Fix: legend overlapping chart on small screens.
- Fix: Pie chart not rendering when series doesn't exist in options.
- Pie Chart: add option to set direction of slices.
- WordCloud: rewritten to support new options (provide frequency in query, limits), scale when resizing, handle long words and more.
- Pivot Table: support hiding totals.
- Counters: apply formatting to target value.
- Maps:
- Ability to customize marker icon and color.
- Customization options for Choropleth maps.
- New Visualization: Details View.
### **UX**
- Replace blank screen with a loading indicator when the application is doing its first load.
- Multiple improvements to dashboards editing: auto-save, grid markings and better refresh indicator.
- Admin can now edit user's groups from the user page.
- Add keyboard shortcut (Ctrl/Cmd+Shift+F) to trigger query formatting.
### API
- Query Result API response minimized to only required fields when called with a non user API key.
- Prefer API key over cookies in authentication.
- User can now regenerate Query API Key.
### Other Changes
- Sends CSP headers to prevent various kinds of security attacks via the browser. Might break unusual usages and embeds of Redash.
- New Failed Scheduled Queries email report (can be enabled from organization settings screen).
- Deprecated HipChat Alert Destination.
- Add options to hide different parts of a Visualization embed UI (parameters, title, link to query).
- Support multi-byte search for query names and descriptions (needs to be enabled in Organization settings screen).
- CSV query results download: correctly serialize booleans and date values.
- Dashboard filters now collect values from all widgets with the same filter.
- Support for custom message and description in alert notifications (currently disabled behind a feature flag until we improve the alert UX).
### Bug Fixes
- Fix: adding widget to dashboard from a query page is broken.
- Fix: default time format option was wrong.
- Fix: when too many errors of a scheduled queries occur it causes an OverflowError.
- Fix: when forking a query maintain the same visualizations order.
## v7.0.0 - 2019-03-17
We're trying a new format for the CHANGELOG in this release. Focusing on the bigger changes, but for whoever interested, you can see all the changes [here](https://github.com/getredash/redash/compare/v6.0.0...master).
@@ -46,8 +46,8 @@ When creating a new bug report, please make sure to:
If you would like to suggest an enhancement or ask for a new feature:
- Please check [the roadmap](https://trello.com/b/b2LUHU7A/redash-roadmap) for existing Trello card for what you want to suggest/ask. If there is, feel free to upvote it to signal interest or add your comments.
- If there is no existing card, open a thread in [the forum](https://discuss.redash.io/c/feature-requests) to start a discussion about what you want to suggest. Try to provide as much details and context as possible and include information about *the problem you want to solve* rather only *your proposed solution*.
- Please check [the forum](https://discuss.redash.io/c/feature-requests/5) for existing threads about what you want to suggest/ask. If there is, feel free to upvote it to signal interest or add your comments.
- If there is no open thread, you're welcome to start one to have a discussion about what you want to suggest. Try to provide as much details and context as possible and include information about *the problem you want to solve* rather only *your proposed solution*.
### Pull Requests
@@ -55,9 +55,9 @@ If you would like to suggest an enhancement or ask for a new feature:
- Include screenshots and animated GIFs in your pull request whenever possible.
- Please add [documentation](#documentation) for new features or changes in functionality along with the code.
- Please follow existing code style:
- Python: we use PEP8 for Python.
- Javascript: we use Airbnb's style guides for [JavaScript](https://github.com/airbnb/javascript#naming-conventions) and [React](https://github.com/airbnb/javascript/blob/master/react) (currently we don't follow Airbnb's convention for naming files, but we're gradually fixing this). To make it automatic and easy, we recommend using [Prettier](https://github.com/prettier/prettier).
- Python: we use [Black](https://github.com/psf/black) to auto format the code.
- Javascript: we use [Prettier](https://github.com/prettier/prettier) to auto-format the code.
### Documentation
The project's documentation can be found at [https://redash.io/help/](https://redash.io/help/). The [documentation sources](https://github.com/getredash/website/tree/master/src/pages/kb) are hosted on GitHub. To contribute edits / new pages, you can use GitHub's interface. Click the "Edit on GitHub" link on the documentation page to quickly open the edit interface.
@@ -66,9 +66,9 @@ The project's documentation can be found at [https://redash.io/help/](https://re
### Release Method
We publish a stable release every ~2 months, although the goal is to get to a stable release every month. You can see the change log on [GitHub releases page](https://github.com/getredash/redash/releases).
We publish a stable release every ~3-4 months, although the goal is to get to a stable release every month.
Every build of the master branch updates the latest *RC release*. These releases are usually stable, but might contain regressions and therefore recommended for "advanced users" only.
Every build of the master branch updates the *redash/redash:preview* Docker Image. These releases are usually stable, but might contain regressions and therefore recommended for "advanced users" only.
When we release a new stable release, we also update the *latest* Docker image tag, the EC2 AMIs and GCE images.
**_Redash_** is our take on freeing the data within our company in a way that will better fit our culture and usage patterns.
Redash is designed to enable anyone, regardless of the level of technical sophistication, to harness the power of data big and small. SQL users leverage Redash to explore, query, visualize, and share data fromany data sources. Their work in turn enables anybody in their organization to use the data. Every day, millions of users at thousands of organizations around the world use Redash to develop insights and make data-driven decisions.
Prior to **_Redash_**, we tried to use traditional BI suites and discovered a set of bloated, technically challenged and slow tools/flows. What we were looking for was a more hacker'ish way to look at data, so we built one.
Redash features:
**_Redash_** was built to allow fast and easy access to billions of records, that we process and collect using Amazon Redshift ("petabyte scale data warehouse" that "speaks" PostgreSQL).
Today **_Redash_** has support for querying multiple databases, including: Redshift, Google BigQuery, PostgreSQL, MySQL, Graphite, Presto, Google Spreadsheets, Cloudera Impala, Hive and custom scripts.
**_Redash_** consists of two parts:
1.**Query Editor**: think of [JS Fiddle](https://jsfiddle.net) for SQL queries. It's your way to share data in the organization in an open way, by sharing both the dataset and the query that generated it. This way everyone can peer review not only the resulting dataset but also the process that generated it. Also it's possible to fork it and generate new datasets and reach new insights.
2.**Visualizations and Dashboards**: once you have a dataset, you can create different visualizations out of it, and then combine several visualizations into a single dashboard. Currently Redash supports charts, pivot table, cohorts and [more](https://redash.io/help/user-guide/visualizations/visualization-types).
1.**Browser-based**: Everything in your browser, with a shareable URL.
2.**Ease-of-use**: Become immediately productive with data without the need to master complex software.
3.**Query editor**: Quickly compose SQL and NoSQL queries with a schema browser and auto-complete.
4.**Visualization and dashboards**: Create [beautiful visualizations](https://redash.io/help/user-guide/visualizations/visualization-types) with drag and drop, and combine them into a single dashboard.
5.**Sharing**: Collaborate easily by sharing visualizations and their associated queries, enabling peer review of reports and queries.
6.**Schedule refreshes**: Automatically update your charts and dashboards at regular intervals you define.
7.**Alerts**: Define conditions and be alerted instantly when your data changes.
8.**REST API**: Everything that can be done in the UI is also available through REST API.
9.**Broad support for data sources**: Extensible data source API with native support for a long list of common databases and platforms.
* [Setting up Redash instance](https://redash.io/help/open-source/setup) (includes links to readymade AWS/GCE images).
* [Setting up Redash instance](https://redash.io/help/open-source/setup) (includes links to ready-made AWS/GCE images).
* [Documentation](https://redash.io/help/).
## Supported Data Sources
Redash supports more than 35 [data sources](https://redash.io/help/data-sources/supported-data-sources).
Redash supports more than 35 SQL and NoSQL [data sources](https://redash.io/help/data-sources/supported-data-sources). It can also be extended to support more. Below is a list of built-in sources:
- Amazon Athena
- Amazon DynamoDB
- Amazon Redshift
- Axibase Time Series Database
- Cassandra
- ClickHouse
- CockroachDB
- CSV
- Databricks (Apache Spark)
- DB2 by IBM
- Druid
- Elasticsearch
- Google Analytics
- Google BigQuery
- Google Spreadsheets
- Graphite
- Greenplum
- Hive
- Impala
- InfluxDB
- JIRA
- JSON
- Apache Kylin
- OmniSciDB (Formerly MapD)
- MemSQL
- Microsoft Azure Data Warehouse / Synapse
- Microsoft Azure SQL Database
- Microsoft SQL Server
- MongoDB
- MySQL
- Oracle
- PostgreSQL
- Presto
- Prometheus
- Python
- Qubole
- Rockset
- Salesforce
- ScyllaDB
- Shell Scripts
- Snowflake
- SQLite
- TreasureData
- Vertica
- Yandex AppMetrrica
- Yandex Metrica
## Getting Help
@@ -38,7 +86,7 @@ Redash supports more than 35 [data sources](https://redash.io/help/data-sources/
## Reporting Bugs and Contributing Code
* Want to report a bug or request a feature? Please open [an issue](https://github.com/getredash/redash/issues/new).
* Want to help us build **_Redash_**? Fork the project, edit in a [dev environment](https://redash.io/help-onpremise/dev/guide.html), and make a pull request. We need all the help we can get!
* Want to help us build **_Redash_**? Fork the project, edit in a [dev environment](https://redash.io/help-onpremise/dev/guide.html) and make a pull request. We need all the help we can get!
Please email security@redash.io to report any security vulnerabilities. We will acknowledge receipt of your vulnerability and strive to send you regular updates about our progress. If you're curious about the status of your disclosure please feel free to email us again. If you want to encrypt your disclosure email, you can use [this PGP key](https://keybase.io/arikfr/key.asc).
// This is for using .inline-tags-control in Angular which renders
// a little differently than React (e.g. in Alert.html)
.inline-tags-control .tags-control {
display: inline-block;
}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.