* feat(CLI): add a new update from flow source CLI
* feat(CLI): use the repository instead of the webserver
* feat(CLI): change command name to SyncFromSource
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
By evaluating multiple condition in a separate queue, we serialize their evaluation which avoir races when we compute the outputs for flow triggers.
This is because evaluation is a multi step process: first you get the existing condtion, then you evaluate, then you store the result. As this is not guarded by a lock you must not do it concurrently.
The race can still occurs if muiltiple executors run but this is less probable. A re-implementation would be needed probably in 2.0 for that.
Fixes https://github.com/kestra-io/kestra-ee/issues/4602
the missing token: ${{ secrets.GH_PERSONAL_TOKEN }} is the only difference between this CI and EE CI, so it is probably the right fix
# Conflicts:
# .github/workflows/setversion-tag.yml
* chore(deps): upgrade micronaut-platform to 4.8.0
* fix: http client version enforcement is not needed anymore as docker-java as updated the dependency
* fix: make applicationContext protected in SchedulerPollingTriggerTest
* fix: upgrade docker-java version to 3.5.0
* fix: make some method protected in the AbstractRunnerTest so it can be override
* fix(core): amend misc label-related issues
* re-enabled bulk update of label value
* re-enabled merging flow-execution labels by key
* made duplicated keys rejection readable
* forced multiple validations within `RequestUtils`
* ensured existing labels can be overriden
* added multiple tests validating complex scenarios
BREAKING CHANGE: switched from first to last label value override
BREAKING CHANGE: preventing empty key/value labels
BREAKING CHANGE: preventing whitespace in key
* fix(core): reflect feedback
* Deduplicated a list inside the `Labels` task.
* Worked around label mutation at `Worker`.
* Attempted to deduplicate labels within `Execution` as possible.
* fix(core): remove irrelevant changes
This is due to the fact that we now process the execution queue concurrently so there is a race when counting currently running executions. This can be seen easily using a ForEachItem as it could create tens or hundreds of executions almost instantly leading to almost all those executions started as they would all see 0 executions running...
Using a dedicated execution running queue, as done in EE, would serialize the messages and fix the issue.
However, if using multiple executor instances and concurrency limit = 1, there is a theoretical race as no locks will be done if no execution is running. A max surge of executions could be as high as the number of executor but this race is less probable to happen in real world scenario.
Fixes#10167
When a flow fail and is restarted and contains either a finally or an afterExecution block, those are not resetted so the restart will skip all task and terminate the flow.
The fix will reset the status of those tasks so they are restarted.
Fixes#10155
In case multiple instances of the executor are started, the execution delay loop and the monitoring SLA loop have a risk of duplicate execution resume or execution SLA violation computation.
This could create some race conditions and duplicate execution update.
But this may also risk to create some deadlocks as two instances of the executor may try to lock the same exection to restart it (or fail it due to SLA).
Make ExecutorService responsible for deleting WorkerJobRunning
when a terminated TaskRun is added to an execution.
Changes:
- Remove unecessary read before delete on WorkerJobRunning table.
Close: #9493
* fix(flows): #9319 error when puase with timeout trigger an execution even after it's terminated
* fix(flows): only skip paused flow when execution is terminated
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
For AdditionalPlugin, as we resolves subtypes using isAssignable, we will resolve ourself as a subtype which leads to infinite loop while parsing the schema.
Provide a way for plugins to define a new type of plugins.
To do that, a plugin must provide both an abstract base class that extends AdditionalPlugin and a set of concret classes.
Both the abstract base class and the concrete classes mut be inside the same plugin. This is a limitation that we may work on later by providing, for example, an SPI to add base classes to the application classloader.
When a worker transitition to the TERMINATED_FORCED, the LivenessCoordinator
should wait for termination grace period to ensure
that all in-flight task-runs had time to be completely processed by the executors
Related-to: #8334Fix: #9094
* fix(filters): change label filtering to 'and' instead of 'or'
link to #8489
* test(execution): add test to validate and behavior
* test(execution): fix test
Instead of consuming multiple time the queue, which lead to concurrent queries on the `queues` table, process concurrently via an ExecutorService the messages from the queue.
We dind't process a new batch of messages until the existing one is totally process to be sure we process in FIFO the same execution message.
Also, go back to a poll size of 100 to mitiguate the performance hit due to this change.
Decreasing it from 100 to 50 didn't show any performance hit but should lower the memory consumption now that we process the queue concurrently in the executor.
## BEFORE - pollSize=100
- 10 tx/s: 150ms
- 25 tx/s: 200ms
- 50 tx/s: 300ms
- 75 tx/s: 5.2s
- 100 tx/s: 15s
## AFTER - pollSize=50
- 10 tx/s: 150ms
- 25 tx/s: 200ms
- 50 tx/s: 300ms
- 75 tx/s: 4.8s
- 100 tx/s: 14s
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
When an execution is queued in the JDBC backend, a record is inserted inside the execution_queued table, we must remove this record when we unqeue an execution.
Fixes#8448
There was an issue when a user edited a task from the No Code editor—the validation endpoint was being called excessively, instead of waiting for the user to finish typing. This has now been resolved with this PR.
Closes https://github.com/kestra-io/kestra/issues/7073.
Co-authored-by: MilosPaunovic <paun992@hotmail.com>
If there are multiple tabs opened in a single panel of the new multi panel view, there was no ability to scroll and see the overflowing ones. With changes in this PR, users can now do just that.
Closes https://github.com/kestra-io/kestra/issues/8270.
Changes in this pull request now allow users to move entire panels where ever they want, plus, there are `Move right` and `Move left` options in the context menu.
Closes https://github.com/kestra-io/kestra/issues/8272.
Co-authored-by: MilosPaunovic <paun992@hotmail.com>
When a user clicks the `Skip tutorial` button, it just does that, shuts the tutorial down but leaves the user on the same page. This change will redirect users to `Flow Creation` page if the `Skip Tutorial` button is clicked.
Closes https://github.com/kestra-io/kestra/issues/8326.
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
There was an issue with title color of tasks on the `No Code` editor (which stayed dark no matter the mode selected), but only viewed in `Firefox` browser. That is sorted out with this pull request.
Closes https://github.com/kestra-io/kestra/issues/8327.
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Adds the ability to close all tabs in a single panel with a single click, improving usability and eliminating the need to close each tab individually.
Closes https://github.com/kestra-io/kestra/issues/8273.
In the new multi panel view, if we have `code` tab opened and delete that file from `files` panel, we'll automatically close the opened tab for the file in question.
Closes https://github.com/kestra-io/kestra/issues/8271.
* fix(): avoid running release workflow on releases branch
* feat(): avoid running CI on draft PR
close#4964
* fix(ci): only publish docker image in workflow release if develop branch or specific asked
close#8136
If multiple Executors restart at the same time and there was a not of worker task to resubmit, there was a possible deadlock as the service instance table is selected for update so it can block other executors.
Using skipped lock avoid that and is still correct as other executors can skip the dead instance handling as it was already in process by the first executor.
findById was not changed in this commit as it's not part of the worker task resubmission process.
If we keep the executionId in it, as it's now used to create the forward logger, a new logger will be created for each execution.
This may also fix a memory leak.
* fix(kafka runner): #2709 filter child forEach tasks before merging the output, and add sleep before restarting flows to ensure failure is persisted
* feat(kafka runner): #2709 wait until executions are persisted as Failed in the database before restarting
* fix(runner): put back the sleep instead of the wait
* clean(runner): remove unused variables
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
* chore(ui): passing prefix down to saved search label component
* chore(ui): check if filters have operation field on encoding
* chore(ui): trigger search automatically on choosing the saved item
* feat: better/simpler display for tab movement
* Revert "remove tests to make sure we merge"
This reverts commit a979232f6c.
* fix some tests
* better tests
* fix: make it work even after first move
* fix the blinking
* add a few tests
* one fix for split panel one for size
* fix test ?
* fix: scroll all the way
* make sure tab 4 is visible
* better borders
* fix dem darn tests
* feat(core): #5467 add inheritance for KV in pebble and Get task
* fix(core): #5467 error when namespace don't contain dot
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
Flow revisions create from older Kestra versions may not be linked to their original source.
In such cases, fall back to the generated source approach to enable plugin default injection.
close: #3206
* refactor: move logic out of EditorView.vue and into store flow.js
* fix create mode
* feat: add multi panel flow editor
* missing flow id
* fix helpers tests ?
* restore helpers test
* fix one failing test
* fix: remove non-working stories
* remove size when one does split
* fix duplicated simulation
* fix tests
* try and fix the test
* remove tests to make sure we merge
Add new `subkey` named argument to the pebble function `secret`
to be able to select a specific field inside a secret containing
a JSON value
close: kestra-io/kestra-ee#3200
* refactor: move logic out of EditorView.vue and into store flow.js
* fix create mode
* missing flow id
* fix helpers tests ?
* restore helpers test
* remove unfinished test files
* fix plugin docs
* remove issues on lowcode
Servers announced their start even when their async startup process
was still in progress. This lead to confusing situations - a server
announced a successfull start but in fact it failed just few
moments later.
Note, the log message needs to be ported to EE server impls.
* fix: remove labels from an execution
* feat(test): add test for removing labels from execution
* fix(test): update test for removing labels from execution
* feat: handle system labels and add unit test for system labels
---------
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
This commit adds the option all to the plugin install command
for installing all available public plugins.
Changes:
* add new service class PluginCatalogService
close: #7375
* Update pull-request.yml to skip tests for PRs with 'docs-only'.
Updated the if condition in the frontend and backend jobs to skip tests if 'docs-only' is present in the Pull Request title, body, or labels.
* Update workflow-test.yml to skip tests for commits with 'docs-only'.
Updated the if condition in the frontend and backend jobs to skip tests if 'docs-only' is present in the commit message.
* Apply suggestions from code review
---------
Co-authored-by: Anna Geller <anna.m.geller@gmail.com>
* feat(build): configure heap size to max 50% of available memory
* Apply suggestions from code review
Co-authored-by: brian-mulier-p <bmmulier@hotmail.fr>
---------
Co-authored-by: brian-mulier-p <bmmulier@hotmail.fr>
* fix(flow editor): when opening a new file and coming back to flow, validation + lowcode are broken
* fix: sync flowYaml only onEdit
* fix: files can now be saved again
* fix: make editor saving button ready for better ux
* allow for a bit of time for storybook to launch the inputs story
* add storybook for flow editor
* setup storybook
Changes:
* add new interface PluginManager
* add new CLI for un-installing plugins
* add new option --locally to CLI plugin install
* refactor service for downloading plugins
* refactor PluginController
* move Version util class from EE to OSS
* migrate aether lib to maven-resolver (#915)
part-of: #915
* Add full examples for dayweek, dayweekinmonth and publicholidays conditions
* Update core/src/main/java/io/kestra/plugin/core/condition/DayWeekInMonth.java
---------
Co-authored-by: AJ Emerich <aemerich@kestra.io>
We usually fail fast, but when a DataException is thrown it means the JDBC driver throws an exception with error code 22: data exception.
As the exception is from the data not the database or the network, there is no point of failfast, we throw a QueueException that may or may not be handled gracefully by the call site.
Without this change, the executor state is deleted when the execution is terminated.
Then the last execution message is emitted, which goes again into the queue wich recreates the executor state.
* [feature] add core plugin sanity checks
* feat: namespace files task test
* fix: formatting
* fix: sanity checks to use assert
* fix: core sanity flows
* tests: add runner class to run sanity checks flows
* [refactor] remove request sanity check as not a core task (already in fs plugin)
* [refactor] rename file and namespace for sanity checks
* [refactor] update path
---------
Co-authored-by: Will Russell <wrussell@kestra.io>
Co-authored-by: Will Russell <will@wrussell.co.uk>
Co-authored-by: Mathieu Gabelle <mgabelle@kestra.io>
* ci():
- split in multiples actions
- simplify translation script
- simplify front build
- run test for back/front only when needed
* fix(): token prop
* ui(translation): new word
* fix(): correct usage of paths-filter action
* fix(): first iteration of new main workflow
* chore(translations): auto generate values for languages other than english
---------
Co-authored-by: GitHub Action <actions@github.com>
* feat(translations): add new blueprint labels for flows, apps, and dashboard
* feat(translations): update left menu titles for blueprints
* chore(translations): auto generate values for languages other than english
* chore(translations): auto generate values for languages other than english
---------
Co-authored-by: GitHub Action <actions@github.com>
New search with operation for all /search endpoints :
- define list of operation and fields.
- Add custom parser for filter[field][operation]=value format
- create new Annotation @QueryFilterFormat to bind to specfic pojo
- Implement new abstract repo search method
* chore(ui): creating a flow from the namespace view should use it's ID as the designated value
* chore(ui): generate random flow ID using combination of animal names and numbers
* chore: store the theme in the store
* use the new theme in charts
* use the theme value in more places
* create a useTheme composable
* create the useScheme composable
* restore nodata
* feat(ui): Align the CSS or style configuration so that all documentation components (Docs, plugin docs, blueprints) use the same markdown style.
* Remove alert styling from DocsLayout since it has been handled within Alert.vue using ---ks variables.
* revert Input_Count.
---------
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
* feat(ui): now display an error when SSE failed
close#1303
* chore(translations): auto generate values for languages other than english
---------
Co-authored-by: GitHub Action <actions@github.com>
* refactor(ui): prevent multiple warning in console by adding inheritAttrs properly
* chore(ui): make plugin selector field not clearable
* feat(ui): allow re-ordering of array items
* fix(ui): remove concurrency when limit set to 0
Missing required positional parameters caused just a stacktrace print.
Now such cases print the error message acompanied by command's help.
Examples:
* `./kestra flow namespace update`
* `./kestra flow dot`
* fix(ui): allow creation of multiple tasks from the no code editor
* chore(ui): make input text be of textarea type for resizability
* chore(ui): allow to add task from topology either before or after the target one
* chore: update "cluster" into "instance" in side menu
* chore(translations): auto generate values for languages other than english
---------
Co-authored-by: GitHub Action <actions@github.com>
* update errors file
* feat: add IAM page
* add layout page and empty template
* feat: tenants demo page
* refactor: store context state in vuex
* feat: make docs open automagically
* feat: add missing admin empty pages
* chore: update "cluster" into "instance" in side menu
* feat: add namespace empty pages + some placeholder text
* add custom blueprints empty page
* chore(translations): auto generate values for languages other than english
* fix a bunch of warnings
* fix blueprints title
* feat: add missing link to kestra apps
* chore(translations): auto generate values for languages other than english
* feat: marketing text
* chore(translations): auto generate values for languages other than english
---------
Co-authored-by: GitHub Action <actions@github.com>
Co-authored-by: Anna Geller <anna.m.geller@gmail.com>
* chore(ui): limit the width of no code editor to a third of the full width
* feat(ui): show only required task properties on top level, else under collapse
* feat(ui): Design change on dashboard creation
close#6943
* chore(translations): auto generate values for languages other than english
---------
Co-authored-by: GitHub Action <actions@github.com>
* Update en.json
* chore(translations): auto generate values for languages other than english
---------
Co-authored-by: GitHub Action <actions@github.com>
* chore(cli): improve CLI help messages
* Improved styling to make the help easily readable.
* Fixed various copy&paste issues.
* Improved wording a bit.
* Fix copy&paste
* chore(ui): make sure to check if property exists
* chore(ui): prevent auto-focus of editor fields
* chore(ui): re-order the list of visible fields in the no code editor
* chore(ui): make concurrency field work in the main form
* chore(ui): initial work on passing component panel to store
* feat(ui): improve the one of task section
* chore(ui): change expected prop type to match what is sent
* chore(ui): allow number input fields to span full width
* chore(ui): remove disabled parameter from buttons which don't need it
* fix: add title to TemplatedTask
* Update TemplatedTask.java
* Update core/src/main/java/io/kestra/plugin/core/templating/TemplatedTask.java
---------
Co-authored-by: Will Russell <will@wrussell.co.uk>
Co-authored-by: Anna Geller <anna.m.geller@gmail.com>
* chore(ui): make the task save button span fully
* chore(ui): make the task save button have the proper top margin
* feat(ui): re-work the task array field for the no code editor
* feat(core): validate in editor if subflow with namespace present
* fix(): added test + return all violations instead of only one
---------
Signed-off-by: Aabhas Sao <aabhassao0@gmail.com>
Co-authored-by: YannC <ycoornaert@kestra.io>
* chore(ui): enhance toggle button for switching between `YAML` and `No Code` editors
* chore(ui): remove the export flow button from main section
* chore(ui): remove the unused context menu options
* chore(ui): add margin after the last entry for variables and inputs
* chore(ui): only show save button on task creation if one is selected
* chore(ui): replace plugin default component from input to editor type
* chore(ui): replace retry and outputs component from input to editor type
* chore(ui): move all general properties to the top level
Send a fake "start" event from the Execution following endpoint so that the UI didn't cancell it.
I'm not sure when the UI would cancel the SSE connection but it can ocurs if any of the view that opens an SSE connection are left but no event are received yet.
Sending a fake event immediatly lower the risk of occuring.
The UI only store a reference to the logs SSE when receive the first event.
In case a flow didn't emit any log, or the logs tab is closed before any logs is emitted, the UI will not have any reference to the SSE so the SSE connection would stay alive forever.
Each SSE connection starts a thread via the logs queue, creating a thread leak.
Sending a first "start" event makes sure the UI has a reference to the SSE.
* feat(core-ee); change log record model and don't use opentelemetry anymore
* fix(core): unit test
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
* feat(*): fixes and improvements for custom dashboard
* chore(translations): auto generate values for languages other than english
---------
Co-authored-by: GitHub Action <actions@github.com>
* feat(core-ee): add log shipper first implementation
* fix(core): change state condition on asynchronous test
* fix(core): exit test shouldExitAndKillTheExecution has 3 execution killed
* fix(core): flaky runner test
* clean(core-ee): add documentation and use a list of log shippers in log synch
* fix(core-ee): do log synch tests for each bdd
* clean(test): move the runner test abstraction to core to use it for jdbc and kafka
* fix(test): fix broken tests, and correct an ELS log search bug
* fix(test): flacky tests on kafka runner
* feat(core): add log repository method to have a flux of log entry
* fix(mySql): change log repository fetch asynch
* feat(core): change logShipper to return void
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
* WIP: commit to be edited
* feat(*): finished implementation for custom dashboard in JDBC
* chore: disable metricRepository all() test because of leap year
* chore(): PR review
MapUtils.merge() un-necessary clone the map when there is only one map that is not-null and not-empty which is not needed as the map is not modified but returned immediatly.
### In CSS code
#### Bootstrap
- When we find `var(--bs-white)`:
- if the prop is `background-color` replace it with `--ks-background-card`
- if the prop is `background` replace it with `--ks-background-card`
- When we find `var(--bs-primary)`:
- if the prop is `color` replace it with `--ks-content-link`
- if the prop is `background-color` replace it with `--ks-background-button-primary`
- if the prop is `background` replace it with `--ks-background-button-primary`
- When we find `var(--bs-purple)`:
- if the prop is `color` replace it with `--ks-content-link`
- if the prop is `background-color` replace it with `--ks-background-button-primary`
- if the prop is `background` replace it with `--ks-background-button-primary`
- When we find `var(--bs-body-color)`:
- if the prop is `color` replace it with `--ks-content-primary`
- When we find `var(--bs-code-color)`:
- if the prop is `color` replace it with `--ks-content-id`
- When we find `var(--bs-body-bg)`:
- if the prop is `background-color` replace it with `--ks-background-body`
- if the prop is `background` replace it with `--ks-background-body`
- When we find `var(--bs-card-bg)`:
- if the prop is `background-color` replace it with `--ks-background-card`
- if the prop is `background` replace it with `--ks-background-card`
- When we find `var(--bs-secondary)`:
- if the prop is `color` replace it with `--ks-content-secondary`
- if the prop is `background-color` replace it with `--ks-border-active`
- When we find `var(--bs-tertiary-color)`:
- if the prop is `color` replace it with `--ks-content-tertiary`
- When we find `var(--bs-border-color)`:
- if the prop is `border` replace it with `--ks-border-primary`
- if the prop is `border-color` replace it with `--ks-border-primary`
- if the prop is `border-bottom` replace it with `--ks-border-primary`
- if the prop is `border-right` replace it with `--ks-border-primary`
- When we find `var(--bs-card-color)`:
- if the prop is `color` replace it with `--ks-content-primary`
- When we find `var(--bs-border-secondary-color)`:
- if the prop is `border` replace it with `--ks-border-secondary`
- if the prop is `border-color` replace it with `--ks-border-secondary`
- if the prop is `border-bottom` replace it with `--ks-border-secondary`
- When we find `var(--bs-warning)`:
- if the prop is `color` replace it with `--ks-content-warning`
#### Element Plus
- When we find `var(--el-bg-color)`:
- if the prop is `background-color` replace it with `--ks-background-button-secondary-hover`
- if the prop is `background` replace it with `--ks-background-button-secondary-hover`
- When we find `var(--el-text-primary)`:
- if the prop is `color` replace it with `--ks-content-primary`
- When we find `var(--el-text-color-regular)`:
- if the prop is `color` replace it with `--ks-content-primary`
- When we find `var(--el-color-primary)`:
- if the prop is `border-color` replace it with `--ks-border-active`
- if the prop is `border` replace it with `--ks-border-active`
- if the prop is `box-shadow` replace it with `--ks-border-active`
- if the prop is `color` replace it with `--ks-content-link`
- When we find `var(--el-border-color)`:
- if the prop is `border` replace it with `--ks-border-primary`
- if the prop is `border-color` replace it with `--ks-border-primary`
- if the prop is `border-right` replace it with `--ks-border-primary`
- if the prop is `border-left` replace it with `--ks-border-primary`
- if the prop is `border-bottom` replace it with `--ks-border-primary`
- if the prop is `box-shadow` replace it with `--ks-border-primary`
- When we find `var(--el-border)`:
- if the prop is `border` replace it with `--ks-border-primary`
- if the prop is `border-color` replace it with `--ks-border-primary`
- if the prop is `border-right` replace it with `--ks-border-primary`
- if the prop is `border-left` replace it with `--ks-border-primary`
- When we find `var(--el-color-text-primary)`:
- if the prop is `color` replace it with `--ks-content-link`
- When we find `var(--el-text-color-primary)`:
- if the prop is `color` replace it with `--ks-content-link`
- if the prop is `background` replace it with `--ks-background-body`
- When we find `var(--el-text-color-secondary)`:
- if the prop is `color` replace it with `--ks-content-link`
- When we find `var(--el-color-alert-info)`:
- if the prop is `color` replace it with `--ks-content-information`
- When we find `var(--el-button-bg-color)`:
- if the prop is `background` replace it with `--ks-background-button-secondary`
- if the prop is `background-color` replace it with `--ks-background-button-secondary`
- if the prop is `border` replace it with `--ks-background-button-secondary`
- When we find `var(--el-text-color-disabled)`:
- if the prop is `color` replace it with `--ks-content-inactive`
- When we find `var(--el-color-warning)`:
- if the prop is `color` replace it with `--ks-content-warning`
#### Kestra
- When we find `var(--content-alert)`:
- if the prop is `color` replace it with `--ks-content-alert`
- When we find `var(--card-bg)`:
- if the prop is `background-color` replace it with `--ks-background-card`
- if the prop is `background` replace it with `--ks-background-card`
- When we find `var(--input-bg)`:
- if the prop is `background-color` replace it with `--ks-background-input`
- if the prop is `background` replace it with `--ks-background-input`
- When we find `var(--log-content-error)`:
- if the prop is `color` replace it with `--ks-log-content-error`
- When we find `var(--log-background-error)`:
- if the prop is `background-color` replace it with `--ks-log-background-error`
- if the prop is `border-top` replace it with `--ks-log-background-error`
- When we find `var(--log-border-error)`:
- if the prop is `border` replace it with `--ks-log-border-error`
- if the prop is `border-color` replace it with `--ks-log-border-error`
### In JS code
- When we find `var(--content-color-` in a string interpolation, we replace it with `var(--ks-content-`.
- When we find `var(--border-color-` in a string interpolation, we replace it with `var(--ks-border-`.
- When we find `var(--background-color-` in a string interpolation, we replace it with `var(--ks-background-`.
- When we find `var(--log-content-` in a string interpolation, we replace it with `var(--ks-log-content-`.
Fixes#6243
When we kill an execution that is running a Pause task that didn't have any subtask, we must transition the task run to KILLED immediatly or the executor will process the Pause task and transition it to SUCCESS.
Previously, PagedResult contains ArrayListTotal which is a list with a total in it. But due to that, ArrayListTotal is not recognized as an array for the OpenAPI spec so all the endpoints that use it generates incorrect specification.
Switching to a list fixes the issue.
The serialized form is not impacted.
* feat(tests): don't load all flows if not needed
* feat(tests): don't load all flows if not needed in runner test and runner retry test
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
* add a story for filter labels
* add some typescript love to filter labels
* add a few stories
* update eslint
* revert changes to label
* fix typings
* wait for the setup to finish
* test(runner tests): add logs to track race condition
* feat(tests): add extension to load flows
* clean(tests): remove logs
* feat(tests): move the extension into the right folder
* feat(tests): add new junit extensions, clean tests and now kestraTest can run runner
* clean(tests): small ficxes after merge
* feat(tests): remove @Test from ExecuteFLow
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
Changes:
When a trigger is evaluated for in a back-fill context, we have to make sure
that current-date is strictly after the next execution date for an execution to be eligible.
fix: #6413
Changes:
To safely execute the liveness coordinator task without error
we should wait for the executor to be fully running
part-of: kestra-io/kestra-ee#2492
* fix(core-ee): change Objects.equals for tenant id to prevent NPE
* fix(core): run tests one by one
* fix(core): add a retry on random failing test
* fix(core): create a specific flow to trigger failure tht check on other namespace to prevent collision with other tests
* rollback(core): put testing config back to previous version to handle it in a separate PR
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
* fix(tests): load only required flows for runner tests
* clean(tests): use existing interface to wrap the test execution
* clean(tests): use existing interface to wrap the test execution
* clean(tests): remove flow loader and put the method in the main test class
* rollback(tests): put back condition trigger to the original namespace
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
* add typescript tsconfig
* make progress on typechecking
* fix build process
* update typings
* progress on typescript
* more progress
* restore irrelevant changes
* remove last set of changes
* feat(ui): context docs, default appId with routing
* use the new app api
* add appId copy tooling
* refactor: use docId instead of appId
* rename DocAppId
* make docId box react to click
Two fixes:
- close the queue onFinally and not onComplete and onCancel to take into accunt errors.
- close the queue onFinally in the execution creation as now it is only done on the success path and not even via a Flux lifecycle method
This may fix or improve some incosistent behavior reported by users on the webserver.
* chore(ui): Improvement in Welcome Page.
* Update Welcome.vue | scoped the styling
* fix bad merge
* remove special behavior of navbar on welcome
* finish the welcome page (thank you)
* fix: better adaptive layout
* use container queries and flex for better responsive design
* chore(translations): auto generate values for languages other than english
---------
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
Co-authored-by: Bart Ledoux <bledoux@kestra.io>
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Co-authored-by: GitHub Action <actions@github.com>
* fix(docs): keep use in docs sidebar when clicking in TOC page
* fix: allow open in a new tab and main menu in docs
* woops
* woops I did it again
* fix: context docs link should never move away from page
* remove vue warnings
This devcontainer provides a quick and easy setup for anyone using VSCode to get up and running quickly with this project to start development on either the frontend or backend. It bootstraps a docker container for you to develop inside of without the need to manually setup the environment.
---
## INSTRUCTIONS
### Setup:
Take a look at this guide to get an idea of what the setup is like as this devcontainer setup follows this approach: https://kestra.io/docs/getting-started/contributing
Once you have this repo cloned to your local system, you will need to install the VSCode extension [Remote Development](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack).
Then run the following command from the command palette:
`Dev Containers: Open Folder in Container...` and select your Kestra root folder.
This will then put you inside a docker container ready for development.
NOTE: you'll need to wait for the gradle build to finish and compile Java files but this process should happen automatically within VSCode.
In the meantime, you can move onto the next step...
---
### Development:
- Create a `.env.development.local` file in the `ui` folder and paste the following:
- Navigate into the `ui` folder and run `npm install` to install the dependencies for the frontend project.
- Now go to the `cli/src/main/resources` folder and create a `application-override.yml` file.
Now you have two choices:
`Local mode`:
Runs the Kestra server in local mode which uses a H2 database, so this is the only config you'd need:
```yaml
micronaut:
server:
cors:
enabled:true
configurations:
all:
allowedOrigins:
- http://localhost:5173
```
You can then open a new terminal and run the following command to start the backend server: `./gradlew runLocal`
`Standalone mode`:
Runs in standalone mode which uses Postgres. Make sure to have a local Postgres instance already running on localhost:
```yaml
kestra:
repository:
type:postgres
storage:
type:local
local:
base-path:"/app/storage"
queue:
type:postgres
tasks:
tmp-dir:
path:/tmp/kestra-wd/tmp
anonymous-usage-report:
enabled:false
server:
basic-auth:
enabled:false
datasources:
postgres:
# It is important to note that you must use the "host.docker.internal" host when connecting to a docker container outside of your devcontainer as attempting to use localhost will only point back to this devcontainer.
# We must ignore missing migrations as we may delete the wrong ones or delete those that are not used anymore.
ignore-migration-patterns:"*:missing,*:future"
out-of-order:true
micronaut:
server:
cors:
enabled:true
configurations:
all:
allowedOrigins:
- http://localhost:5173
```
Then add the following settings to the `.vscode/launch.json` file:
```json
{
"version":"0.2.0",
"configurations":[
{
"type":"java",
"name":"Kestra Standalone",
"request":"launch",
"mainClass":"io.kestra.cli.App",
"projectName":"cli",
"args":"server standalone"
}
]
}
```
You can then use the VSCode `Run and Debug` extension to start the Kestra server.
Additionally, if you're doing frontend development, you can run `npm run dev` from the `ui` folder after having the above running (which will provide a backend) to access your application from `localhost:5173`. This has the benefit to watch your changes and hot-reload upon doing frontend changes.
#### Plugins
If you want your plugins to be loaded inside your devcontainer, point the `source` field to a folder containing jars of the plugins you want to embed in the following snippet in `devcontainer.json`:
If you want to commit to GitHub, make sure to navigate to the `~/.ssh` folder and either create a new SSH key or override the existing `id_ed25519` file and paste an existing SSH key from your local machine into this file. You will then need to change the permissions of the file by running: `chmod 600 id_ed25519`. This will allow you to then push to GitHub.
description:'(deprecated) Plugin version window for old Kestra releases using .plugins file (0.22 to 0.24). If omitted, then plugin list will be fetched from the API compatible versions endpoint'
required:false
type:string
default:"[0.22,0.23)"
dry-run:
description:'Dry run mode that will not write or release anything'
- **jdbc-h2,jdbc-mysql,jdbc-postgres:** Add an index on queues.key ([412837952](https://github.com/kestra-io/kestra/commit/412837952))
- **ui:** Ability to hide secret value when typing in secrets ([59cc56186](https://github.com/kestra-io/kestra/commit/59cc56186))
### 🩹 Fixes
- **core:** Default namespace in namespace file ([ffd06a57e](https://github.com/kestra-io/kestra/commit/ffd06a57e))
- **ui:** Prevent infinite loading loop in Namespace KV Store & Secrets pages if there is none ([8fb6e6af2](https://github.com/kestra-io/kestra/commit/8fb6e6af2))
- Add afterExecution to basic.md ([#8126](https://github.com/kestra-io/kestra/pull/8126))
### 🩹 Fixes
- **cli**: prevent FlowUpdatesCommand to crash due to plugin loader ([YannC](https://github.com/kestra-io/kestra/commit/3ce2cdaeb915d98debfb635215d8604abbc869c2))
- **core:** Use a stable flow logger name ([7cfbb91e7](https://github.com/kestra-io/kestra/commit/7cfbb91e7))
- **core:** HttpClient log the URL even if it's a secret" ([ae223c8d7](https://github.com/kestra-io/kestra/commit/ae223c8d7))
- **core:** Mask secrets on log attributes ([94dcba126](https://github.com/kestra-io/kestra/commit/94dcba126))
- **ui:** Keep fetching if filtered kvs & secrets have no elements after fetch ([5757b576e](https://github.com/kestra-io/kestra/commit/5757b576e))
- **gradle:** Windows selfrun.bat ([9cb51ba0e](https://github.com/kestra-io/kestra/commit/9cb51ba0e))
- **jdbc:** Possible deadlock on service instance ([6ead4e63c](https://github.com/kestra-io/kestra/commit/6ead4e63c))
- **cli:** Fix NPE for commands not requiring plugins ([#8212](https://github.com/kestra-io/kestra/pull/8212))
- **core:** Be tolerant of decryption issue ([38f68dae5](https://github.com/kestra-io/kestra/commit/38f68dae5))
- **jdbc:** #8219 unquoted timestamp field breaking query ([#8222](https://github.com/kestra-io/kestra/pull/8222), [#8219](https://github.com/kestra-io/kestra/issues/8219))
### 🏡 Chore
- **version:** Update the CHANGELOG.MD with details for 0.22.0 ([b78748ebf](https://github.com/kestra-io/kestra/commit/b78748ebf))
- **ci:** Modify publish docker to align on EE ([fedbffbdf](https://github.com/kestra-io/kestra/commit/fedbffbdf))
- **ci:** Align plugins handle for docker publish on EE CI ([632c5836d](https://github.com/kestra-io/kestra/commit/632c5836d))
- ae223c8: Revert "fix(core): HttpClient log the URL even if it's a secret" (Loïc Mathieu)
- Upgrade to 0.22.1 ([9857930da](https://github.com/kestra-io/kestra/commit/9857930da))
- **core,jdbc:** Switch back to a cached thread pool for async JDBC queue ([c2e62d653](https://github.com/kestra-io/kestra/commit/c2e62d653))
- **build:** Configure heap size to max 50% of available memory ([#7800](https://github.com/kestra-io/kestra/pull/7800))
- **core:** Redact KESTRA_JAVA_OPTS from the env available to executions ([ecfe925ec](https://github.com/kestra-io/kestra/commit/ecfe925ec))
- Add finally to the flow assets in basic.md ([#7857](https://github.com/kestra-io/kestra/pull/7857))
- **core:** Add afterExecution to the topology ([f7019af9d](https://github.com/kestra-io/kestra/commit/f7019af9d))
- **ui:** Add `afterExecution` block to no code editor ([#7848](https://github.com/kestra-io/kestra/pull/7848))
- **cli:** Add new --all option to plugin install cmd ([#7375](https://github.com/kestra-io/kestra/pull/7375))
- **core:** Add Kestra env name and URL to the expression context ([1ba54cd08](https://github.com/kestra-io/kestra/commit/1ba54cd08))
- **webserver:** Add Kestra URL to the config endpoint ([70e6d47c1](https://github.com/kestra-io/kestra/commit/70e6d47c1))
- **ui:** Add copy button to kv store listing rows ([#7907](https://github.com/kestra-io/kestra/pull/7907))
- **ui:** Implement a default tab setting for flows ([#7917](https://github.com/kestra-io/kestra/pull/7917))
- ⚠️ Make kv pebble function raise error by default ([#7855](https://github.com/kestra-io/kestra/pull/7855))
- **ui:** Add the option to copy single/all logs to clipboard ([#7755](https://github.com/kestra-io/kestra/pull/7755))
- **ui:** Add beta badge global component ([#7934](https://github.com/kestra-io/kestra/pull/7934))
- **model, core:** Document tasks props that are internal storage URI ([745c64c4b](https://github.com/kestra-io/kestra/commit/745c64c4b))
- **core:** Add correlationId to the Flow trigger ([a7433c6f6](https://github.com/kestra-io/kestra/commit/a7433c6f6))
- ***:** Allow disabling flow logs and use a specific logger for executions, tasks and triggers ([11a166486](https://github.com/kestra-io/kestra/commit/11a166486))
- **core:** Add new subkey arg to secret pebble function ([0987d0b34](https://github.com/kestra-io/kestra/commit/0987d0b34))
- Multi panel editor without the refactor ([#7971](https://github.com/kestra-io/kestra/pull/7971))
- Parse docs is 2 steps to avoid user waiting with nothing ([#7149](https://github.com/kestra-io/kestra/pull/7149))
- **core,jdbc:** Reset the trigger inside the JdbcExecutor ([4a3d6b30d](https://github.com/kestra-io/kestra/commit/4a3d6b30d))
- **ui:** Introduce global Secrets page ([d9ac26716](https://github.com/kestra-io/kestra/commit/d9ac26716))
- **docs:** Add example for best practice with multiline json http post request ([#8023](https://github.com/kestra-io/kestra/pull/8023))
- **core:** #5467 add inheritance for KV in pebble and Get task ([#8031](https://github.com/kestra-io/kestra/pull/8031), [#5467](https://github.com/kestra-io/kestra/issues/5467))
- **cicd:** Add npm install on vulnerabilities check ([9790f0237](https://github.com/kestra-io/kestra/commit/9790f0237))
- **ci:** Workflow test ouptputs + changes for codecov ([c3e830c2c](https://github.com/kestra-io/kestra/commit/c3e830c2c))
- **ci:** Inputs instead of secrets ([c53239470](https://github.com/kestra-io/kestra/commit/c53239470))
- **core:** #7227 cron schedule with timezone and backfile not triggering ([#7285](https://github.com/kestra-io/kestra/pull/7285), [#7227](https://github.com/kestra-io/kestra/issues/7227))
- Force run docs ([#7289](https://github.com/kestra-io/kestra/pull/7289))
- Collapsed menu colors in light mode ([0becf7433](https://github.com/kestra-io/kestra/commit/0becf7433))
- **ui:** Match chart colors ([#7290](https://github.com/kestra-io/kestra/pull/7290))
- **makefile:** Build plugin now build main branch too ([#7297](https://github.com/kestra-io/kestra/pull/7297))
- **core:** Flacky trigger with backfile test ([#7295](https://github.com/kestra-io/kestra/pull/7295))
- **core:** Http client was not using deprecated setter ([25370d10b](https://github.com/kestra-io/kestra/commit/25370d10b))
- **core:** Do not validate subflow if namespace or id is pebble ([#7294](https://github.com/kestra-io/kestra/pull/7294))
- **h2:** Remove indenting in sql file ([#7306](https://github.com/kestra-io/kestra/pull/7306))
- **core:** Add request.yaml file back ([#7308](https://github.com/kestra-io/kestra/pull/7308))
- **core:** Possible NPE when an execution has no labels ([7dbf86d54](https://github.com/kestra-io/kestra/commit/7dbf86d54))
- **cicd:** Npm install in the wrong folder for vulnerabilities checks ([7bf42cb1c](https://github.com/kestra-io/kestra/commit/7bf42cb1c))
- Add proper ellipsis to sidemenu ([#7361](https://github.com/kestra-io/kestra/pull/7361))
- Trim bookmarks better ([#7359](https://github.com/kestra-io/kestra/pull/7359))
- **ui:** Make sure bulk selection is taking into account only selected items ([#7362](https://github.com/kestra-io/kestra/pull/7362))
- Make menu hierarchy get closer to the original designs ([#7102](https://github.com/kestra-io/kestra/pull/7102))
- **core:** Render list ([b45a44bd3](https://github.com/kestra-io/kestra/commit/b45a44bd3))
- **jdbc:** Delete the executor state at the correct stage ([1d65fd96b](https://github.com/kestra-io/kestra/commit/1d65fd96b))
- **ui:** Refresh dashboard list ([#7370](https://github.com/kestra-io/kestra/pull/7370))
- **core:** Handle http request with no content type ([239fb6a68](https://github.com/kestra-io/kestra/commit/239fb6a68))
- **cicd:** Add mariadb plugins on docker image ([5b29a0d07](https://github.com/kestra-io/kestra/commit/5b29a0d07))
- **scheduler:** Delete trigger when flow is not found ([#7366](https://github.com/kestra-io/kestra/pull/7366))
- **cli:** Disable by default OTEL metrics ([def8fa3ff](https://github.com/kestra-io/kestra/commit/def8fa3ff))
- Changing language should work with providers too ([b8d0ae3ec](https://github.com/kestra-io/kestra/commit/b8d0ae3ec))
- **ui:** Fix slack button on error toast ([ad651cdc5](https://github.com/kestra-io/kestra/commit/ad651cdc5))
- **ui:** Properly handle the operation labels in filter component ([#7399](https://github.com/kestra-io/kestra/pull/7399))
- **core:** Taskrun list can be null ([ddfed2e65](https://github.com/kestra-io/kestra/commit/ddfed2e65))
- **ui:** Correct english translations ([#7401](https://github.com/kestra-io/kestra/pull/7401))
- **core:** ForEachItem inside an If task ([d9d2f8697](https://github.com/kestra-io/kestra/commit/d9d2f8697))
- **test:** Attempt at making the test not flakky ([#7400](https://github.com/kestra-io/kestra/pull/7400))
- Enable rendering of commands properties inside CommandsWrapper ([#7381](https://github.com/kestra-io/kestra/pull/7381))
- Restore red dot when there is news ([fbd893434](https://github.com/kestra-io/kestra/commit/fbd893434))
- **tests:** Wider maxDuration for retry-failed-flow-duration.yml ([d55ce16f5](https://github.com/kestra-io/kestra/commit/d55ce16f5))
- **ui:** Better duration consistency on Gantt chart ([4a55485cd](https://github.com/kestra-io/kestra/commit/4a55485cd))
- **tests:** Logs are asynchronously inserted so we wait for them to be fully in ([13cb0fb96](https://github.com/kestra-io/kestra/commit/13cb0fb96))
- **ui:** Improve modifying inputs from no code editor ([#7440](https://github.com/kestra-io/kestra/pull/7440))
- **core:** Provide tenantId when looking for subflow ([#7442](https://github.com/kestra-io/kestra/pull/7442))
- **core:** Move back to the old worker thread pool because it was restricting it to 1 thread ([372327581](https://github.com/kestra-io/kestra/commit/372327581))
- **core:** Remove props with default from `required` in json schema to avoid validation errors ([15b85ac95](https://github.com/kestra-io/kestra/commit/15b85ac95))
- **core:** Render `delete` property at the beginning in Docker task runner ([16e3830c9](https://github.com/kestra-io/kestra/commit/16e3830c9))
- **tests:** Increase timeout on JdbcServiceLivenessCoordinatorTest.taskResubmitSkipExecution ([cebe8f354](https://github.com/kestra-io/kestra/commit/cebe8f354))
- **ui:** Executions naviation based on start Date. ([#7626](https://github.com/kestra-io/kestra/pull/7626))
- **core:** No longer lowercasing PluginClassIdentifier to have proper validation upon Plugin deserialization ([17e54134c](https://github.com/kestra-io/kestra/commit/17e54134c))
- **ui:** Additional check for text label of filters section ([446a034d6](https://github.com/kestra-io/kestra/commit/446a034d6))
- **ui:** Improve check for text label of filters section ([036a7cf4f](https://github.com/kestra-io/kestra/commit/036a7cf4f))
- **ui:** Use watch with ref instead of accessing the value ([5f21eb579](https://github.com/kestra-io/kestra/commit/5f21eb579))
- **ui:** Allow sidebar theme toggle to update the editor theme ([#7648](https://github.com/kestra-io/kestra/pull/7648))
- **ui:** Prevent context docs open on editor custom blueprints click ([#7716](https://github.com/kestra-io/kestra/pull/7716))
- **runner-memory:** Delete MemorySchedulerTriggerState back due to cherry-pick ([593558dd2](https://github.com/kestra-io/kestra/commit/593558dd2))
- **ui:** Improved fetch of type for pluginDoc and avoid removing doc if map has "type" as property but without doc (like ENUM value) ([#7727](https://github.com/kestra-io/kestra/pull/7727))
- **demo:** On pages stop showing the docs without a button ([07e4598fa](https://github.com/kestra-io/kestra/commit/07e4598fa))
- Repair collapsed menu submenus ([467861652](https://github.com/kestra-io/kestra/commit/467861652))
- **flow editor:** Enhance behavior when switching file tabs ([#7722](https://github.com/kestra-io/kestra/pull/7722))
- **tests:** Reject promise with 404 instead of empty resolve if non-mocked store call in flowAutoCompletionProvider.spec.ts ([166262209](https://github.com/kestra-io/kestra/commit/166262209))
- **jdbc:** Resubmit worker job to the good worker group ([7696d41d5](https://github.com/kestra-io/kestra/commit/7696d41d5))
- **ci:** Generate_translations.py is now deleting keys that are no longer in en translation to avoid ghost translations ([440a94290](https://github.com/kestra-io/kestra/commit/440a94290))
- **core:** ThresholdFilter is now stricly lower" ([4276a0afd](https://github.com/kestra-io/kestra/commit/4276a0afd))
- **core:** Failing schedule test ([#7783](https://github.com/kestra-io/kestra/pull/7783))
- **core:** Wait for service-manager-task thread to be stopped ([01036c829](https://github.com/kestra-io/kestra/commit/01036c829))
- **tests:** Increase seconds diff between dates ([#7785](https://github.com/kestra-io/kestra/pull/7785))
- **core:** #7740 http configuration bearer token may change to basic because of allowFailed ([#7788](https://github.com/kestra-io/kestra/pull/7788), [#7740](https://github.com/kestra-io/kestra/issues/7740))
- **jdbc:** Return correct total when paginating custom dashboard chart ([#7790](https://github.com/kestra-io/kestra/pull/7790))
- **webserver:** Add endpoint for inherited secrets ([9b5b2b981](https://github.com/kestra-io/kestra/commit/9b5b2b981))
- **ui:** Make switch view buttons from dashboard editor the same as flow editor ones ([82a346b2c](https://github.com/kestra-io/kestra/commit/82a346b2c))
- **ui:** Remove errors from dashboard validation if it's fixed ([4a1282768](https://github.com/kestra-io/kestra/commit/4a1282768))
- **platform:** Move slf4j api to enforce platform to fix it's version in test ([#8007](https://github.com/kestra-io/kestra/pull/8007))
- **core:** Handling for trailing slash in the KESTRA_URL configuration ([#6373](https://github.com/kestra-io/kestra/pull/6373))
- **ui:** Prevent function parameters autocompletion from deleting parenthesis ([74455ad99](https://github.com/kestra-io/kestra/commit/74455ad99))
- **ui:** Handle properly layout of global Secrets when there is a secret manager ([edbf14c1b](https://github.com/kestra-io/kestra/commit/edbf14c1b))
- **core:** Ensure defaults can be injected in flows ([#3206](https://github.com/kestra-io/kestra/pull/3206))
- **ui:** Allow multi label filtering ([#8022](https://github.com/kestra-io/kestra/pull/8022))
- **ui:** Fail-safe secrets API calls ([c64c2c710](https://github.com/kestra-io/kestra/commit/c64c2c710))
- **ui:** Amend operator value of labels inside the filter ([#8028](https://github.com/kestra-io/kestra/pull/8028))
- **ui:** Fail-safe secrets API calls on global secrets view ([28d1f005a](https://github.com/kestra-io/kestra/commit/28d1f005a))
- **core:** Fix NPE when closing standalone runner ([6c9dc8fba](https://github.com/kestra-io/kestra/commit/6c9dc8fba))
- **ui:** Make sure global secret view iterates over all secrets ([75e763550](https://github.com/kestra-io/kestra/commit/75e763550))
- **cli:** Make worker args available through static KestraContext ([dea66ca25](https://github.com/kestra-io/kestra/commit/dea66ca25))
- **core:** Flatten map should not throw an exception ([4c93a2b0e](https://github.com/kestra-io/kestra/commit/4c93a2b0e))
- **webserver:** First eval without masking secret function to error in case of missing secret ([8f4ce5fc1](https://github.com/kestra-io/kestra/commit/8f4ce5fc1))
- **ui:** Properly detect yaml to inject json schema into MonacoEditor ([8be17827c](https://github.com/kestra-io/kestra/commit/8be17827c))
- **core:** HttpClient log the URL even if it's a secret ([54aa93570](https://github.com/kestra-io/kestra/commit/54aa93570))
- **core:** Properly fix the issue with MapUtils.flattenToNestedMap ([b8e8333f6](https://github.com/kestra-io/kestra/commit/b8e8333f6))
- **translations:** Standalone action for translations ([#7597](https://github.com/kestra-io/kestra/pull/7597))
- **translations:** Localize to languages other than English ([#7605](https://github.com/kestra-io/kestra/pull/7605))
- **translations:** Localize to languages other than English " ([#7605](https://github.com/kestra-io/kestra/pull/7605), [#7609](https://github.com/kestra-io/kestra/pull/7609))
- **translations:** Localize to languages other than English ([#7610](https://github.com/kestra-io/kestra/pull/7610))
- **translations:** Localize to languages other than English ([#7613](https://github.com/kestra-io/kestra/pull/7613))
- **translations:** Localize to languages other than English ([#7618](https://github.com/kestra-io/kestra/pull/7618))
- **ui:** Improve label for text search in filters section ([#7631](https://github.com/kestra-io/kestra/pull/7631))
- **translations:** Localize to languages other than English ([#7633](https://github.com/kestra-io/kestra/pull/7633))
- **ui:** Remove crud details from execution overview ([#7634](https://github.com/kestra-io/kestra/pull/7634))
- **ui:** Improve empty state of the namespace files editor ([#7495](https://github.com/kestra-io/kestra/pull/7495))
- **translations:** Localize to languages other than English ([#7635](https://github.com/kestra-io/kestra/pull/7635))
- **core:** Eval value property once for flowable task Switch ([bfd82e0b5](https://github.com/kestra-io/kestra/commit/bfd82e0b5))
- **ui:** Improve the topology tooltip label for adding task button ([#7656](https://github.com/kestra-io/kestra/pull/7656))
- **ui:** Properly sanitize markdown content before rendering ([#7662](https://github.com/kestra-io/kestra/pull/7662))
- **ui:** Make sure chart stacks are following the same order every time ([#7664](https://github.com/kestra-io/kestra/pull/7664))
- **ui:** Properly sanitize markdown content before rendering ([#7697](https://github.com/kestra-io/kestra/pull/7697))
- **ui:** Auto expand first element in execution overview cascaders ([#7715](https://github.com/kestra-io/kestra/pull/7715))
- **core:** Make registry unregister usable with immutable list ([9a56b763f](https://github.com/kestra-io/kestra/commit/9a56b763f))
- **ui:** Properly sanitize markdown content before rendering ([#7724](https://github.com/kestra-io/kestra/pull/7724))
- **ui:** Uniforming empty state for components ([#7737](https://github.com/kestra-io/kestra/pull/7737))
- **translations:** Localize to languages other than English ([#7739](https://github.com/kestra-io/kestra/pull/7739))
- **ui:** Use uniformed pagination component for custom dashboard tables ([#7744](https://github.com/kestra-io/kestra/pull/7744))
- **translations:** Localize to languages other than English ([#7746](https://github.com/kestra-io/kestra/pull/7746))
- **ui:** Remove single empty space between label key and value so it can be copied ([#7774](https://github.com/kestra-io/kestra/pull/7774))
- **docker:** Fixed version of qemu ([49fe36250](https://github.com/kestra-io/kestra/commit/49fe36250))
- **docker:** Revert fixed qemu version ([cfc0c9f9f](https://github.com/kestra-io/kestra/commit/cfc0c9f9f))
- **publish-docker:** Attempts with ubuntu 0.20 ([#7431](https://github.com/kestra-io/kestra/pull/7431))
- **publish-docker:** Attempts with command on qemu docker image ([a89ef7158](https://github.com/kestra-io/kestra/commit/a89ef7158))
- **test:** Force test if ref is a tag ([a020e3f3a](https://github.com/kestra-io/kestra/commit/a020e3f3a))
### ⚠️ Breaking Changes
### EE: Default tenant deprecation
Multi-tenancy was introduced in Kestra 0.13. For backward compatibility with older versions (≤0.12), you could use the concept of a default tenant, which imitated the multitenancy feature with the so-called “null”-tenant. One and a half years later, in Kestra 0.22, we are deprecating the default tenant functionality and plan to remove it in the future. We will provide a detailed migration guide for all customers who still use the default tenant. Until then, you can continue using `defaultTenant` by setting the corresponding configuration flag to `true`:
```yaml
kestra:
tenants:
enabled:true
defaultTenant:true
```
Note that in Kestra 0.22 and higher, `defaultTenant` is NOT enabled by default, so you must explicitly set that configuration option to `true` to keep using the default tenant.
Also, keep in mind that prior to Kestra 0.22, `tenants.enabled` was by default set to `false` and now they are enabled.
### EE: Azure log exporter
The log exporter plugin for Azure `io.kestra.plugin.ee.azure.LogExporter`, introduced in Kestra 0.21, got split into two `io.kestra.plugin.ee.azure.monitor.LogExporter` and `io.kestra.plugin.ee.azure.storage.LogExporter` to reflect that you can now export your log to Azure either using Azure Monitor or using Azure Blob Storage.
### EE: Enterprise Edition API changes
Before Kestra 0.22, the Service Account name had to be globally unique within the instance. As a result, attempting to create a Service Account `cicd` in a `dev` tenant would raise an error `"Username already exists"` if your `prod` tenant also has a Service Account with the name `cicd`.
To support multiple service accounts with the same name, we’ve renamed the `username` property to `name` in the JSON payload for the following REST API endpoint: `POST /api/v1{/tenant}/users/service-accounts{/id}`.
### EE: Too many failed login attempts now lock the account
To improve the security of your Enterprise Edition instance, we now automatically lock user accounts after a `threshold` number of failed login attempts made within `monitoring-window`. Both, the number of failed attempts, the monitoring window to track consecutive number of failed attempts and (soon) the duration of how long the user remains locked are configurable.
```yaml
security:
login:
failed-attempts:
threshold:10
monitoring-window:PT15M # period to count failed attempts
# lockout-duration: PT24H # period the account remains locked — will be added in the next release
```
Note that this change is only relevant for users who leverage LDAP or basic authentication (not relevant for SSO-users). Superadmin can unlock the user manually by resetting their password from the user's detail page.
### Change to `readinessProbe` and `livenessProbe`
Before [this PR](https://github.com/kestra-io/helm-charts/pull/62/files), both probes pointed to `/health`. This caused Kubernetes to restart the pod when an external component was unavailable. To resolve this, we updated the value file to configure the liveness and readiness probes to use the health paths recommended by Micronaut:
- Liveness probe now points to `/health/liveness`
- Readiness probe now points to `/health/readiness`.
### Plugins using the `version` property
With the introduction of plugin versioning, we reserve the `version` keyword for internal use, allowing us to specify the Kestra plugin version. As a result, we’ve renamed the `version` property for a few plugins that already used it, incl. the following:
-`io.kestra.plugin.elasticsearch.Get` → renamed as `docVersion`
-`io.kestra.plugin.opensearch.Get` → renamed as `docVersion`
-`io.kestra.plugin.mqtt.RealtimeTrigger` → renamed as `mqttVersion`
-`io.kestra.plugin.mqtt.Trigger` → renamed as `mqttVersion`
-`io.kestra.plugin.serdes.parquet.IonToParquet` → renamed as `parquetVersion`
Note that your **custom plugins** will need an equivalent approach of renaming any plugin that uses the `version` property, as this is now a core property reserved for plugin management. If any of your custom plugins rely on a `version` property, they won't compile anymore unless you rename that property to a different name.
### Change in the default value for the `kv()` function
Before Kestra 0.22, the `kv()` function had the property `errorOnMissing` set to `false` by default. We changed it to be `true` by default. If you want to keep the previous behavior of returning `null` without an error when attempting to fetch non-existing KV-pairs, use the syntax `"{{ kv('NON_EXISTING_KV_PAIR', errorOnMissing=false) }}"`.
<img src="https://kestra.io/startvideo.png" alt="Get started in 4 minutes with Kestra" width="640px" />
@@ -47,7 +54,7 @@ Kestra is an open-source, event-driven orchestration platform that makes both **
- **Structure & Resilience**: tame chaos and bring resilience to your workflows with **namespaces**, **labels**, **subflows**, **retries**, **timeout**, **error handling**, **inputs**, **outputs** that generate artifacts in the UI, **variables**, **conditional branching**, **advanced scheduling**, **event triggers**, **backfills**, **dynamic tasks**, **sequential and parallel tasks**, and skip tasks or triggers when needed by setting the flag `disabled` to `true`.
🧑💻 The YAML definition gets automatically adjusted any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is **always managed declaratively in code**, even if you modify your workflows in other ways (UI, CI/CD, Terraform, API calls).
🧑💻 The YAML definition gets automatically adjusted any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is **always managed declaratively in code**, even if you modify your workflows in other ways (UI, CI/CD, Terraform, API calls).
If you're on Windows and use Command Prompt (CMD):
```cmd
docker run --pull=always --rm -it -p 8080:8080 --user=root ^
-v "/var/run/docker.sock:/var/run/docker.sock"^
-v "C:/Temp:/tmp" kestra/kestra:latest server local
```
If you're on Windows and use WSL (Linux-based environment in Windows):
```bash
docker run --pull=always --rm -it -p 8080:8080 --user=root \
-v "/var/run/docker.sock:/var/run/docker.sock"\
-v "C:/Temp:/tmp" kestra/kestra:latest server local
```
Check our [Installation Guide](https://kestra.io/docs/installation) for other deployment options (Docker Compose, Podman, Kubernetes, AWS, GCP, Azure, and more).
Access the Kestra UI at [http://localhost:8080](http://localhost:8080) and start building your first flow!
@@ -178,7 +206,7 @@ Stay connected and get support:
We welcome contributions of all kinds!
- **Report Issues:** Found a bug or have a feature request? Open an [issue on GitHub](https://github.com/kestra-io/kestra/issues).
- **Contribute Code:** Check out our [Contributor Guide](https://kestra.io/docs/getting-started/contributing) for initial guidelines, and explore our [good first issues](https://go.kestra.io/contribute) for beginner-friendly tasks to tackle first.
- **Contribute Code:** Check out our [Contributor Guide](https://kestra.io/docs/getting-started/contributing) for initial guidelines, and explore our [good first issues](https://go.kestra.io/contributing) for beginner-friendly tasks to tackle first.
- **Develop Plugins:** Build and share plugins using our [Plugin Developer Guide](https://kestra.io/docs/plugin-developer-guide/).
- **Contribute to our Docs:** Contribute edits or updates to keep our [documentation](https://github.com/kestra-io/docs) top-notch.
We provide security updates for the following versions of Kestra:
- The `latest` release
- Up to two previous minor versions released as a backport upon customer request.
If you are using an unsupported version, we recommend upgrading to the `latest` version to receive security fixes.
## Reporting a Vulnerability
If you discover a security vulnerability in Kestra, please report it to us privately to ensure a responsible disclosure process. You can contact our security team at:
**security@kestra.io**
### Guidelines for Reporting
- Provide a detailed description of the issue, including steps to reproduce it if possible.
- Do not disclose the vulnerability publicly until we have confirmed and patched the issue.
- If you believe the issue has critical severity, please indicate so in your report to help us prioritize.
## Our Commitment
- We will acknowledge your report within **2 business days**.
- We will work to verify and address the issue as quickly as possible.
- Once the issue is resolved, we will notify you of the fix.
## Acknowledgments
We are happy to credit those who report vulnerabilities responsibly in our release notes, unless you prefer to remain anonymous. If you would like to be acknowledged, please include this in your report.
@CommandLine.Parameters(index="0",description="the namespace to update")
@CommandLine.Parameters(index="0",description="The namespace to update")
publicStringnamespace;
@CommandLine.Parameters(index="1",description="the key to update")
@CommandLine.Parameters(index="1",description="The key to update")
publicStringkey;
@CommandLine.Parameters(index="2",description="the value to assign to the key. If the value is an object, it must be in JSON format. If the value must be read from file, use -f parameter.")
@CommandLine.Parameters(index="2",description="The value to assign to the key. If the value is an object, it must be in JSON format. If the value must be read from file, use -f parameter.")
publicStringvalue;
@Option(names={"-e","--expiration"},description="the duration after which the key should expire.")
@Option(names={"-e","--expiration"},description="The duration after which the key should expire.")
publicStringexpiration;
@Option(names={"-t","--type"},description="the type of the value. Optional and useful to override the deduced type (eg. numbers, booleans or JSON as full string). Valid values: ${COMPLETION-CANDIDATES}.")
@Option(names={"-t","--type"},description="The type of the value. Optional and useful to override the deduced type (eg. numbers, booleans or JSON as full string). Valid values: ${COMPLETION-CANDIDATES}.")
publicTypetype;
@Option(names={"-f","--file-value"},description="the file from which to read the value to set. If this is provided, it will take precedence over any specified value.")
@Option(names={"-f","--file-value"},description="The file from which to read the value to set. If this is provided, it will take precedence over any specified value.")
@CommandLine.Parameters(index="0..*",description="the plugins to install")
@Option(names={"--locally"},description="Specifies if plugins must be installed locally. If set to false the installation depends on your Kestra configuration.")
booleanlocally=true;
@Option(names={"--all"},description="Install all available plugins")
booleanall=false;
@Parameters(index="0..*",description="Plugins to install. Represented as Maven artifact coordinates (i.e., <groupId>:<artifactId>:(<version>|LATEST)")
List<String>dependencies=newArrayList<>();
@CommandLine.Option(names={"--repositories"},description="url to additional maven repositories")
@Option(names={"--repositories"},description="URL to additional Maven repositories")
@@ -31,22 +28,22 @@ public class ExecutorCommand extends AbstractServerCommand {
@Inject
privateStartExecutorServicestartExecutorService;
@CommandLine.Option(names={"--skip-executions"},split=",",description="a list of execution identifiers to skip, separated by a coma; for troubleshooting purpose only")
@CommandLine.Option(names={"--skip-executions"},split=",",description="The list of execution identifiers to skip, separated by a coma; for troubleshooting purpose only")
@CommandLine.Option(names={"--skip-flows"},split=",",description="a list of flow identifiers (tenant|namespace|flowId) to skip, separated by a coma; for troubleshooting purpose only")
@CommandLine.Option(names={"--skip-flows"},split=",",description="The list of flow identifiers (tenant|namespace|flowId) to skip, separated by a coma; for troubleshooting purpose only")
@CommandLine.Option(names={"--skip-namespaces"},split=",",description="a list of namespace identifiers (tenant|namespace) to skip, separated by a coma; for troubleshooting purpose only")
@CommandLine.Option(names={"--skip-namespaces"},split=",",description="The list of namespace identifiers (tenant|namespace) to skip, separated by a coma; for troubleshooting purpose only")
@CommandLine.Option(names={"--skip-tenants"},split=",",description="a list of tenants to skip, separated by a coma; for troubleshooting purpose only")
@CommandLine.Option(names={"--skip-tenants"},split=",",description="The list of tenants to skip, separated by a coma; for troubleshooting purpose only")
@CommandLine.Option(names={"--start-executors"},split=",",description="a list of Kafka Stream executors to start, separated by a command. Use it only with the Kafka queue, for debugging purpose.")
@CommandLine.Option(names={"--start-executors"},split=",",description="The list of Kafka Stream executors to start, separated by a command. Use it only with the Kafka queue, for debugging purpose.")
@CommandLine.Option(names={"--not-start-executors"},split=",",description="a list of Kafka Stream executors to not start, separated by a command. Use it only with the Kafka queue, for debugging purpose.")
@CommandLine.Option(names={"--not-start-executors"},split=",",description="The list of Kafka Stream executors to not start, separated by a command. Use it only with the Kafka queue, for debugging purpose.")
@CommandLine.Option(names={"--skip-indexer-records"},split=",",description="a list of indexer record keys, separated by a coma; for troubleshooting purpose only")
@@ -46,7 +44,7 @@ public class StandAloneCommand extends AbstractServerCommand {
@CommandLine.Option(names={"-f","--flow-path"},description="the flow path containing flow to inject at startup (when running with a memory flow repository)")
privateFileflowPath;
@CommandLine.Option(names={"--worker-thread"},description="the number of worker threads, defaults to two times the number of available processors. Set it to 0 to avoid starting a worker.")
@CommandLine.Option(names={"--worker-thread"},description="the number of worker threads, defaults to four times the number of available processors. Set it to 0 to avoid starting a worker.")
privateintworkerThread=defaultWorkerThread();
@CommandLine.Option(names={"--skip-executions"},split=",",description="a list of execution identifiers to skip, separated by a coma; for troubleshooting purpose only")
@@ -61,6 +59,9 @@ public class StandAloneCommand extends AbstractServerCommand {
@CommandLine.Option(names={"--skip-tenants"},split=",",description="a list of tenants to skip, separated by a coma; for troubleshooting purpose only")
@CommandLine.Option(names={"--skip-indexer-records"},split=",",description="a list of indexer record keys, separated by a coma; for troubleshooting purpose only")
@@ -30,11 +31,17 @@ public class WebServerCommand extends AbstractServerCommand {
@Inject
privateExecutorsUtilsexecutorsUtils;
@Inject
privateSkipExecutionServiceskipExecutionService;
@Option(names={"--no-tutorials"},description="Flag to disable auto-loading of tutorial flows.")
booleantutorialsDisabled=false;
privatebooleantutorialsDisabled=false;
@Option(names={"--no-indexer"},description="Flag to disable starting an embedded indexer.")
booleanindexerDisabled=false;
privatebooleanindexerDisabled=false;
@CommandLine.Option(names={"--skip-indexer-records"},split=",",description="a list of indexer record keys, separated by a coma; for troubleshooting purpose only")
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.