* feat(CLI): add a new update from flow source CLI
* feat(CLI): use the repository instead of the webserver
* feat(CLI): change command name to SyncFromSource
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
# Conflicts:
# core/src/main/java/io/kestra/core/utils/ListUtils.java
By evaluating multiple condition in a separate queue, we serialize their evaluation which avoir races when we compute the outputs for flow triggers.
This is because evaluation is a multi step process: first you get the existing condtion, then you evaluate, then you store the result. As this is not guarded by a lock you must not do it concurrently.
The race can still occurs if muiltiple executors run but this is less probable. A re-implementation would be needed probably in 2.0 for that.
Fixes https://github.com/kestra-io/kestra-ee/issues/4602
the missing token: ${{ secrets.GH_PERSONAL_TOKEN }} is the only difference between this CI and EE CI, so it is probably the right fix
# Conflicts:
# .github/workflows/setversion-tag.yml
Ensure that preconditions are always re-rendered for any
new executions
Changes:
* add new fluent skipCache methods on RunContextProperty and Property
classes
Fix: #10651
Parallel flowable tasks like `Parallel`, `Dag` and `ForEach` are racy. When a task fail in a branch, other concurrent branches that have flowable may never ends.
We make sure that all children are terminated when a flowable is itself terminated.
Fixes#6780
This is because the execution is never considered fully terminated so concurrency limit is not handled properly.
This should also affect SLA, trigger lock, and other cleaning stuff.
The root issue is that, with a worker task from afterExecution, there are no other update on the execution itself (as it's already terminated) so no execution messages are again processed by the executor.
Because of that, the worker task result message from the afterExecution block is the last message, but unfortunatly as messages from the worker task result have no flow attached, the computation of the final termination is incorrect.
The solution is to load the flow if null inside the executor and the execution is terminated which should only occurs inside afterExecution.
Fixes#10657Fixes#8459Fixes#8609
This occurs in Postgres using the `\u0000` unicode sequence. Postgres refuse to store any JSONB with this sequence as it has no textual representation.
We now properly detect that and fail the task.
Fixes#10326
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
* fix(core): amend misc label-related issues
* re-enabled bulk update of label value
* re-enabled merging flow-execution labels by key
* made duplicated keys rejection readable
* forced multiple validations within `RequestUtils`
* ensured existing labels can be overriden
* added multiple tests validating complex scenarios
BREAKING CHANGE: switched from first to last label value override
BREAKING CHANGE: preventing empty key/value labels
BREAKING CHANGE: preventing whitespace in key
* fix(core): reflect feedback
* Deduplicated a list inside the `Labels` task.
* Worked around label mutation at `Worker`.
* Attempted to deduplicate labels within `Execution` as possible.
* fix(core): remove irrelevant changes
This is due to the fact that we now process the execution queue concurrently so there is a race when counting currently running executions. This can be seen easily using a ForEachItem as it could create tens or hundreds of executions almost instantly leading to almost all those executions started as they would all see 0 executions running...
Using a dedicated execution running queue, as done in EE, would serialize the messages and fix the issue.
However, if using multiple executor instances and concurrency limit = 1, there is a theoretical race as no locks will be done if no execution is running. A max surge of executions could be as high as the number of executor but this race is less probable to happen in real world scenario.
Fixes#10167
* fix(cli): #10062 add tenant to load flows properly at the startup
* fix(cli): #10062 add fallback tenant to ee service
* fix(cli): #10062 use tenant id in all cli
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
In some test situation, awaitExecution may receive old messages so we strenghten the predicate to be sure to wait for the correct execution: the one that ends successfully
When a flow fail and is restarted and contains either a finally or an afterExecution block, those are not resetted so the restart will skip all task and terminate the flow.
The fix will reset the status of those tasks so they are restarted.
Fixes#10155
In case multiple instances of the executor are started, the execution delay loop and the monitoring SLA loop have a risk of duplicate execution resume or execution SLA violation computation.
This could create some race conditions and duplicate execution update.
But this may also risk to create some deadlocks as two instances of the executor may try to lock the same exection to restart it (or fail it due to SLA).
This is because the executor didn't have the flow inside it so the execution is not correctly terminated.
It may fix other issues (like flow triggers, purge, ...)
Fixes#9618
* fix(core): #9609 delete tutorial flows and triggers before migrating the database
* fix(core): #9609 delete tutorial flows and triggers before migrating the database for EE version
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
Make ExecutorService responsible for deleting WorkerJobRunning
when a terminated TaskRun is added to an execution.
Changes:
- Remove unecessary read before delete on WorkerJobRunning table.
Close: #9493
* fix(ui): Better duplicate key mananage in the pair component
close#9220
* fix(ui): add a have-error prop on inputText that show a red shadow
* refactor: simplify inputpair component (#9491)
* fix: only show lock if disabled
* alertState define order
---------
Co-authored-by: Barthélémy Ledoux <bledoux@kestra.io>
* fix(flows): #9319 error when puase with timeout trigger an execution even after it's terminated
* fix(flows): only skip paused flow when execution is terminated
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
For AdditionalPlugin, as we resolves subtypes using isAssignable, we will resolve ourself as a subtype which leads to infinite loop while parsing the schema.
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
States that now will be filtered for the `In Progress` table:
- `CREATED`
- `RESTARTED`
- `QUEUED`
- `RUNNING`
- `PAUSED`
- `RETRYING`
- `KILLING`
Closes https://github.com/kestra-io/kestra/issues/9125.
* apply correctly the default filter for namespace and flow pages
* apply a default date on charts if none is set
* add flow_id filter in the global filter
* fix(core): bring back radio styles.
* bring radio back
Co-authored-by: Barthélémy Ledoux <bledoux@kestra.io>
---------
Co-authored-by: Barthélémy Ledoux <bledoux@kestra.io>
When a worker transitition to the TERMINATED_FORCED, the LivenessCoordinator
should wait for termination grace period to ensure
that all in-flight task-runs had time to be completely processed by the executors
Related-to: #8334
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
* fix/8050 - Improve padding and font sizes on the Blueprints tab in the Editor
fix(ui): improve padding and font sizes on Blueprints tab in Editor
Image 1:
- Added a "Blueprints" tittle
- Added 32px horizontal padding and 16px bottom padding to blueprint container
- Change font tittle
- Reduced blueprint title font size
- Added spacing between Source title and code block
- Changed background to ks-background-input
- Changed editor to ks-background-input
Image 2:
- Changed top spacing
- Changed background to ks-background-input
- Changed "bold" tittle blueprint to "regular"
Fixes#8950
* fix/8050 - Improve padding and font sizes on the Blueprints tab in the Editor
fix(ui): improve padding and font sizes on Blueprints tab in Editor
- FIx padding between tittle "Source" and code block
- changed css &.custom-dark-vs-theme
- Add Community top spacing
Fixes#8950
* fix(ui): improve padding and font sizes on Blueprints tab in Editor
- Updated back button with rounded background and ChevronLeft icon
- Reduced blueprint title font size and improved header layout
- Added proper spacing and alignment for header elements
Fixes#8950
* revert gitignore and application.yml
* fix(ui): minor tweaks.
---------
Co-authored-by: Piyush Bhaskar <102078527+Piyush-r-bhaskar@users.noreply.github.com>
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
* feat:
- Implement width property
- Replace custom dashboard
- Started to integrate the KPI chart
* feat(ui): introduce dashboard chart layout system
* feat(ui): introduce dashboard chart kpi card
* chore(ui): amend layout widths for sm screen size
* chore(ui): prevent editing of default dashboard
* chore(ui): centering the KPI text inside the box
* chore(ui): dashboard edit preview to respect set layouts
* chore(ui): initial work on setting the default flow & namespace dashboards
* fix(ui): make sure there is no naming clashes
* feat: KPI chart backend implementation
* feat: validation annotations
* chore(ui): make chart legend align to right side
* chore(ui): properly show chart labels
* chore(ui): improve state and ID components inside custom tabels
* chore(ui): add proper link to execution in tables
* feat: implemented Triggers as Datasource for custom dashboards
close kestra-io/kestra-ee#3740
* feat: modified the Markdown chart so now it accept different sources
* feat: rename KPI property to numerator & where
close#3739
* chore(ui): improve markdown component
* chore(ui): markdown charts
* chore(ui): markdown charts
* chore(ui): markdown charts remove padding
* chore(ui): markdown charts
* feat: fixes + define custom dashboard equivalent to current default dashboard with some modification
* fix: round double value
* chore(ui): improve flows and ns charts
* chore(ui): make sure that table shows execution links only if namespace and flowId exist
* chore(ui): make sure markdown is properly shown on dashboard edititng
* fix: correctly do preview instead of load on homepage
* fix: correctly preview markdown chart and add description in default flow dashboard
* fix: apply review changes
* fix: modify test following classes modifications on charts
* tests: restore package-lock
* remove chromatic tools and a warning
---------
Co-authored-by: MilosPaunovic <paun992@hotmail.com>
Co-authored-by: Bart Ledoux <bledoux@kestra.io>
* fix(ui): show proper no result
* Migrated the `SelectTable` component away from `NoData` to the
placeholder text attribute.
* Enabled `NoData` for logs since their empty result case was also
not clear.
* fix(core): make selection work and few more tweak.
---------
Co-authored-by: Piyush Bhaskar <102078527+Piyush-r-bhaskar@users.noreply.github.com>
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
* fix(core): few tweaks to align the placeholder and ui as per design.
* fix: minor tweak
---------
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
* fix(core): navigating back to plugins home.
* fix: keep the scroll to top
Co-authored-by: Barthélémy Ledoux <bledoux@kestra.io>
---------
Co-authored-by: Barthélémy Ledoux <bledoux@kestra.io>
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
* fix(filters): change label filtering to 'and' instead of 'or'
link to #8489
* test(execution): add test to validate and behavior
* test(execution): fix test
* test(controller): add logs to understand why the test fail
* test(controller): add logs to make sure the executions are running
* test(controller): add logs to make sure the executions are running
* test(controller): add logs to make sure the executions are running
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
* refactor: remove SECTIONS from local constants
* fix(ui): make delete button work for section, errors & tasks
* fix: allow delete of error, finally and afterexec
* fix: allow deleting fro any section
* use updated YAML_UTILS
* fix keyname usage
* update ui-libs for yaml utils
* fix: delete all tasks and keep order in creation
* chore: update ui-libs
When a queue is at max poll size, this means that it is at full capacity. In this case skip the sleep and process immediatly the next batch of message.
This improve latency at high thoughput without adding too much load to the database.
We can even go further by skipping sleep each time the poll returns messages but this would imply database cost so for now we balance performance and database cost by only skipping sleep when at max capacity.
Added test cases for:
Listing all available subcommands in the PluginCommand CLI help output.
Verifying that external plugins are not loaded by default in PluginCommand.
Updated existing tests for:
Enhanced help output validation to check for the presence of all expected subcommands/
Explicitly asserted the behavior of the loadExternalPlugins() method.
Ensured coverage for:
Edge cases where subcommands might be missing from the help output.
Input variants by directly invoking the command and capturing CLI output.
Error handling by confirming correct CLI configuration and plugin loading behavior.
* feat(core): #3427 add OSS tenant migration scripts
* clean(core): fixes after review
* clean(core): make only one command for oss and EE migration
* fix(core): user synchronisation command and clean PR
---------
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
* feat(core)!: WIP make tenant id required
* feat(core)!: WIP make tenant id required
* test(core)!: WIP fix storage unit test
* build(deps): bump com.google.guava:guava from 33.4.7-jre to 33.4.8-jre
Bumps [com.google.guava:guava](https://github.com/google/guava) from 33.4.7-jre to 33.4.8-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)
---
updated-dependencies:
- dependency-name: com.google.guava:guava
dependency-version: 33.4.8-jre
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump io.micronaut.platform:micronaut-platform
Bumps [io.micronaut.platform:micronaut-platform](https://github.com/micronaut-projects/micronaut-platform) from 4.8.0 to 4.8.2.
- [Release notes](https://github.com/micronaut-projects/micronaut-platform/releases)
- [Commits](https://github.com/micronaut-projects/micronaut-platform/compare/v4.8.0...v4.8.2)
---
updated-dependencies:
- dependency-name: io.micronaut.platform:micronaut-platform
dependency-version: 4.8.2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump flyingSaucerVersion from 9.11.6 to 9.12.0
Bumps `flyingSaucerVersion` from 9.11.6 to 9.12.0.
Updates `org.xhtmlrenderer:flying-saucer-core` from 9.11.6 to 9.12.0
- [Release notes](https://github.com/flyingsaucerproject/flyingsaucer/releases)
- [Changelog](https://github.com/flyingsaucerproject/flyingsaucer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flyingsaucerproject/flyingsaucer/compare/v9.11.6...v9.12.0)
Updates `org.xhtmlrenderer:flying-saucer-pdf` from 9.11.6 to 9.12.0
- [Release notes](https://github.com/flyingsaucerproject/flyingsaucer/releases)
- [Changelog](https://github.com/flyingsaucerproject/flyingsaucer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flyingsaucerproject/flyingsaucer/compare/v9.11.6...v9.12.0)
---
updated-dependencies:
- dependency-name: org.xhtmlrenderer:flying-saucer-core
dependency-version: 9.12.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: org.xhtmlrenderer:flying-saucer-pdf
dependency-version: 9.12.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump software.amazon.awssdk:bom from 2.31.21 to 2.31.25
Bumps software.amazon.awssdk:bom from 2.31.21 to 2.31.25.
---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
dependency-version: 2.31.25
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump com.github.oshi:oshi-core from 6.8.0 to 6.8.1
Bumps [com.github.oshi:oshi-core](https://github.com/oshi/oshi) from 6.8.0 to 6.8.1.
- [Release notes](https://github.com/oshi/oshi/releases)
- [Changelog](https://github.com/oshi/oshi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/oshi/oshi/compare/oshi-parent-6.8.0...oshi-parent-6.8.1)
---
updated-dependencies:
- dependency-name: com.github.oshi:oshi-core
dependency-version: 6.8.1
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump org.opensearch.client:opensearch-java
Bumps [org.opensearch.client:opensearch-java](https://github.com/opensearch-project/opensearch-java) from 2.22.0 to 2.23.0.
- [Release notes](https://github.com/opensearch-project/opensearch-java/releases)
- [Changelog](https://github.com/opensearch-project/opensearch-java/blob/v2.23.0/CHANGELOG.md)
- [Commits](https://github.com/opensearch-project/opensearch-java/compare/v2.22.0...v2.23.0)
---
updated-dependencies:
- dependency-name: org.opensearch.client:opensearch-java
dependency-version: 2.23.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump co.elastic.logging:logback-ecs-encoder
Bumps [co.elastic.logging:logback-ecs-encoder](https://github.com/elastic/ecs-logging-java) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/elastic/ecs-logging-java/releases)
- [Commits](https://github.com/elastic/ecs-logging-java/compare/v1.6.0...v1.7.0)
---
updated-dependencies:
- dependency-name: co.elastic.logging:logback-ecs-encoder
dependency-version: 1.7.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* tests(system): isolate SchedulerScheduleTest tests with tenantId
* Feat/storage outputs (#8361)
* feat(executions): Store outputs inside the internal storage (1/2)
* feat(executions): Store outputs inside the internal storage (2/2)
* feat(test): allow passing tenantId to tests
---------
Co-authored-by: Ludovic DEHON <tchiot.ludo@gmail.com>
* chore(deps): regular dependency update (#8484)
Performing a weekly round of dependency updates in the NPM ecosystem to keep everything up to date.
* fix(ui): full view height for single task logs (#8042)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
* feat: synchronize task edition with editor (#8433)
* fix(controls): adjust bottom position of contorls in multiPanelsEditor (#8465)
* fix(executions): unqueing execution must remove the execution queued
When an execution is queued in the JDBC backend, a record is inserted inside the execution_queued table, we must remove this record when we unqeue an execution.
Fixes#8448
* chore(core): localize to languages other than english (#8485)
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
* tests(webserver): fix flaky test which could query previous tests tasks
* tests(system): debug flaky error of shouldPauseExecutionByQueryRunningFlows
* tests(system): debug flaky error of shouldPauseExecutionByQueryRunningFlows
* tests(system): try with different task id
* tests(system): bump sleep-short sleep time to 10s
* fix(execution)*: decode and hide nested inputs of type SECRET
Fixes#7964
* refactor(core): pass the dynamic concurrency schema to no code editor (#8488)
There was an issue with passing hard-coded concurrency schema to be rendered in No Code editor, which is now amended and we're passing down the previously fetched one
* chore(deps): update gradle version
* doc(basic.md): add link to configuration for kestra property variables (#8490)
* fix(flows): properly check average duration for dashboard graphs (#8457)
There was a problem on flows view with the main chart not showing proper data until user clicks on duration toggle.
Closes https://github.com/kestra-io/kestra/issues/8435.
Closes https://github.com/kestra-io/kestra-ee/issues/3499.
* docs(core-pause): update pauseDuration properties, titles, descriptions (#8495)
* fix(system): restrict the JdbcConcurrencyLimitService to the JDBC runner
* fix(core): fix indexer metric description (#8500)
* Add examples with expression and trimmed values (#6154)
* chore(ui): improvement to drilldown for Default and Custom Charts. (#7885)
* chore(ui): improvement to drilldown for Default and Custom Charts.
* minor tweak
* test: fix the Barchart stories to test drilldown
---------
Co-authored-by: Bart Ledoux <bledoux@kestra.io>
* fix(ui): restart trigger position for backfill column (#8246)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
Co-authored-by: Bart Ledoux <bledoux@kestra.io>
* feat(plugin): add a way to provide additional type of plugins
Provide a way for plugins to define a new type of plugins.
To do that, a plugin must provide both an abstract base class that extends AdditionalPlugin and a set of concret classes.
Both the abstract base class and the concrete classes mut be inside the same plugin. This is a limitation that we may work on later by providing, for example, an SPI to add base classes to the application classloader.
* fix(ui): save existing flow after making changes (#8378)
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
* fix(core): failing DocumentationGeneratorTest.returnDoc()
* chore(core): refactor component to composition API structure and with some styling (#8504)
* feat(flows): add validation for use of inputs and outputs with '-' in the name (#8379)
* test(core): fix breaking change in local flow repository (#8517)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
* feat(system)!: remove the SQLServer runner
Part-of: https://github.com/kestra-io/kestra-ee/issues/3504
* chore(build): add Postgres stat extension
* feat(plugins): add Langchain4J plugins
* chore(system): add warn log when emit logQueue failed (#8432)
* feat(plugins): add Go Script plugin
* fix(jdbc): add service_id index on service_instance table
* chore(flows): improve the blueprints view within the flow editing panels (#7983)
Changes here consist of removing the tags from blueprint view on Multi Panel flow editor, along with couple of other UI improvements.
Closes https://github.com/kestra-io/kestra/issues/7881.
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
* fix(flows): properly load blueprints in multi panel view (#8524)
While using the new Multi Panel view blueprints were not loading properly due to wrong paramtere being sent to action. now that's sorted.
Closes https://github.com/kestra-io/kestra/issues/8523.
* feat(flows): improve the display of array inputs when running an execution (#7953)
This PR is introducing a change of how the `array` inputs are displayed inside the flow run dialog, to be more user-friendly.
Closes https://github.com/kestra-io/kestra/issues/6947.
---------
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
* fix(system): change default config values for liveness
Change kestra.server.liveness.interval from 5s to 10s
to be less agressive on liveness check. Align
other default liveness configs with kafka implementation.
* fix(ui): amend Absolute date filter's looks (#8501)
* chore(core): localize to languages other than english (#8528)
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
* docs(flow-trigger): add note about no Pebble in conditions
* fix(ui): remove parts of filter using backspace (#8105)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
* fix(ui): open link in markdown in other tab. (#8258)
* fix(ui): open link in markdown in other tab.
* chore(core): restrict attribute to external links.
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
---------
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
* fix(system): load OpenTelemetry lib in the app classloader
Without that, as we have it here, plugins may have class loading issue if they use OpenTelemetry internally (like in the Elasticsearch client).
* feat(ui): Add search in internal docs (#8458)
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
* fix(triggers): inject default later inside the Scheduler
Today, as they are injected eagerly, they are done even if no trigger exists.
This is counter-performant, and in case the flow is an error will log each seconds.
Doing it a little later will be better.
* build(deps): bump software.amazon.awssdk:bom from 2.31.25 to 2.31.30
Bumps software.amazon.awssdk:bom from 2.31.25 to 2.31.30.
---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
dependency-version: 2.31.30
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump org.wiremock:wiremock-jetty12 from 3.12.1 to 3.13.0
Bumps [org.wiremock:wiremock-jetty12](https://github.com/wiremock/wiremock) from 3.12.1 to 3.13.0.
- [Release notes](https://github.com/wiremock/wiremock/releases)
- [Commits](https://github.com/wiremock/wiremock/compare/3.12.1...3.13.0)
---
updated-dependencies:
- dependency-name: org.wiremock:wiremock-jetty12
dependency-version: 3.13.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump jacksonVersion from 2.18.3 to 2.19.0
Bumps `jacksonVersion` from 2.18.3 to 2.19.0.
Updates `com.fasterxml.jackson:jackson-bom` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson-bom/compare/jackson-bom-2.18.3...jackson-bom-2.19.0)
Updates `com.fasterxml.jackson.core:jackson-core` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson-core/compare/jackson-core-2.18.3...jackson-core-2.19.0)
Updates `com.fasterxml.jackson.core:jackson-databind` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson/commits)
Updates `com.fasterxml.jackson.core:jackson-annotations` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson/commits)
Updates `com.fasterxml.jackson.module:jackson-module-parameter-names` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson-modules-java8/compare/jackson-modules-java8-2.18.3...jackson-modules-java8-2.19.0)
Updates `com.fasterxml.jackson.dataformat:jackson-dataformat-yaml` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson-dataformats-text/compare/jackson-dataformats-text-2.18.3...jackson-dataformats-text-2.19.0)
Updates `com.fasterxml.jackson.dataformat:jackson-dataformat-smile` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson-dataformats-binary/compare/jackson-dataformats-binary-2.18.3...jackson-dataformats-binary-2.19.0)
Updates `com.fasterxml.jackson.dataformat:jackson-dataformat-cbor` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson-dataformats-binary/compare/jackson-dataformats-binary-2.18.3...jackson-dataformats-binary-2.19.0)
Updates `com.fasterxml.jackson.dataformat:jackson-dataformat-ion` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson-dataformat-ion/commits)
Updates `com.fasterxml.jackson.dataformat:jackson-dataformat-xml` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson-dataformat-xml/compare/jackson-dataformat-xml-2.18.3...jackson-dataformat-xml-2.19.0)
Updates `com.fasterxml.jackson.datatype:jackson-datatype-guava` from 2.18.3 to 2.19.0
- [Commits](https://github.com/FasterXML/jackson-datatypes-collections/compare/jackson-datatypes-collections-2.18.3...jackson-datatypes-collections-2.19.0)
Updates `com.fasterxml.jackson.datatype:jackson-datatype-jsr310` from 2.18.3 to 2.19.0
Updates `com.fasterxml.jackson.datatype:jackson-datatype-jdk8` from 2.18.3 to 2.19.0
---
updated-dependencies:
- dependency-name: com.fasterxml.jackson:jackson-bom
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.core:jackson-core
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.core:jackson-databind
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.core:jackson-annotations
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.module:jackson-module-parameter-names
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-yaml
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-smile
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-cbor
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-ion
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-xml
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.datatype:jackson-datatype-guava
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.datatype:jackson-datatype-jsr310
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
- dependency-name: com.fasterxml.jackson.datatype:jackson-datatype-jdk8
dependency-version: 2.19.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore(execution): update display names for executions. (#8527)
* fix(core): change incorrectly used search parameter (#8534)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
* fix(ui): set isCreating to false when opening flow edit mode (#8549)
* fix(core): safely access section and identifier query params (#8542)
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
* fix(ui): update storybook editor tests with provided keys (#8550)
Co-authored-by: Piyush Bhaskar <102078527+Piyush-r-bhaskar@users.noreply.github.com>
* chore(core): localize to languages other than english (#8554)
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
* fix(triggers): amend broken filtering on triggers tab (#8553)
There was a problem with both namespace and state filters on Triggers page which is now properly sorted.
Closes https://github.com/kestra-io/kestra/issues/8529.
* chore: attempt to fix flaky tests (#8537)
SingleFlowCommandsTest:
The flow Delete -> Create -> Update sequence is weird - delete got HTTP 404.
Reworked to Create -> Update -> Delete sequence.
PurgeLogsTest:
The log repository contained the prepared single entry but also might
contain additional entries from previously logged messages.
* fix(namespaces): namespaceFiles with same name are wrongly overwritten (#8562)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
* feat(core): forward execution labels in Flow Trigger
* chore(triggers)*: properly handle switches for triggers disabled from within flow source (#8106)
It was not clear as to which trigger can not be enabled and why. Now, that is much more clear with the proper tooltips and disabling of switch toggling.
Closes https://github.com/kestra-io/kestra/issues/8011.
Closes https://github.com/kestra-io/kestra/issues/5736.
* fix(executions): fix execution failure due to UnsupportedOperationException (#8563)
Fix: #8563
* chore(core): localize to languages other than english (#8568)
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
* feat(system): add TestSuite model,taskFixture impl
* fix: redirect to edit when saving new flow (#8560)
Co-authored-by: Piyush Bhaskar <102078527+Piyush-r-bhaskar@users.noreply.github.com>
* feat(system)*: decrease defaut JDBC queue poll size
Decreasing it from 100 to 50 didn't show any performance hit but should lower the memory consumption now that we process the queue concurrently in the executor.
## BEFORE - pollSize=100
- 10 tx/s: 150ms
- 25 tx/s: 200ms
- 50 tx/s: 300ms
- 75 tx/s: 5.2s
- 100 tx/s: 15s
## AFTER - pollSize=50
- 10 tx/s: 150ms
- 25 tx/s: 200ms
- 50 tx/s: 300ms
- 75 tx/s: 4.8s
- 100 tx/s: 14s
* feat(flows): Allow to define an onPause task on the Pause task
The onPause task will be executed immediatly when the execution is paused.
Part-of: #3601
* feat(core)!: WIP make tenant id required
* feat(core)!: WIP make tenant id required
* Feat/storage outputs (#8361)
* feat(executions): Store outputs inside the internal storage (1/2)
* feat(executions): Store outputs inside the internal storage (2/2)
* feat(test): allow passing tenantId to tests
---------
Co-authored-by: Ludovic DEHON <tchiot.ludo@gmail.com>
* tests(webserver): fix flaky test which could query previous tests tasks
* feat(executions): Add workerId to each worker task attemps
Closes#7799
* Feat/storage outputs (#8361)
* feat(executions): Store outputs inside the internal storage (1/2)
* feat(executions): Store outputs inside the internal storage (2/2)
* feat(test): allow passing tenantId to tests
---------
Co-authored-by: Ludovic DEHON <tchiot.ludo@gmail.com>
* wip(core): fix unit tests
* test(core): fix unit tests
* test(core): fix unit tests
* test(core): fix unit tests
* feat(core): make tenant id required everywhere
* feat(core): make tenant required in create user command
* feat(core): clean the PR
* feat(core): add tenant id to dashboard controller
* fix(core): tests after merging
* clean(core): fixes after review
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Roman Acevedo <roman.acevedo62@gmail.com>
Co-authored-by: Loïc Mathieu <loikeseke@gmail.com>
Co-authored-by: Ludovic DEHON <tchiot.ludo@gmail.com>
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Co-authored-by: Piyush Bhaskar <102078527+Piyush-r-bhaskar@users.noreply.github.com>
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Action <actions@github.com>
Co-authored-by: AJ Emerich <aemerich@kestra.io>
Co-authored-by: ben8t <46634684+Ben8t@users.noreply.github.com>
Co-authored-by: Bart Ledoux <bledoux@kestra.io>
Co-authored-by: Satvik Kushwaha <59243339+satvik2131@users.noreply.github.com>
Co-authored-by: Karuna Tata <karuna.tata@devrev.ai>
Co-authored-by: Hashim Khalifa <105060840+hashimzs@users.noreply.github.com>
Co-authored-by: lwyang <1670906161@qq.com>
Co-authored-by: 杨利伟 <yangliwei@xiaomi.com>
Co-authored-by: Florian Hussonnois <fhussonnois@kestra.io>
Co-authored-by: yuri <1969yuri1969@gmail.com>
Co-authored-by: AJ Emerich <aj-emerich@proton.me>
Co-authored-by: rajatsingh23 <48049052+rajatsingh23@users.noreply.github.com>
* feat(core): replace select with tabs for schema selection.
* feat(core): change selection to tab and remove display of type and tooltip.
* fix: remove obsolete component and add class.
* chore(core): task enclosed under a border.
* fixes enclosed border only for anyOf type
---------
Co-authored-by: Barthélémy Ledoux <bledoux@kestra.io>
Instead of consuming multiple time the queue, which lead to concurrent queries on the `queues` table, process concurrently via an ExecutorService the messages from the queue.
We dind't process a new batch of messages until the existing one is totally process to be sure we process in FIFO the same execution message.
Also, go back to a poll size of 100 to mitiguate the performance hit due to this change.
Surprisingly, this method appear in some CPU and allocation profile as having a high cost, especially on the scheduler.
Switching to using a StringJoiner brings 4x perf improvements in method execution time (great improvement also on allocation but didn't have a measurement).
One of the two is failing pretty often, using @ExecuteFlow instead of @LoadFlow seems to make them non-flaky.
We can also remove the second ad those are duplicated test but maybe the person that duplicate it has some reason...
* feat(demo): add video support to various demo components and enhance layout
* update video sources
* feat(core): Embed videos directly on EE-feature teasers.
* chore(core): localize to languages other than english
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
* Apply suggestions from code review
---------
Co-authored-by: GitHub Action <actions@github.com>
Co-authored-by: Anna Geller <anna.m.geller@gmail.com>
Enable concatenation of arbitrary scalar values (e.g., numbers, booleans)
into a string when setting a typed plugin property expecting a single value:
e.g., the `value` property of the `io.kestra.plugin.core.kv.Set` task.
Fix: kestra-io/kestra-ee#3570
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
advance on CRUD UI for Add unit tests for flows — new tab on the Flows page, runnable from the UI, API or from CI/CD (needs CLI and GitHub Action) kestra-ee#3110
linked to feat(system): add TestSuite list and edit pages kestra-ee#3585
This is a breaking change as existing flow trigger wich didn't filter to any states on conditiosn or preconditions was before triggering executions only on terminal states and will now trigger executions on terminated and paused states.
Fixes https://github.com/kestra-io/kestra-ee/issues/3535
Purge service instance in EMPTY state after a certain duration, 30 days by default, to avoid never ending groth on the service_instances table.
Fixes#8514
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
This change is introducing a new setting for users to choose - the automatic table refresh interval.
---------
Co-authored-by: MilosPaunovic <paun992@hotmail.com>
Label for triggering autocompletion was `CMD/CTRL + SPACE`, but on `MacOS`, that was triggering a Spotlight search. As the `CTRL + SPACE` works on `MacOS` also, the label is tweaked.
Closes https://github.com/kestra-io/kestra/issues/8617
---
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
- By default, switch from quick polling to long polling after 30s of inacitvity on a queue
- By default, configure long polling interval to 500ms
- Use steps to goes from quick polling to long polling (5 steps automatically computed)
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
Made changes to backfilling executions input to fill the default input value by the input value set within a schedule.
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
Decreasing it from 100 to 50 didn't show any performance hit but should lower the memory consumption now that we process the queue concurrently in the executor.
## BEFORE - pollSize=100
- 10 tx/s: 150ms
- 25 tx/s: 200ms
- 50 tx/s: 300ms
- 75 tx/s: 5.2s
- 100 tx/s: 15s
## AFTER - pollSize=50
- 10 tx/s: 150ms
- 25 tx/s: 200ms
- 50 tx/s: 300ms
- 75 tx/s: 4.8s
- 100 tx/s: 14s
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
SingleFlowCommandsTest:
The flow Delete -> Create -> Update sequence is weird - delete got HTTP 404.
Reworked to Create -> Update -> Delete sequence.
PurgeLogsTest:
The log repository contained the prepared single entry but also might
contain additional entries from previously logged messages.
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
Today, as they are injected eagerly, they are done even if no trigger exists.
This is counter-performant, and in case the flow is an error will log each seconds.
Doing it a little later will be better.
* fix(ui): open link in markdown in other tab.
* chore(core): restrict attribute to external links.
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
---------
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
Change kestra.server.liveness.interval from 5s to 10s
to be less agressive on liveness check. Align
other default liveness configs with kafka implementation.
While using the new Multi Panel view blueprints were not loading properly due to wrong paramtere being sent to action. now that's sorted.
Closes https://github.com/kestra-io/kestra/issues/8523.
Provide a way for plugins to define a new type of plugins.
To do that, a plugin must provide both an abstract base class that extends AdditionalPlugin and a set of concret classes.
Both the abstract base class and the concrete classes mut be inside the same plugin. This is a limitation that we may work on later by providing, for example, an SPI to add base classes to the application classloader.
* chore(ui): improvement to drilldown for Default and Custom Charts.
* minor tweak
* test: fix the Barchart stories to test drilldown
---------
Co-authored-by: Bart Ledoux <bledoux@kestra.io>
There was an issue with passing hard-coded concurrency schema to be rendered in No Code editor, which is now amended and we're passing down the previously fetched one
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.
Co-authored-by: GitHub Action <actions@github.com>
When an execution is queued in the JDBC backend, a record is inserted inside the execution_queued table, we must remove this record when we unqeue an execution.
Fixes#8448
echo "No changes to commit. Exiting with success."
exit 0
fi
git commit -m "chore(core): localize to languages other than english" -m "Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference."
gh pr create --title "Translations from en.json" --body "This PR was created automatically by a GitHub Action." --base develop --head $BRANCH_NAME --assignee anna-geller --reviewer anna-geller
description:'(deprecated) Plugin version window for old Kestra releases using .plugins file (0.22 to 0.24). If omitted, then plugin list will be fetched from the API compatible versions endpoint'
required:false
type:string
default:"[0.23,0.24)"
dry-run:
description:'Dry run mode that will not write or release anything'
@@ -42,6 +44,9 @@ public class KvUpdateCommand extends AbstractApiCommand {
@Option(names={"-f","--file-value"},description="The file from which to read the value to set. If this is provided, it will take precedence over any specified value.")
publicPathfileValue;
@Inject
privateTenantIdSelectorServicetenantService;
@Override
publicIntegercall()throwsException{
super.call();
@@ -56,7 +61,7 @@ public class KvUpdateCommand extends AbstractApiCommand {
@CommandLine.Option(names={"--skip-indexer-records"},split=",",description="a list of indexer record keys, separated by a coma; for troubleshooting purpose only")
@@ -44,6 +45,9 @@ public class StandAloneCommand extends AbstractServerCommand {
@CommandLine.Option(names={"-f","--flow-path"},description="the flow path containing flow to inject at startup (when running with a memory flow repository)")
privateFileflowPath;
@CommandLine.Option(names="--tenant",description="Tenant identifier, Required to load flows from path with the enterprise edition")
privateStringtenantId;
@CommandLine.Option(names={"--worker-thread"},description="the number of worker threads, defaults to four times the number of available processors. Set it to 0 to avoid starting a worker.")
privateintworkerThread=defaultWorkerThread();
@@ -59,6 +63,9 @@ public class StandAloneCommand extends AbstractServerCommand {
@CommandLine.Option(names={"--skip-tenants"},split=",",description="a list of tenants to skip, separated by a coma; for troubleshooting purpose only")
@CommandLine.Option(names={"--skip-indexer-records"},split=",",description="a list of indexer record keys, separated by a coma; for troubleshooting purpose only")
@@ -28,11 +31,17 @@ public class WebServerCommand extends AbstractServerCommand {
@Inject
privateExecutorsUtilsexecutorsUtils;
@Inject
privateSkipExecutionServiceskipExecutionService;
@Option(names={"--no-tutorials"},description="Flag to disable auto-loading of tutorial flows.")
booleantutorialsDisabled=false;
privatebooleantutorialsDisabled=false;
@Option(names={"--no-indexer"},description="Flag to disable starting an embedded indexer.")
booleanindexerDisabled=false;
privatebooleanindexerDisabled=false;
@CommandLine.Option(names={"--skip-indexer-records"},split=",",description="a list of indexer record keys, separated by a coma; for troubleshooting purpose only")
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.