Compare commits

...

634 Commits

Author SHA1 Message Date
Florian Hussonnois
c0baedf7f1 fix(flows): enhance error logs when injecting default values
Avoid logging unnecessary full exception error stacktrace
when an error occurred while injecting plugin defaults -
Those errors can be logged often and can be temporary.
2025-04-22 14:12:14 +02:00
dependabot[bot]
a06421dd84 build(deps): bump co.elastic.logging:logback-ecs-encoder
Bumps [co.elastic.logging:logback-ecs-encoder](https://github.com/elastic/ecs-logging-java) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/elastic/ecs-logging-java/releases)
- [Commits](https://github.com/elastic/ecs-logging-java/compare/v1.6.0...v1.7.0)

---
updated-dependencies:
- dependency-name: co.elastic.logging:logback-ecs-encoder
  dependency-version: 1.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-22 11:23:18 +02:00
dependabot[bot]
de6fcab785 build(deps): bump org.opensearch.client:opensearch-java
Bumps [org.opensearch.client:opensearch-java](https://github.com/opensearch-project/opensearch-java) from 2.22.0 to 2.23.0.
- [Release notes](https://github.com/opensearch-project/opensearch-java/releases)
- [Changelog](https://github.com/opensearch-project/opensearch-java/blob/v2.23.0/CHANGELOG.md)
- [Commits](https://github.com/opensearch-project/opensearch-java/compare/v2.22.0...v2.23.0)

---
updated-dependencies:
- dependency-name: org.opensearch.client:opensearch-java
  dependency-version: 2.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-22 11:22:01 +02:00
dependabot[bot]
b2f68a7b97 build(deps): bump com.github.oshi:oshi-core from 6.8.0 to 6.8.1
Bumps [com.github.oshi:oshi-core](https://github.com/oshi/oshi) from 6.8.0 to 6.8.1.
- [Release notes](https://github.com/oshi/oshi/releases)
- [Changelog](https://github.com/oshi/oshi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/oshi/oshi/compare/oshi-parent-6.8.0...oshi-parent-6.8.1)

---
updated-dependencies:
- dependency-name: com.github.oshi:oshi-core
  dependency-version: 6.8.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-22 11:10:17 +02:00
dependabot[bot]
01cb30f933 build(deps): bump software.amazon.awssdk:bom from 2.31.21 to 2.31.25
Bumps software.amazon.awssdk:bom from 2.31.21 to 2.31.25.

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-version: 2.31.25
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-22 11:09:06 +02:00
dependabot[bot]
d2cda63cfa build(deps): bump flyingSaucerVersion from 9.11.6 to 9.12.0
Bumps `flyingSaucerVersion` from 9.11.6 to 9.12.0.

Updates `org.xhtmlrenderer:flying-saucer-core` from 9.11.6 to 9.12.0
- [Release notes](https://github.com/flyingsaucerproject/flyingsaucer/releases)
- [Changelog](https://github.com/flyingsaucerproject/flyingsaucer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flyingsaucerproject/flyingsaucer/compare/v9.11.6...v9.12.0)

Updates `org.xhtmlrenderer:flying-saucer-pdf` from 9.11.6 to 9.12.0
- [Release notes](https://github.com/flyingsaucerproject/flyingsaucer/releases)
- [Changelog](https://github.com/flyingsaucerproject/flyingsaucer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flyingsaucerproject/flyingsaucer/compare/v9.11.6...v9.12.0)

---
updated-dependencies:
- dependency-name: org.xhtmlrenderer:flying-saucer-core
  dependency-version: 9.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.xhtmlrenderer:flying-saucer-pdf
  dependency-version: 9.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-22 11:08:11 +02:00
dependabot[bot]
f89187db6a build(deps): bump io.micronaut.platform:micronaut-platform
Bumps [io.micronaut.platform:micronaut-platform](https://github.com/micronaut-projects/micronaut-platform) from 4.8.0 to 4.8.2.
- [Release notes](https://github.com/micronaut-projects/micronaut-platform/releases)
- [Commits](https://github.com/micronaut-projects/micronaut-platform/compare/v4.8.0...v4.8.2)

---
updated-dependencies:
- dependency-name: io.micronaut.platform:micronaut-platform
  dependency-version: 4.8.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-22 11:07:01 +02:00
dependabot[bot]
8e4fe892e9 build(deps): bump com.google.guava:guava from 33.4.7-jre to 33.4.8-jre
Bumps [com.google.guava:guava](https://github.com/google/guava) from 33.4.7-jre to 33.4.8-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-version: 33.4.8-jre
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-22 11:04:45 +02:00
Barthélémy Ledoux
eb13dce0ff fix(ui): use creating=true instead of identifier=new (#8443) 2025-04-22 09:46:02 +02:00
Florian Hussonnois
a14518b810 refactor(plugin): add dedicated service to parse tasks log line
Add new TaskLogLineMatcher class for matching and
 capturing structured data from task execution logs.

Related-to: kestra-io/kestra-ee#3441
2025-04-22 09:37:32 +02:00
Piyush Bhaskar
c64f15a035 fix(container): update tab container classes for better layout handling. (#8456) 2025-04-22 12:19:29 +05:30
Piyush Bhaskar
f79541616e chore(ui): Improve the display of Resume/Kill executions (#8227)
* chore(ui): Improve the display of Resume/Kill executions

* minor tweak

---------

Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
2025-04-21 11:22:35 +05:30
Frank Tianyu Zeng
cb6a6bfd91 fix(revision): restore side-by-side view in revision history after flow edit (#8439)
Co-authored-by: Piyush Bhaskar <102078527+Piyush-r-bhaskar@users.noreply.github.com>
2025-04-21 11:12:36 +05:30
Ludovic DEHON
b2ae2ff6f7 feat(build): fix some sonar alert 2025-04-20 22:23:41 +02:00
Ludovic DEHON
a7836ca673 feat(build): add sonar 2025-04-19 00:49:09 +02:00
YannC
7bc60a1056 fix(ui): add new property in filters (#8444) 2025-04-18 21:27:55 +02:00
YannC
09943a1e7b fix(webserver): set ids parameters as body instead of queryvalue (#8438) 2025-04-18 11:58:11 +02:00
Sayed Murtadha Ahmed
8b6af7a808 chore(core)*: make sure that notifications with large message text are closeable (#8431)
There was a problem when an error message is too big, the toaster could not be closed. This change is amending that issue.

Closes https://github.com/kestra-io/kestra/issues/8352.
2025-04-17 11:44:17 +02:00
YannC.
e74e7ff8e5 fix(ui): correct path for namespace files on namespace page
close #8140
2025-04-17 10:34:46 +02:00
Anna Geller
9adf3a5444 fix(docs): improve PublishMetrics example and basic.md KV doc (#8427) 2025-04-16 20:02:36 +02:00
Karuna Tata
c069b2fbb3 chore(core)*: deboucing task validation from no code editor (#8418)
There was an issue when a user edited a task from the No Code editor—the validation endpoint was being called excessively, instead of waiting for the user to finish typing. This has now been resolved with this PR.

Closes https://github.com/kestra-io/kestra/issues/7073.

Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2025-04-16 12:31:03 +02:00
YannC
534b7f4ec7 refactor(core): changes to improve openapi spec
* better method name
* correct annotation to descript body
2025-04-16 12:26:32 +02:00
Miloš Paunović
7b1ee4a9e0 refactor(core): prevent inheriting unused attributes in components (#8419)
Remove default inheriting of attributes in Vue components, which are not used. This pollutes the console with warnings and it's not necessary, so this PR is solving the problem.
2025-04-16 12:08:45 +02:00
Miloš Paunović
38a9ebcbef refactor(core): replace deprecated vue flow options parameter (#8417)
Remove usage of the deprecated options object parameter with the id one, as per the latest API guidelines. This change ensures compatibility with future versions and removes the related console warning.

Related to #7804.
2025-04-16 10:29:47 +02:00
weibo1
eea47c6e40 feat(system): add duration metrics for handle() method 2025-04-16 09:05:00 +02:00
weibo1
e1c4ae22f2 feat(system): add index to commonly queried fields in the WHERE conditions of the triggers table 2025-04-16 09:05:00 +02:00
Miloš Paunović
b7861a139e chore(deps): regular dependency update (#8415)
Performing a weekly round of dependency updates in the NPM ecosystem to keep everything up to date.
2025-04-16 08:46:32 +02:00
Miloš Paunović
954d64ecaa fix(core)*: allow horizontal scrolling of tabs in multi panel view (#8414)
If there are multiple tabs opened in a single panel of the new multi panel view,  there was no ability to scroll and see the overflowing ones. With changes in this PR, users can now do just that.

Closes https://github.com/kestra-io/kestra/issues/8270.
2025-04-16 12:05:37 +05:30
Karuna Tata
f61ba36023 feat(core)*: allow moving of entire panels (#8377)
Changes in this pull request now allow users to move entire panels where ever they want, plus, there are `Move right` and `Move left` options in the context menu.

Closes https://github.com/kestra-io/kestra/issues/8272.

Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2025-04-16 07:47:09 +02:00
Loïc Mathieu
fbd989ccab feat(core): allow to specify Pause task resume behavior (#8242)
When a pause task is resumed either because it wait until the end of it's duration or is resumed manually, it can now use a behavior to describe what to do next: resume, warn, fail, or cancel the execution.

Fixes #8242
2025-04-15 17:39:53 +02:00
yuri
12affd4b4b fix(build): amend Node Gradle to reflect changes (#7970) 2025-04-15 15:15:34 +02:00
Satvik Kushwaha
b75730a0ca chore(dashboards)*: amend hover color of table top buttons on dashboard (#8388)
Tables on main dashboard which have the `See all` buttons needed UI tweaking of color on hover, which is handled in this pull request.

Closes https://github.com/kestra-io/kestra/issues/8376.

Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2025-04-15 14:14:49 +02:00
Barthélémy Ledoux
4170615765 fix(ui): navigation between flow editor and tabs - blank screen (#8392) 2025-04-15 11:41:18 +02:00
Barthélémy Ledoux
5cfb6aa1f5 test(ui): ignore unhandled canceled promises in storybook (#8394) 2025-04-15 11:39:39 +02:00
AJ Emerich
41d660e18e docs(core): update plugin titles and descriptions (#8390) 2025-04-15 11:22:47 +02:00
yuri
8af4f1928a chore(webserver): amend copy&pasted description (#8346) 2025-04-15 11:12:09 +02:00
github-actions[bot]
1488caccc7 chore(core): localize to languages other than english (#8391)
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.

Co-authored-by: GitHub Action <actions@github.com>
2025-04-15 11:09:33 +02:00
Karuna Tata
85fc48963f feat(core): add a confirmation dialog before leaving the settings page (#8365)
If the user changed something on the `Settings` page and then just navigated to another page, all new changes would all be lost. This PR is adding a confirmation dialog before route leave, if there are any changes made, to confirm either saving or discarding them.

Closes https://github.com/kestra-io/kestra/issues/8364.

Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2025-04-15 10:31:49 +02:00
Barthélémy Ledoux
b706dec9d2 test(core): make storybook tests pass with less warnings (#8382)
Lots of warnings that we can see in the UI unit tests make them flaky. In this PR we're trying to avoid pollution as much as possible so we can at least test.

Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-04-15 10:16:14 +02:00
Karuna Tata
ceac4d38f9 feat(core)*: implement redirection to flow creation page on skipping tutorial (#8333)
When a user clicks the `Skip tutorial` button, it just does that, shuts the tutorial down but leaves the user on the same page. This change will redirect users to `Flow Creation` page if the `Skip Tutorial` button is clicked.

Closes https://github.com/kestra-io/kestra/issues/8326.

Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-04-15 08:11:41 +02:00
dependabot[bot]
ec7bf52e08 build(deps-dev): bump vite in /ui in the npm_and_yarn group (#8362)
Bumps the npm_and_yarn group in /ui with 1 update: [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).


Updates `vite` from 6.2.5 to 6.2.6
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.2.6/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.2.6/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.2.6
  dependency-type: direct:development
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-14 15:22:09 +02:00
Loïc Mathieu
247594299c feat(system)!: Describe our internal metrics
Add a description to all our internal metrics.

BREAKING CHANGE: the following metrics has been removed:
- executor.taskrun.next.count
- executor.workertaskresult.count

The metric scheduler.execution.running.duration has been renamed to scheduler.execution.lock.duration
2025-04-14 15:17:23 +02:00
Loïc Mathieu
513139976c feat(system): Allow to describe metrics (#1989) 2025-04-14 15:17:23 +02:00
Miloš Paunović
2cab9de57c refactor(namespaces): remove the unused maximize property (#8380)
There is no more need for `maximize` property to be passed for locked tabs, and therefore it's removed in this commit.
2025-04-14 13:55:45 +02:00
dependabot[bot]
cfae13c045 build(deps): bump io.micrometer:micrometer-core from 1.14.5 to 1.14.6
Bumps [io.micrometer:micrometer-core](https://github.com/micrometer-metrics/micrometer) from 1.14.5 to 1.14.6.
- [Release notes](https://github.com/micrometer-metrics/micrometer/releases)
- [Commits](https://github.com/micrometer-metrics/micrometer/compare/v1.14.5...v1.14.6)

---
updated-dependencies:
- dependency-name: io.micrometer:micrometer-core
  dependency-version: 1.14.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-14 11:55:59 +02:00
dependabot[bot]
6190f8774a build(deps): bump com.google.guava:guava from 33.4.6-jre to 33.4.7-jre
Bumps [com.google.guava:guava](https://github.com/google/guava) from 33.4.6-jre to 33.4.7-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-version: 33.4.7-jre
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-14 10:55:50 +02:00
dependabot[bot]
b98a0a783d build(deps): bump commons-io:commons-io from 2.18.0 to 2.19.0
Bumps commons-io:commons-io from 2.18.0 to 2.19.0.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-version: 2.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-14 10:10:24 +02:00
dependabot[bot]
3da2dc6257 build(deps): bump flyingSaucerVersion from 9.11.5 to 9.11.6
Bumps `flyingSaucerVersion` from 9.11.5 to 9.11.6.

Updates `org.xhtmlrenderer:flying-saucer-core` from 9.11.5 to 9.11.6
- [Release notes](https://github.com/flyingsaucerproject/flyingsaucer/releases)
- [Changelog](https://github.com/flyingsaucerproject/flyingsaucer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flyingsaucerproject/flyingsaucer/compare/v9.11.5...v9.11.6)

Updates `org.xhtmlrenderer:flying-saucer-pdf` from 9.11.5 to 9.11.6
- [Release notes](https://github.com/flyingsaucerproject/flyingsaucer/releases)
- [Changelog](https://github.com/flyingsaucerproject/flyingsaucer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flyingsaucerproject/flyingsaucer/compare/v9.11.5...v9.11.6)

---
updated-dependencies:
- dependency-name: org.xhtmlrenderer:flying-saucer-core
  dependency-version: 9.11.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.xhtmlrenderer:flying-saucer-pdf
  dependency-version: 9.11.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-14 10:10:06 +02:00
dependabot[bot]
6feb027696 build(deps): bump org.aspectj:aspectjweaver from 1.9.23 to 1.9.24
Bumps [org.aspectj:aspectjweaver](https://github.com/eclipse/org.aspectj) from 1.9.23 to 1.9.24.
- [Release notes](https://github.com/eclipse/org.aspectj/releases)
- [Commits](https://github.com/eclipse/org.aspectj/commits)

---
updated-dependencies:
- dependency-name: org.aspectj:aspectjweaver
  dependency-version: 1.9.24
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-14 10:09:21 +02:00
dependabot[bot]
83d6095669 build(deps): bump software.amazon.awssdk:bom from 2.31.16 to 2.31.21
Bumps software.amazon.awssdk:bom from 2.31.16 to 2.31.21.

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-version: 2.31.21
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-14 10:09:02 +02:00
dependabot[bot]
444e3d2a77 build(deps): bump software.amazon.awssdk.crt:aws-crt
Bumps [software.amazon.awssdk.crt:aws-crt](https://github.com/awslabs/aws-crt-java) from 0.37.0 to 0.38.1.
- [Release notes](https://github.com/awslabs/aws-crt-java/releases)
- [Commits](https://github.com/awslabs/aws-crt-java/compare/v0.37.0...v0.38.1)

---
updated-dependencies:
- dependency-name: software.amazon.awssdk.crt:aws-crt
  dependency-version: 0.38.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-14 10:08:28 +02:00
Satvik Kushwaha
752405ac78 fix(flows)*: amend text color problem in no code editor on firefox (#8367)
There was an issue with title color of tasks on the `No Code` editor (which stayed dark no matter the mode selected), but only viewed in `Firefox` browser. That is sorted out with this pull request.

Closes https://github.com/kestra-io/kestra/issues/8327.

Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-04-14 09:35:39 +02:00
Miloš Paunović
860c1b218c feat(core)*: allow closing all tabs at once in a single panel (#8360)
Adds the ability to close all tabs in a single panel with a single click, improving usability and eliminating the need to close each tab individually.

Closes https://github.com/kestra-io/kestra/issues/8273.
2025-04-11 15:14:04 +02:00
Barthélémy Ledoux
0bd017556a fix(ui): command array should display properly in no-code (#8349) 2025-04-11 14:47:17 +02:00
Miloš Paunović
63ec5cab27 build(core): update commit message and description for translation-related pull requests (#8358)
This change updates the commit message and description generated by the translation CI process to be more descriptive and consistent with our commit conventions. It improves the clarity and traceability of automated translation updates in the commit history.
2025-04-11 14:38:31 +02:00
brian-mulier-p
e73f15a538 fix(core): bring back documentation on some plugins (#8354)
swagger is now resolved from Kestra classpath

closes #8265
2025-04-11 13:00:53 +02:00
github-actions[bot]
f871fa838e chore(translations)*: localize to languages other than english (#8353) 2025-04-11 12:11:38 +02:00
Miloš Paunović
c2028759e4 feat(core)*: introduce a dropdown menu for each panel (#8351)
This is the initial work to support the two related issues mentioned above, which is introducing the context menu for each tab panel, which will give us the place to list related actions.

Relates to https://github.com/kestra-io/kestra/issues/8272.
Relates to https://github.com/kestra-io/kestra/issues/8273.
2025-04-11 11:56:04 +02:00
Nicolas K.
21d6e0fa62 test(system): rework concurrency paused tests so they actually test t… (#8339)
* test(system): rework concurrency paused tests so they actually test the pause + concurrency behavior

* test(system): Add a flow name check in the queue because kafka queue is not cleaned between tests

---------

Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-04-11 11:16:01 +02:00
Bart Ledoux
ab666bff11 test(ui): failing test for autocompletiona after contributor's PR 2025-04-11 09:42:23 +02:00
Miloš Paunović
92c082e2e0 chore(flows)*: hide horizontal scroll on editor based inputs (#8347)
There was always a horizontal scrollbar present at editor compnent when used as input field, which was previously handled for the No Code editor in https://github.com/kestra-io/kestra/pull/8216.

Relates to https://github.com/kestra-io/kestra-ee/issues/3404.
2025-04-11 08:09:22 +02:00
Frank Tianyu Zeng
5a3a54fd57 feat(variables): add missing functions to editor autocompletion (#8245)
closes #7733

---------

Co-authored-by: brian.mulier <bmmulier@hotmail.fr>
2025-04-10 23:32:40 +02:00
Florian Hussonnois
1576051ebb fix(flow): re-enable NotEmpty constraint on property tasks for Sequential 2025-04-10 18:03:35 +02:00
Loïc Mathieu
4ec2a5d064 chore(system): don't emit two time a workertaskresult for RUNNING
We emit a WorkerTaskResult in RUNNING, then create an attempt, emit a new WorkerTaskResult with the attemps and then start running the task.
We can only emit one time a WorkerTaskResult as the second would be emitted microseconds after so it's just noise.
2025-04-10 18:02:42 +02:00
Florian Hussonnois
846e20a100 fix(flow): enhance error handling when injecting plugin defaults
Add a new checked exception FlowProcessingException to enhance error
handling when parsing, validating a flow, and injecting plugin defaults.

Related-to: #7894
2025-04-10 17:13:40 +02:00
Barthélémy Ledoux
433a332123 feat: use TaskTasks.vue to show subtask (#8332) 2025-04-10 16:59:10 +02:00
Loïc Mathieu
9b8b240d7c feat(core): use a fixed-size mask to mask secrets (#8131)
Fixes 8131
2025-04-10 15:17:05 +02:00
Miloš Paunović
e6937c4a8c feat(core)*: close code tab if file is deleted (#8331)
In the new multi panel view, if we have `code` tab opened and delete that file from `files` panel, we'll automatically close the opened tab for the file in question.

Closes https://github.com/kestra-io/kestra/issues/8271.
2025-04-10 14:51:43 +02:00
Nicolas K.
70dbe6a219 test(system): fix flaky runner test by adding sleep in subtask (#8329)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-04-10 14:41:21 +02:00
Mathieu Gabelle
8ceee4cfdc refactor: migrate plugin.core flowable tasks to dynamic proeprties (#8313)
* refactor: migrate plugin.core flowable tasks to dynamic proeprties

migrated properties to dynamic properties if possible
updated tests accordingly
remove unused import and fix sonar issues
2025-04-10 13:59:57 +02:00
Florian Hussonnois
b97347df97 fix(flow): handle parsing error when reading flows from repository
This commit fixes NPE in JdbcExecutor that can occurred when
reading invalid an flow, and add better handling of parsing error in JDBC flow
repositories.

Related-to: #7894
2025-04-10 13:28:11 +02:00
Piyush Bhaskar
b177a1f304 chore(ui): Add a trigger from Triggers tab (#7754)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
2025-04-10 11:11:34 +02:00
Piyush Bhaskar
999406aee4 fix(ui): uniform scrollbar across UI (#7758)
* scrollbar  uniforming

* fix(ui): uniform scrollbar across ui.

* minor tweaks

* minor tweak

---------

Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
2025-04-10 11:10:46 +02:00
Barthélémy Ledoux
31fd0303b5 fix: make monaco composition API (#8320)
* refactor: make monaco composition API

* fix kv

* rename var in monaco

* enable autocomplete in new panel

* fix: set theme even in diff mode

* refactor: avoid using computed when unnecessary
2025-04-10 10:57:28 +02:00
Florian Hussonnois
32b4a9e0be fix(core): handle parsing error when reading flows from repository
This commit fixes a regression after applying plugin default
versions in the AbstractJdbcFlowRepository
2025-04-09 22:58:24 +02:00
Barthélémy Ledoux
2c411a27cc build: use storybook test addon to get coverage (#8316) 2025-04-09 20:53:53 +02:00
Florian Hussonnois
504ff282ef fix: add missing indices for service instance table 2025-04-09 19:01:42 +02:00
Nicolas K.
365d82eb96 feat(#233)!: remove warningOnError behavior (#8321)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-04-09 18:00:20 +02:00
YannC
9a67466d9c chore(deps): upgrade micronaut-platform to 4.8.0 (#8232)
* chore(deps): upgrade micronaut-platform to 4.8.0

* fix: http client version enforcement is not needed anymore as docker-java as updated the dependency

* fix: make applicationContext protected in SchedulerPollingTriggerTest

* fix: upgrade docker-java version to 3.5.0

* fix: make some method protected in the AbstractRunnerTest so it can be override
2025-04-09 17:06:38 +02:00
Nicolas K.
a04db6510b feat(makefile): add pull-plugin script and fixe clone-plugin script (#8283)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-04-09 17:03:05 +02:00
Will Russell
ab2fea660a doc(Core): task example indentation error 2025-04-09 15:25:25 +01:00
ben8t
1c9016ba45 fix(sanitycheck): update flows id and namespace
* fix:sleep assert comparison

* fix:edit sanitycheck according to new process

* fix:typo
2025-04-09 14:46:47 +02:00
Miloš Paunović
935ff944a5 fix(ui): make sure that dropping tab places it into right place (#8315)
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
2025-04-09 14:41:01 +02:00
Will Russell
fd4667383c doc(Devcontainer): link instructions to contributing guide 2025-04-09 12:15:44 +01:00
Loïc Mathieu
5bec2fa8b3 feat(cli,core)!: change KESTRA_ env prefix by ENV_
This avoid possible security issue as you can use env var to override Kestra configuration properties and Kestra related configuration properties starts with `kestra`.

Fixes https://github.com/kestra-io/kestra-ee/issues/3131
2025-04-09 12:48:54 +02:00
Piyush Bhaskar
c24fcfd0ca chore(ui): Add a trigger from Triggers tab (#7754)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Co-authored-by: Barthélémy Ledoux <ledouxb@me.com>
2025-04-09 11:25:34 +02:00
Piyush Bhaskar
22424c69b6 chore(ui): improve the filter label colors on hover (#8141) 2025-04-09 11:17:18 +02:00
Bart Ledoux
938b3cc08b tests: fix storybook multipanel test 2025-04-09 11:15:14 +02:00
Piyush Bhaskar
5fdf2f3085 chore(ui): amend scrollbar issue on the no code editor fields (#8216) 2025-04-09 11:10:07 +02:00
Tanvir Ahmed
062957982b fix: [#8282] - Fixed issue with Devcontainer not running on arm64 architecture (#8285)
* fix: [#8282] - Fixed issue with Devcontainer not running on arm64 architecture

* chore: Updated implementation of dynamic install of Java JDK
2025-04-09 10:07:16 +01:00
Bart Ledoux
c08213b48d ci: remove some odd arguments to frontend checkout 2025-04-09 10:49:25 +02:00
Karuna Tata
152d96f018 chore(ui): remove reorder buttons when there's only one value in no code arrays (#8266) 2025-04-09 09:52:02 +02:00
AJ Emerich
4865843b10 fix(docs): add titles and examples to Dashboard chart types (#8233)
* fix(docs): add titles to Dashboard chart types

* fix(docs): add examples to Dashboard chart types

* fix(docs): add bar chart example
2025-04-09 09:19:46 +02:00
Miloš Paunović
47309a0782 chore(deps): regular dependency update (#8310) 2025-04-09 08:11:58 +02:00
Bart Ledoux
7d7340b4ba fix: remove the big plus when dragging 2025-04-08 16:11:58 +02:00
Bart Ledoux
eb1509959c chore: remove console.log drop 2025-04-08 15:50:59 +02:00
Nicolas K.
5285bea930 feat(Unit Tests) #8171 convert hamcrest to assertj (#8276)
* feat(Unit Tests) #8171 convert hamcrest to assertj

* fix(Unit Tests) #8171 failing unit test after assertj migration

---------

Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-04-08 15:48:54 +02:00
Bart Ledoux
cc5a1c9f68 fix: remove props warning in flow editor 2025-04-08 15:47:04 +02:00
MilosPaunovic
725e5e5d78 fix(ui): allow tab button to take effect while using editor 2025-04-08 15:09:30 +02:00
Bart Ledoux
56b903b8fd fix: repair docs home page 2025-04-08 15:05:08 +02:00
Miloš Paunović
21c0c86238 fix(ui): amend namespace files creation & editing problems (#8279) 2025-04-08 13:43:20 +02:00
Miloš Paunović
65ab695001 chore(ui): pass prop to maximize the height of namespace file editor (#8278) 2025-04-08 13:34:33 +02:00
Miloš Paunović
1310e59cf9 chore(ui): amend the sizing of editor panels (#8277) 2025-04-08 13:30:22 +02:00
YannC
c35352e2b4 feat: Deprecated BOOLEAN input for new BOOL input with no undefined state (#8257)
close #8225
2025-04-08 10:22:11 +02:00
Miloš Paunović
59589b1c2d refactor(ui): uniform the namespace components for oss and ee versions (#8256) 2025-04-08 09:29:19 +02:00
brian.mulier
4729430a00 feat(ui): ability to hide secret value when typing in secrets
closes kestra-io/kestra-ee#3358
2025-04-08 00:10:22 +02:00
brian.mulier
1aa37d5756 fix(core): duration defaults are rendered as duration string instead of timestamp
closes #3742
2025-04-07 22:06:24 +02:00
brian.mulier
1a121951d6 fix(jdbc): conflict on migration numbers 2025-04-07 21:59:14 +02:00
YannC
92ee3f749e fix: correct version migration for H2 sql file 2025-04-07 19:20:49 +02:00
dependabot[bot]
ae7bb88ff0 build(deps): bump com.google.guava:guava from 33.4.0-jre to 33.4.6-jre
Bumps [com.google.guava:guava](https://github.com/google/guava) from 33.4.0-jre to 33.4.6-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-07 17:45:18 +02:00
Florian Hussonnois
8f29a72df7 refactor: add GenericFlow to support un-typed flow deserialization
Add new FlowId, FlowInterface and GenericFlow classes to support
deserialization of flow with un-typed plugins (i.e., tasks, triggers)
in order to inject defaults prior to strongly-typed deserialization.
2025-04-07 17:32:06 +02:00
Loïc Mathieu
fc8732f96e chore: use @Nullable from Jakarata annotations 2025-04-07 17:01:52 +02:00
Loïc Mathieu
14f4449d99 chore(deps): Upgrade Guava to 33.4.5-jre (#8005) 2025-04-07 17:01:52 +02:00
AJ Emerich
dd80a91ab3 fix(docs): remove note about Podman rootless (#8259)
closes https://github.com/kestra-io/docs/issues/2404
2025-04-07 16:54:06 +02:00
Florian Hussonnois
840f010921 fix(core): fix NPE when generating flow graph
Fix NPE when generating flow graph and a task cannot be deserialized

Fix: kestra-io/kestra-ee#3369
2025-04-07 16:33:08 +02:00
Loïc Mathieu
8462b178cb feat(jdbc-h2,jdbc-mysql,jdbc-postgres): add an index on queues.key 2025-04-07 15:55:54 +02:00
brian.mulier
901625786d fix(ui)!: prevent infinite loading loop in Namespace KV Store & Secrets pages if there is none 2025-04-07 15:07:50 +02:00
YannC
4def8c5764 chore(ci): Implement JReleaser for GitHub Release (#8231) 2025-04-07 13:34:33 +02:00
brian.mulier
65a204356c fix(ui): bump ui-libs to 0.0.168 2025-04-07 12:03:06 +02:00
dependabot[bot]
73e3fd08e9 build(deps): bump software.amazon.awssdk:bom from 2.31.11 to 2.31.16
Bumps software.amazon.awssdk:bom from 2.31.11 to 2.31.16.

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-version: 2.31.16
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-07 11:29:42 +02:00
dependabot[bot]
0665b52014 build(deps): bump org.owasp.dependencycheck from 12.1.0 to 12.1.1
Bumps org.owasp.dependencycheck from 12.1.0 to 12.1.1.

---
updated-dependencies:
- dependency-name: org.owasp.dependencycheck
  dependency-version: 12.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-07 11:05:05 +02:00
dependabot[bot]
781f9dc8d8 build(deps): bump com.google.cloud:libraries-bom from 26.58.0 to 26.59.0
Bumps [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) from 26.58.0 to 26.59.0.
- [Release notes](https://github.com/googleapis/java-cloud-bom/releases)
- [Changelog](https://github.com/googleapis/java-cloud-bom/blob/main/release-please-config.json)
- [Commits](https://github.com/googleapis/java-cloud-bom/compare/v26.58.0...v26.59.0)

---
updated-dependencies:
- dependency-name: com.google.cloud:libraries-bom
  dependency-version: 26.59.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-07 11:04:49 +02:00
dependabot[bot]
92e4570158 build(deps): bump org.jooq:jooq from 3.20.2 to 3.20.3
Bumps org.jooq:jooq from 3.20.2 to 3.20.3.

---
updated-dependencies:
- dependency-name: org.jooq:jooq
  dependency-version: 3.20.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-07 11:04:36 +02:00
brian.mulier
ce47b4ee5e fix(ui): bump ui-libs to 0.0.167 2025-04-07 09:54:35 +02:00
brian.mulier
69193c6096 Revert "fix(core): add additionalProperties=false to tasks to have a warning if there is unknown properties"
This reverts commit 6fee99a78a.
2025-04-07 09:54:30 +02:00
Bart Ledoux
9d437957fa chore: add a rule to prevent angle brackets cast in TS 2025-04-07 09:49:10 +02:00
Ludovic DEHON
dfea86fb07 chore(deps): update pebble to 3.2.4
affected by CVE-2025-1686
2025-04-06 18:23:55 +02:00
brian.mulier
6fee99a78a fix(core): add additionalProperties=false to tasks to have a warning if there is unknown properties 2025-04-05 02:14:20 +02:00
brian.mulier
2c766a5497 feat(webserver): move oneOf to anyOf in JsonSchemaGenerator to have better autocompletion 2025-04-05 01:52:41 +02:00
brian.mulier
959737f545 feat(ui): introduce patched version of monaco-yaml & yaml-language-server to have better autocompletion
waiting for https://github.com/redhat-developer/yaml-language-server/pull/1048
2025-04-05 01:05:27 +02:00
brian.mulier
2c1b6ffe3c fix(ui): exclude additional files from UI coverage report 2025-04-04 16:53:29 +02:00
brian.mulier
9b819f6925 fix(ui): avoid jsx confusion on cast that prevent Storybook from running 2025-04-04 16:24:02 +02:00
brian.mulier
a81c0a5737 fix(ui): move bunch of utils to typescript 2025-04-04 15:38:15 +02:00
brian.mulier
ebcb0bd2a2 feat(ui): add plugin icons to auto-completions + make autocompletion work upon writing full package 2025-04-04 15:28:53 +02:00
Loïc Mathieu
2a26c415bf feat(jdbc): avoid using the WorkerTaskResult queue when possible
Instead, directly process the result (add the taskrun) to avoid one rountrip inside the queue.
2025-04-04 14:15:13 +02:00
Loïc Mathieu
1e06b9f1c0 feat(jdbc): allow disabling queue cleaning. 2025-04-04 13:43:09 +02:00
Loïc Mathieu
0b64da5e84 fix(core): default namespace in namespace file 2025-04-04 11:51:00 +02:00
AJ Emerich
d95b65082f fix(docs): add title, description, and example to Dashboard chart data (#8220)
* fix(docs): add title, description, and example to Metrics chart data

* fix(docs): add title, description, example for Logs and Executions

* fix(docs): fix line break in Assert plugin docs
2025-04-04 08:18:14 +02:00
Mathieu Gabelle
1bba3e4035 refactor: migrate plugin condition to dynamic properties (#6907)
* refactor: migrate plugin condition to dynamic properties
2025-04-04 08:18:01 +02:00
Will Russell
efe0c6e1e4 fix(docs): update task type in example to updated name 2025-04-03 17:19:02 +01:00
Nicolas K.
fce7ec135e fire(core): flaky allow failure for each item test (#8228)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-04-03 18:00:15 +02:00
Nicolas K.
5a2456716f fix(jdbc): #8219 unquoted timestamp field breaking query (#8222)
* fix(jdbc): #8219 unquoted timestamp field breaking query

* fix(jdbc): #8219 fix hard coded quoted field

---------

Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-04-03 14:54:05 +02:00
Loïc Mathieu
f43f8c2dc0 fix(core): be tolerant of decryption issue
If we cannot decrupt outputs, let's ignore the outpot and log a warning.
This may only happen on configuratin mismatch between nodes.
2025-04-03 13:57:41 +02:00
YannC
94811d4e06 chore(ci): align plugins handle for docker publish on EE CI 2025-04-03 13:16:41 +02:00
YannC
b7166488be chore(ci): modify publish docker to align on EE 2025-04-03 11:56:11 +02:00
brian.mulier
78066fef62 fix(ui): remove all warnings from MonacoEditor.vue 2025-04-02 19:33:04 +02:00
brian.mulier
c70abcb85f fix(ui): auto-resize suggest window to fit tasks
closes #5823
2025-04-02 17:01:36 +02:00
YannC
774a4ef7a1 Feat(): ci changes (#8217)
* fix(): avoid running release workflow on releases branch

* feat(): avoid running CI on draft PR

close #4964

* fix(ci): only publish docker image in workflow release if develop branch or specific asked

close #8136
2025-04-02 15:51:28 +02:00
Florian Hussonnois
a48fd02ed7 fix(cli): fix NPE for commands not requiring plugins (#8212)
Fix: #8212
2025-04-02 14:51:41 +02:00
Loïc Mathieu
030d948521 chore(core): remove un-used attribute in FlowWithWorkerTrigger 2025-04-02 12:18:23 +02:00
Loïc Mathieu
c4aa6c1097 fix(jdbc): possible deadlock on service instance
If multiple Executors restart at the same time and there was a not of worker task to resubmit, there was a possible deadlock as the service instance table is selected for update so it can block other executors.
Using skipped lock avoid that and is still correct as other executors can skip the dead instance handling as it was already in process by the first executor.

findById was not changed in this commit as it's not part of the worker task resubmission process.
2025-04-02 10:53:54 +02:00
Loïc Mathieu
3d1a3d0e7a fix(ci): use the right GitHub token for test report 2025-04-02 10:53:28 +02:00
Loïc Mathieu
30ab030244 fix(gradle): Windows selfrun.bat
Fixes https://github.com/kestra-io/kestra-ee/issues/3324
2025-04-02 09:39:59 +02:00
brian.mulier
cc083385f0 fix(ui): better autocompletion relevance
closes #7709
2025-04-01 19:33:42 +02:00
Shruti Mantri
c14462f5fa feat: add afterExecution to basic.md (#8126)
* feat: add afterExecution to basic.md

* Update ui/src/assets/docs/basic.md

---------

Co-authored-by: AJ Emerich <aemerich@kestra.io>
2025-04-01 16:42:51 +02:00
Mathieu Gabelle
d6e470d788 fix!: update pullPolicy default value to IF_NOT_PRESENT (#8170) 2025-04-01 11:24:59 +02:00
Loïc Mathieu
58ae507e21 fix(core): mask secrets on log attributes
Fixes #3282
2025-04-01 10:09:11 +02:00
Loïc Mathieu
71110ccfc3 Revert "fix(core): HttpClient log the URL even if it's a secret"
This reverts commit 54aa935702.
2025-04-01 10:09:11 +02:00
YannC
fdcc07b546 fix(): allows namespace to be search by in filter 2025-04-01 10:07:52 +02:00
Mathieu Gabelle
221236e079 fix(scripts)!: update pull policy to IF_NOT_PRESENT (#8169)
BREAKING CHANGE: in accordance with new Docker Hub pull policy regulation, the default kestra pull policy will change from ALWAYS to IF_NOT_PRESENT
2025-04-01 08:12:39 +02:00
Ludovic DEHON
d14deaceb0 fix(core): add a meaningful log for flow that can inject defaults 2025-03-31 18:22:16 +02:00
YannC
bfdc48bbbe fix(cli): prevent FlowUpdatesCommand to crash due to plugin loader 2025-03-31 17:45:32 +02:00
brian.mulier
e6b2f1f79a fix(ui-ee): keep fetching if filtered kvs & secrets have no elements after fetch
might close kestra-io/kestra-ee#3311
2025-03-31 16:12:45 +02:00
Loïc Mathieu
0632052837 fix(core): use a stable flow logger name
If we keep the executionId in it, as it's now used to create the forward logger, a new logger will be created for each execution.
This may also fix a memory leak.
2025-03-31 14:56:13 +02:00
Florian Hussonnois
3df9d49aa0 ci: fix setversion-tag and devtools 2025-03-31 13:50:27 +02:00
Loïc Mathieu
318f2b7d5a chore(cli,jdbc-postgres): fix some compilation warnings 2025-03-31 13:17:35 +02:00
Roman Acevedo
800970a88f chore(deps): add bouncycastle:bcpkix-jdk18on to platform 2025-03-31 11:52:33 +02:00
dependabot[bot]
f717063a83 build(deps): bump org.sonarqube from 6.0.1.5171 to 6.1.0.5360
Bumps org.sonarqube from 6.0.1.5171 to 6.1.0.5360.

---
updated-dependencies:
- dependency-name: org.sonarqube
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-31 11:20:34 +02:00
Barthélémy Ledoux
9a7fb64943 fix: make flowWarnings show to unlock saving (#8157)
closes #8115
2025-03-31 10:59:46 +02:00
Barthélémy Ledoux
45bddb8d09 fix: task array needed better typings (#8158)
closes #8117
2025-03-31 10:59:32 +02:00
dependabot[bot]
881b009d9b build(deps): bump flyingSaucerVersion from 9.11.4 to 9.11.5
Bumps `flyingSaucerVersion` from 9.11.4 to 9.11.5.

Updates `org.xhtmlrenderer:flying-saucer-core` from 9.11.4 to 9.11.5
- [Release notes](https://github.com/flyingsaucerproject/flyingsaucer/releases)
- [Changelog](https://github.com/flyingsaucerproject/flyingsaucer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flyingsaucerproject/flyingsaucer/compare/v9.11.4...v9.11.5)

Updates `org.xhtmlrenderer:flying-saucer-pdf` from 9.11.4 to 9.11.5
- [Release notes](https://github.com/flyingsaucerproject/flyingsaucer/releases)
- [Changelog](https://github.com/flyingsaucerproject/flyingsaucer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flyingsaucerproject/flyingsaucer/compare/v9.11.4...v9.11.5)

---
updated-dependencies:
- dependency-name: org.xhtmlrenderer:flying-saucer-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.xhtmlrenderer:flying-saucer-pdf
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-31 10:46:21 +02:00
dependabot[bot]
ab818713f6 build(deps): bump software.amazon.awssdk:bom from 2.31.6 to 2.31.11
Bumps software.amazon.awssdk:bom from 2.31.6 to 2.31.11.

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-31 10:27:54 +02:00
Loïc Mathieu
d68ffa3109 chore: disable tests that are too flaky
An issue will be open to track them and re-enabled them later.
2025-03-31 10:27:46 +02:00
dependabot[bot]
addd76f9bb build(deps): bump com.azure:azure-sdk-bom from 1.2.32 to 1.2.33
Bumps [com.azure:azure-sdk-bom](https://github.com/azure/azure-sdk-for-java) from 1.2.32 to 1.2.33.
- [Release notes](https://github.com/azure/azure-sdk-for-java/releases)
- [Commits](https://github.com/azure/azure-sdk-for-java/compare/azure-sdk-bom_1.2.32...azure-sdk-bom_1.2.33)

---
updated-dependencies:
- dependency-name: com.azure:azure-sdk-bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-31 09:58:43 +02:00
dependabot[bot]
6be939c1bd build(deps): bump software.amazon.awssdk.crt:aws-crt
Bumps [software.amazon.awssdk.crt:aws-crt](https://github.com/awslabs/aws-crt-java) from 0.36.3 to 0.37.0.
- [Release notes](https://github.com/awslabs/aws-crt-java/releases)
- [Commits](https://github.com/awslabs/aws-crt-java/compare/v0.36.3...v0.37.0)

---
updated-dependencies:
- dependency-name: software.amazon.awssdk.crt:aws-crt
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-31 09:44:07 +02:00
dependabot[bot]
4ea876d3fe build(deps): bump com.google.cloud:libraries-bom from 26.57.0 to 26.58.0
Bumps [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) from 26.57.0 to 26.58.0.
- [Release notes](https://github.com/googleapis/java-cloud-bom/releases)
- [Changelog](https://github.com/googleapis/java-cloud-bom/blob/main/release-please-config.json)
- [Commits](https://github.com/googleapis/java-cloud-bom/compare/v26.57.0...v26.58.0)

---
updated-dependencies:
- dependency-name: com.google.cloud:libraries-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-31 09:43:26 +02:00
Miloš Paunović
42d8005eff chore(ui): show blueprint id field in case of missing title (#8154) 2025-03-31 09:40:20 +02:00
Nicolas K.
58a360fae0 feat(core-ee): #7501 split file log exporter to multiple files (#8138)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-28 17:42:41 +01:00
Loïc Mathieu
49b647e1fc fix(core): add missing docker plugin subgroup icon 2025-03-28 16:15:41 +01:00
Florian Hussonnois
b314fc393b fix(cli): properly register plugins uninstall cmd 2025-03-28 15:40:09 +01:00
Florian Hussonnois
0a298cad17 fix(core): allow dash in plugin version qualifier 2025-03-28 15:18:37 +01:00
Shruti Mantri
61170e6067 feat: add example for HasRetryAttempt condition (#8133) 2025-03-28 14:14:42 +00:00
brian.mulier
cfbffad31a fix(ui): search bars are properly working in secrets & KV pages
closes #8110
closes kestra-io/kestra-ee#3290
2025-03-28 15:02:02 +01:00
Miloš Paunović
41e2dac4ca chore(ui): add padlock icon to secrets menu item (#8129) 2025-03-28 12:44:28 +01:00
Miloš Paunović
0fa8386cb3 chore(ui): amend file tree context menu link colors (#8123) 2025-03-28 12:10:08 +01:00
github-actions[bot]
0f45c009ab chore(translations): localize to languages other than English (#8120) 2025-03-28 11:27:25 +01:00
MilosPaunovic
b86177f329 chore(translations): uniform translation keys with entrerprise edition 2025-03-28 11:21:08 +01:00
brian.mulier
fe396c455b fix(ui): global secret page design
closes kestra-io/kestra-ee#3268
2025-03-28 11:18:34 +01:00
brian.mulier
0830e11645 fix(ui): repair tenant translation 2025-03-28 11:18:34 +01:00
brian.mulier
4d7f6b2bb1 fix(ui): add routeContext where it was missing 2025-03-28 11:18:34 +01:00
brian.mulier
955c6b728b fix(webserver): handle out-of-bounds (>) namespaces fetch 2025-03-28 11:18:34 +01:00
brian.mulier
d2d26351bd fix(core): namespace service now properly detects namespaces with flows inside 2025-03-28 11:18:34 +01:00
Loïc Mathieu
f14b638f73 fix(core): compilation issue 2025-03-28 09:26:01 +01:00
Loïc Mathieu
259b5b5282 Revert "fix(core): require condition in Flow trigger (#7494)"
This reverts commit c5767fd313.
2025-03-28 09:12:40 +01:00
Miloš Paunović
b1c50374b4 chore(ui): handle editor blueprint loading problem (#8113) 2025-03-28 08:57:24 +01:00
Roman Acevedo
de2d923bd4 fix: doc and deprecated field was not showing for dynamic non-string properties (#8006)
fix: doc and deprecated field was not showing for dynamic non-string properties (#8006)
2025-03-27 18:33:23 +01:00
Florian Hussonnois
89c76208a4 fix(core): avoid flow validation error on plugin alias duplicates 2025-03-27 17:59:04 +01:00
YannC
12eb8367ec feat(): add new crudeventtype "account_locked" (#8103) 2025-03-27 17:08:04 +01:00
Nicolas K.
7421f1e93d fix(kafka runner): #2709 filter child forEach tasks before merging th… (#8095)
* fix(kafka runner): #2709 filter child forEach tasks before merging the output, and add sleep before restarting flows to ensure failure is persisted

* feat(kafka runner): #2709 wait until executions are persisted as Failed in the database before restarting

* fix(runner): put back the sleep instead of the wait

* clean(runner): remove unused variables

---------

Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-27 16:56:37 +01:00
Loïc Mathieu
5642a53893 fix(core): properly fix the issue with MapUtils.flattenToNestedMap 2025-03-27 16:28:11 +01:00
Loïc Mathieu
d5a2f4430f fix(core): HttpClient log the URL even if it's a secret
Fixes https://github.com/kestra-io/kestra/issues/8092
2025-03-27 16:00:43 +01:00
rajatsingh23
0299e0d5ce chore(ui): Hide duration switch on empty flow dashboard (#8083)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-03-27 15:48:01 +01:00
Barthélémy Ledoux
c69ac99a7f refactor: remove dead code from monacoeditor (#8088)
* refactor(MonacoEditor): improve editor state management and type safety after refactor

* fix: make theme proper valid theme

* fix only types

* remove getEditor

* more fixes

* add vuex store types in options API

* update vuex types

* fix types

* remove unused mapping

* update vuex shims

* fix types
2025-03-27 14:08:59 +01:00
brian.mulier
fd1b4d5234 fix(ui): properly detect yaml to inject json schema into MonacoEditor
closes #8090
2025-03-27 13:39:00 +01:00
Miloš Paunović
4ef3600954 chore(ui): pass custom height property to execution output debug editors (#8100) 2025-03-27 13:36:29 +01:00
Miloš Paunović
366df0f37f chore(ui): include labels of saved search filter on page reload (#8099) 2025-03-27 13:17:11 +01:00
brian.mulier
5e87655a0e fix(webserver): first eval without masking secret function to error in case of missing secret
closes #8094
2025-03-27 13:09:31 +01:00
Miloš Paunović
0f01699d27 chore(ui): make app & dashboard editors re-sizable (#8096) 2025-03-27 12:01:40 +01:00
Loïc Mathieu
0794b2bf8e fix(core): flatten map should not throw an exception
As it is called inside the Executor, it must be fail-safe.
2025-03-27 10:57:32 +01:00
Loïc Mathieu
0becaa0b97 fix(core): charset should not be taken from
Fixes #8072
2025-03-27 10:02:23 +01:00
AJ Emerich
7d3bb34fd4 feat(docs): add kestra.environment and kestra.url expressions (#8085)
https://github.com/kestra-io/kestra-ee/issues/3095
2025-03-27 09:53:24 +01:00
Florian Hussonnois
b8c55baff1 fix(cli): make worker args available through static KestraContext
part-of: kestra-io/kestra-ee#3259
2025-03-26 15:53:51 +01:00
Loïc Mathieu
1f8e5ad18e feat(*): add new methods findAllAsync for the backup 2025-03-26 14:04:01 +01:00
Miloš Paunović
945f59afd0 chore(ui): improve saved search filtering functionality (#8073)
* chore(ui): passing prefix down  to saved search label component

* chore(ui): check if filters have operation field on encoding

* chore(ui): trigger search automatically on choosing the saved item
2025-03-26 11:16:31 +01:00
YannC
491fb54e2c fix(): add missing command to flow subcommand 2025-03-26 10:23:51 +01:00
github-actions[bot]
154f145d4a chore(translations): localize to languages other than English (#8071) 2025-03-26 10:16:39 +01:00
yuri
8a73edd772 feat(ui): improve styling of saved filter searches (#8040)
Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2025-03-26 09:57:02 +01:00
Miloš Paunović
660fcd7ec6 chore(deps): regular dependency update (#8070) 2025-03-26 08:46:50 +01:00
YannC
675aa666ab chore(ci): pass plugin version to docker workflow 2025-03-25 18:43:03 +01:00
Florian Hussonnois
451de81d03 fix(build): fix gradle property for release 2025-03-25 17:38:57 +01:00
YannC
323527d7e9 chore(ci): lower build-artifacts workflow so github release can use it 2025-03-25 17:28:45 +01:00
Florian Hussonnois
ea7c709f56 chore: upgrade to version 'v0.23.0-SNAPSHOT' 2025-03-25 16:34:02 +01:00
Barthélémy Ledoux
bc8bd7b00b feat: cleaner multipanel tab move (#8029)
* feat: better/simpler display for tab movement

* Revert "remove tests to make sure we merge"

This reverts commit a979232f6c.

* fix some tests

* better tests

* fix: make it work even after first move

* fix the blinking

* add a few tests

* one fix for split panel one for size

* fix test ?

* fix: scroll all the way

* make sure tab 4 is visible

* better borders

* fix dem darn tests
2025-03-25 15:46:05 +01:00
YannC
40f6334b0b fix(): datetime filtering issues (#8027)
close #7959
2025-03-25 14:48:27 +01:00
Nicolas K.
71151f5ac2 feat(core): #5467 add inheritance for KV in pebble and Get task (#8031)
* feat(core): #5467 add inheritance for KV in pebble and Get task

* fix(core): #5467 error when namespace don't contain dot

---------

Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-25 14:46:17 +01:00
brian.mulier
75e7635505 fix(ui): make sure global secret view iterates over all secrets 2025-03-25 13:43:05 +01:00
Florian Hussonnois
6c9dc8fba4 fix(core): fix NPE when closing standalone runner 2025-03-25 12:26:50 +01:00
brian.mulier
28d1f005a4 fix(ui): fail-safe secrets API calls on global secrets view 2025-03-25 12:09:07 +01:00
AJ Emerich
7fd7a6fcbc feat(docs): add example for best practice with multiline json http post request (#8023) 2025-03-25 11:42:27 +01:00
yuri
57707faf37 fix(ui): amend operator value of labels inside the filter (#8028) 2025-03-25 11:20:04 +01:00
brian.mulier
c64c2c7104 fix(ui): fail-safe secrets API calls 2025-03-25 11:08:22 +01:00
YannC
8f9b2fc0db fix(): use correct property for blueprint type
close #3227
2025-03-25 10:22:49 +01:00
Satvik Kushwaha
aab9e55794 chore(ui): limit flow run dialog maximum height (#7938)
Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2025-03-25 10:02:50 +01:00
Miloš Paunović
f9d48d6f74 fix(ui): allow multi label filtering (#8022) 2025-03-25 09:52:54 +01:00
Florian Hussonnois
267d848d3f fix(core): ensure defaults can be injected in flows (#3206)
Flow revisions create from older Kestra versions may not be linked to their original source.
In such cases, fall back to the generated source approach to enable plugin default injection.

close: #3206
2025-03-25 09:17:55 +01:00
brian.mulier
edbf14c1b8 fix(ui): handle properly layout of global Secrets when there is a secret manager 2025-03-25 02:45:21 +01:00
brian.mulier
d9ac267161 feat(ui): introduce global Secrets page
closes kestra-io/kestra-ee#1257
2025-03-25 01:39:18 +01:00
brian.mulier
74455ad993 fix(ui): prevent function parameters autocompletion from deleting parenthesis 2025-03-25 01:39:18 +01:00
YannC
e2dd02173f fix(): use correctly context-path when needed (#8018) 2025-03-24 20:30:55 +01:00
YannC
c7b3a42f34 fix(): make absolute date work with metrics (#8015)
close #7951
2025-03-24 18:46:45 +01:00
Rudhra Bharathy G
fe882bbee6 fix(): moved the logic to the backend (#5917)
close #5740
Co-authored-by: YannC <ycoornaert@kestra.io>
2025-03-24 17:59:54 +01:00
YannC
1efb21c087 fix(): make KestraFilter works with legacy default dashboard + avoid showing custom dashboard in namespace/flow pages (#8012)
close #7959
2025-03-24 16:33:48 +01:00
github-actions[bot]
beb6182104 chore(translations): localize to languages other than English (#8013)
Co-authored-by: GitHub Action <actions@github.com>
2025-03-24 16:24:15 +01:00
Miloš Paunović
7f0d3521db chore(ui): add anchors to plugin documentation (#8014) 2025-03-24 16:11:47 +01:00
benjoEK1337
7e4453632c fix(core): Handling for trailing slash in the KESTRA_URL configuration (#6373)
* fix(core): Handling for trailing slash in the KESTRA_URL configuration

* Update webserver/src/main/java/io/kestra/webserver/controllers/api/ExecutionController.java

---------

Co-authored-by: Miloš Paunović <paun992@hotmail.com>
Co-authored-by: Loïc Mathieu <loikeseke@gmail.com>
2025-03-24 16:10:43 +01:00
Loïc Mathieu
4a3d6b30d2 feat(core,jdbc): reset the trigger inside the JdbcExecutor
Fixes #7237
2025-03-24 15:54:53 +01:00
Florian Hussonnois
fe7c14c048 chore: refactor StorageInterfaceFactory to be a bean
changes:
* extract class for ServiceType

related-to: kestra-io/kestra-ee#3021
2025-03-24 15:45:24 +01:00
Barthélémy Ledoux
7b007aafc0 feat: parse docs is 2 steps to avoid user waiting with nothing (#7149) 2025-03-24 15:25:46 +01:00
Barthélémy Ledoux
f6aa2729b2 feat: multi panel editor without the refactor (#7971)
* refactor: move logic out of EditorView.vue and into store flow.js

* fix create mode

* feat: add multi panel flow editor

* missing flow id

* fix helpers tests ?

* restore helpers test

* fix one failing test

* fix: remove non-working stories

* remove size when one does split

* fix duplicated simulation

* fix tests

* try and fix the test

* remove tests to make sure we merge
2025-03-24 15:13:26 +01:00
Nicolas K.
0ca25445fd fix(platform): move slf4j api to enforce platform to fix it's version in test (#8007)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-24 14:25:16 +01:00
Piyush Bhaskar
4900963cb9 fix(ui): amend task source tabs colors (#8010) 2025-03-24 14:01:48 +01:00
Piyush Bhaskar
21d264f444 chore(ui): highlight first item in global search bar autocomplete (#8009) 2025-03-24 13:43:27 +01:00
Miloš Paunović
b6a3d0cd54 chore(ui): removing dynamic section of collapsible plugin properties (#8008) 2025-03-24 13:11:29 +01:00
Bart Ledoux
3244b1c293 fix: remove unwanted change in Curl 2025-03-24 11:52:49 +01:00
Florian Hussonnois
0987d0b349 feat(core): add new subkey arg to secret pebble function
Add new `subkey` named argument to the pebble function `secret`
to be able to select a specific field inside a secret containing
a JSON value

close: kestra-io/kestra-ee#3200
2025-03-24 11:43:14 +01:00
dependabot[bot]
5a2ac895ec build(deps): bump software.amazon.awssdk.crt:aws-crt
Bumps [software.amazon.awssdk.crt:aws-crt](https://github.com/awslabs/aws-crt-java) from 0.36.2 to 0.36.3.
- [Release notes](https://github.com/awslabs/aws-crt-java/releases)
- [Commits](https://github.com/awslabs/aws-crt-java/compare/v0.36.2...v0.36.3)

---
updated-dependencies:
- dependency-name: software.amazon.awssdk.crt:aws-crt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-24 11:14:10 +01:00
Loïc Mathieu
7421693445 fix(core): race while initializing trigger + possible duplicate update 2025-03-24 11:10:17 +01:00
Loïc Mathieu
19894dbcd6 fix(core): race in the FlowListener
It sometimes throw ConcurrentModificationException while sending flow updates to the consumers.
2025-03-24 11:10:17 +01:00
dependabot[bot]
62e37f3b15 build(deps): bump software.amazon.awssdk:bom from 2.31.1 to 2.31.6
Bumps software.amazon.awssdk:bom from 2.31.1 to 2.31.6.

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-24 10:48:44 +01:00
dependabot[bot]
37af61f41d build(deps): bump com.github.oshi:oshi-core from 6.7.1 to 6.8.0
Bumps [com.github.oshi:oshi-core](https://github.com/oshi/oshi) from 6.7.1 to 6.8.0.
- [Release notes](https://github.com/oshi/oshi/releases)
- [Changelog](https://github.com/oshi/oshi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/oshi/oshi/compare/oshi-parent-6.7.1...oshi-parent-6.8.0)

---
updated-dependencies:
- dependency-name: com.github.oshi:oshi-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-24 10:29:32 +01:00
dependabot[bot]
491f07296a build(deps): bump com.google.cloud:libraries-bom from 26.56.0 to 26.57.0
Bumps [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) from 26.56.0 to 26.57.0.
- [Release notes](https://github.com/googleapis/java-cloud-bom/releases)
- [Changelog](https://github.com/googleapis/java-cloud-bom/blob/main/release-please-config.json)
- [Commits](https://github.com/googleapis/java-cloud-bom/compare/v26.56.0...v26.57.0)

---
updated-dependencies:
- dependency-name: com.google.cloud:libraries-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-24 10:29:14 +01:00
dependabot[bot]
6d59630a67 build(deps): bump com.microsoft.playwright:playwright
Bumps [com.microsoft.playwright:playwright](https://github.com/microsoft/playwright-java) from 1.50.0 to 1.51.0.
- [Release notes](https://github.com/microsoft/playwright-java/releases)
- [Commits](https://github.com/microsoft/playwright-java/compare/v1.50.0...v1.51.0)

---
updated-dependencies:
- dependency-name: com.microsoft.playwright:playwright
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-24 10:28:36 +01:00
YannC
a098650559 fix(): restarting trigger does not break the list anymore (#8004)
close #7712
2025-03-24 10:21:19 +01:00
Barthélémy Ledoux
505049a69e refactor: move flow editor logic into flow store (#7968)
* refactor: move logic out of EditorView.vue and into store flow.js

* fix create mode

* missing flow id

* fix helpers tests ?

* restore helpers test

* remove unfinished test files

* fix plugin docs

* remove issues on lowcode
2025-03-24 09:48:17 +01:00
yuri1969
a1abd28a3d fix(build): Gradle space-assignment deprecation 2025-03-24 09:26:44 +01:00
yuri1969
49a29c4bf1 fix(core): amend server start announcements
Servers announced their start even when their async startup process
was still in progress. This lead to confusing situations - a server
announced a successfull start but in fact it failed just few
moments later.

Note, the log message needs to be ported to EE server impls.
2025-03-24 09:25:00 +01:00
Miloš Paunović
49af8f336a fix(ui): include parameters into request for plugin schema fetching (#8002) 2025-03-24 09:12:52 +01:00
Nicolas K.
8642771a2b fix(core): working dir interface contract by putting back putFile(path, content) (#7980)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-21 16:58:51 +01:00
Nicolas K.
e44487c8d8 feat(core): #7932 add file exist comportment to NamespaceFiles (#7979)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-21 15:17:12 +01:00
Ludovic DEHON
f29aab0719 feat(core): speed up namespace file download and add a log & metrics
close #7871
close #7870
2025-03-21 13:44:16 +01:00
Loïc Mathieu
73c7a2d3d3 feat(core): allow null or empty proxy address to bypass proxy conf
Fixes #7961
2025-03-21 12:36:11 +01:00
brian.mulier
773a6e9093 fix(ui): change plugin doc properly upon switching plugin type
closes #7876
2025-03-21 11:51:44 +01:00
Ludovic DEHON
df1bbcfb7f fix(cicd): display ui unit test status 2025-03-21 09:53:49 +01:00
brian.mulier
f3419084fa fix(ui): filter tests were not up-to-date 2025-03-21 09:49:07 +01:00
brian.mulier
ab7788aeaf feat(ui): introduce global KV Store view
part of kestra-io/kestra-ee#1257
closes kestra-io/kestra-ee#1743
2025-03-21 09:49:07 +01:00
YannC
99e06632e0 fix(): prevent using LABELS in JDBC
link #6655
2025-03-20 21:54:41 +01:00
YannC
7b3d5a593e fix(): remove old '$eq' to avoid error in dashboard
link #7959
2025-03-20 21:09:09 +01:00
Bart Ledoux
cca7ed0bff fix: update ui-libs 2025-03-20 18:33:56 +01:00
Loïc Mathieu
481138e433 fix(core): add metric Publish task doc & icon
fixes #7154
2025-03-20 16:24:38 +01:00
Barthélémy Ledoux
267ffb69dc fix: only run posthog in prod mode (#7952) 2025-03-20 14:40:47 +01:00
Loïc Mathieu
43f1374aa3 chore: add GraalVM plugin 2025-03-20 13:10:39 +01:00
Loïc Mathieu
fe944ccc5b chore: add jattach into our Docker image 2025-03-20 11:57:55 +01:00
YannC
cec8702fc6 feat(): allow to specify a parent namespace in the flow updates command, and the possibility to allow child namespace in the flow bulk create command (#7956) 2025-03-20 11:24:23 +01:00
YannC
f898a679c4 fix(): Adapt the usage of the new query filter for custom dashboard (#7935)
close #7091
2025-03-19 17:13:56 +01:00
Nicolas K.
9a9d59f433 fix(core-ee): NPE when execution labels are null (#7950)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-19 17:09:17 +01:00
Loïc Mathieu
0579e23a20 fix(core): triggers don't have an execution ID already 2025-03-19 15:39:08 +01:00
YannC
343367d8c5 feat(): save last dashboard used in local storage (#7948)
close kestra-io/kestra-ee#2615
2025-03-19 14:45:45 +01:00
Miloš Paunović
d47e8083ac fix(ui): prevent filter text prefix to show up when searching locally (#7947) 2025-03-19 13:51:24 +01:00
Miloš Paunović
7c4d6b02a7 chore(ui): introduce a refresh functionality to execution logs page (#7946) 2025-03-19 12:07:34 +01:00
Miloš Paunović
95dd1b8a55 chore(deps): regular dependency update (#7944) 2025-03-19 10:41:29 +01:00
Loïc Mathieu
0ec2d88429 Revert "build(deps): bump protobufVersion from 3.25.5 to 4.30.1"
This reverts commit 6946c92683.
2025-03-19 10:27:44 +01:00
dependabot[bot]
c3b504a9ed build(deps): bump pdfjs-dist from 4.10.38 to 5.0.375 in /ui (#7942)
Bumps [pdfjs-dist](https://github.com/mozilla/pdf.js) from 4.10.38 to 5.0.375.
- [Release notes](https://github.com/mozilla/pdf.js/releases)
- [Commits](https://github.com/mozilla/pdf.js/compare/v4.10.38...v5.0.375)

---
updated-dependencies:
- dependency-name: pdfjs-dist
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-19 10:25:58 +01:00
Loïc Mathieu
8cba4dab62 fix(core): possible NPE if no manifest 2025-03-19 10:03:12 +01:00
Miloš Paunović
1b22d1e3d1 fix(ui): amend dependabot errors with parsing of package-lock.json file (#7941) 2025-03-19 09:33:20 +01:00
Loïc Mathieu
6194f244c8 feat(core): require existing namespace
Part-of: https://github.com/kestra-io/kestra-ee/issues/3071
2025-03-18 15:56:36 +01:00
Loïc Mathieu
11a1664865 feat(*): allow disabling flow logs and use a specific logger for executions, tasks and triggers
Fixes #7753
2025-03-18 15:53:30 +01:00
Miloš Paunović
db394f6371 chore(ui): remove the flow topology control button from image export (#7937) 2025-03-18 15:32:21 +01:00
Loïc Mathieu
a7433c6f62 feat(core): add correlationId to the Flow trigger
Fixes #7920
2025-03-18 14:35:59 +01:00
Loïc Mathieu
745c64c4b0 feat(model, core): document tasks props that are internal storage URI
Fixes https://github.com/kestra-io/kestra/issues/681
2025-03-18 14:29:57 +01:00
YannC
5d15e2081c fix(*): Improve log timestamp precision + allow to override timestamp… (#7847)
* fix(*): Improve log timestamp precision + allow to override timestamp in log consumer

close #5915
close #7243
close #5662

* chore(): review changes
2025-03-18 14:25:24 +01:00
Miloš Paunović
7d098eaa4e feat(ui): add beta badge global component (#7934) 2025-03-18 11:56:43 +01:00
Florian Hussonnois
0ee9abb372 fix: OpenTelemetry should be optional
OpenTelementry must be optional when OTEL is globally disabled
in micronaut.
2025-03-18 10:34:23 +01:00
github-actions[bot]
3f542ae737 chore(translations): localize to languages other than English (#7929) 2025-03-18 10:20:49 +01:00
rajatsingh23
811cc7722a feat(ui): add the option to copy single/all logs to clipboard (#7755)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-03-18 09:49:54 +01:00
Florian Hussonnois
f4f8ecb247 feat!: make kv pebble function raise error by default (#7855)
BREAKING CHANGE: use 'true' as default value for kv function arg errorOnMissing.

close: #7855
2025-03-18 09:24:16 +01:00
Florian Hussonnois
3566c4d365 fix(cli): return exit 0 in CLI plugins cmd
The CLI plugins install command must return exit code 0
even when no plugin are installed
2025-03-18 09:23:58 +01:00
YannC
a7421987b7 chore(): change docker image used for AbstractTaskRunnerTest 2025-03-18 09:19:17 +01:00
ben8t
c640b8208b [fix] fix sleep assert comparison (#7924) 2025-03-18 09:16:20 +01:00
ben8t
db71b56dcd Update documentation Switch core task (#6792) 2025-03-18 09:13:39 +01:00
Florian Hussonnois
8076fcc990 fix: disable micronaut otel by default 2025-03-18 08:47:19 +01:00
Miloš Paunović
bd520c2150 fix(ui): amend displaying large amount of logs (#7923)
* chore(ui): amend plugin documentation console errors

* fix(ui): amend displaying large amount of logs
2025-03-18 08:35:55 +01:00
Florian Hussonnois
2c731a0192 fix(core): avoid ClassCastException when parsing flow inputs (#7882)
Use toString() instead of casting objects directly to String
to avoid undesirable ClasCastException we expect a string type

close: #7882
2025-03-17 14:48:15 +01:00
github-actions[bot]
3b291b75cf chore(translations): localize to languages other than English (#7918) 2025-03-17 14:23:48 +01:00
Miloš Paunović
1563d1d2da feat(ui): implement a default tab setting for flows (#7917) 2025-03-17 13:44:09 +01:00
dependabot[bot]
5c73953c81 build(deps): bump org.aspectj:aspectjweaver from 1.9.22.1 to 1.9.23
Bumps [org.aspectj:aspectjweaver](https://github.com/eclipse/org.aspectj) from 1.9.22.1 to 1.9.23.
- [Release notes](https://github.com/eclipse/org.aspectj/releases)
- [Commits](https://github.com/eclipse/org.aspectj/commits)

---
updated-dependencies:
- dependency-name: org.aspectj:aspectjweaver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-17 13:16:50 +01:00
dependabot[bot]
f89aa8d278 build(deps): bump org.slf4j:slf4j-api from 2.0.16 to 2.0.17
Bumps org.slf4j:slf4j-api from 2.0.16 to 2.0.17.

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-17 13:16:41 +01:00
github-actions[bot]
7efa3975fc chore(translations): localize to languages other than English (#7915) 2025-03-17 12:54:11 +01:00
Miloš Paunović
a3362b0c85 chore(ui): properly sanitize execution errors markdown content before rendering (#7914) 2025-03-17 12:52:02 +01:00
dependabot[bot]
ab9ba91e5a build(deps): bump org.jooq:jooq from 3.19.18 to 3.20.2
Bumps org.jooq:jooq from 3.19.18 to 3.20.2.

---
updated-dependencies:
- dependency-name: org.jooq:jooq
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-17 12:04:50 +01:00
Loïc Mathieu
355e24c9da chore(core): merge outputs only when necessary 2025-03-17 11:36:59 +01:00
github-actions[bot]
78dcca64ae chore(translations): localize to languages other than English (#7909) 2025-03-17 11:28:11 +01:00
Anna Geller
57c749d812 Revert "Adjusted the CI to skip frontend and backend tests for commits/PRs co…" (#7910)
This reverts commit 71a59e7b85.
2025-03-17 11:18:15 +01:00
Ash
2d7c233a5e chore(ui): improve charts horizontal scrollbar (#7508) 2025-03-17 10:48:16 +01:00
Piyush Bhaskar
cfac2c6fef chore(ui): change display on gantt page for created & queued executions (#7706) 2025-03-17 10:31:40 +01:00
Piyush Bhaskar
06352e46dc chore(ui): use the correct shadow on charts (#7752) 2025-03-17 10:22:55 +01:00
Piyush Bhaskar
08312aef97 chore(ui): show loading skeletons on dashboard (#7417) 2025-03-17 10:19:50 +01:00
github-actions[bot]
e05554e12d chore(translations): localize to languages other than English (#7908) 2025-03-17 10:10:57 +01:00
Miloš Paunović
cbb6ca77f7 feat(ui): add copy button to kv store listing rows (#7907) 2025-03-17 09:59:11 +01:00
dependabot[bot]
3dcd3c9785 build(deps): bump nl.basjes.gitignore:gitignore-reader
Bumps [nl.basjes.gitignore:gitignore-reader](https://github.com/nielsbasjes/codeowners) from 1.11.1 to 1.11.3.
- [Release notes](https://github.com/nielsbasjes/codeowners/releases)
- [Changelog](https://github.com/nielsbasjes/codeowners/blob/main/CHANGELOG.md)
- [Commits](https://github.com/nielsbasjes/codeowners/compare/v1.11.1...v1.11.3)

---
updated-dependencies:
- dependency-name: nl.basjes.gitignore:gitignore-reader
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-17 09:52:28 +01:00
dependabot[bot]
7f6e15ec4a build(deps): bump aquasecurity/trivy-action from 0.29.0 to 0.30.0
Bumps [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action) from 0.29.0 to 0.30.0.
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](https://github.com/aquasecurity/trivy-action/compare/0.29.0...0.30.0)

---
updated-dependencies:
- dependency-name: aquasecurity/trivy-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-17 09:51:53 +01:00
dependabot[bot]
ab061e9a12 build(deps): bump dorny/test-reporter from 1 to 2
Bumps [dorny/test-reporter](https://github.com/dorny/test-reporter) from 1 to 2.
- [Release notes](https://github.com/dorny/test-reporter/releases)
- [Changelog](https://github.com/dorny/test-reporter/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dorny/test-reporter/compare/v1...v2)

---
updated-dependencies:
- dependency-name: dorny/test-reporter
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-17 09:51:41 +01:00
dependabot[bot]
8eb91b75e9 build(deps): bump com.github.oshi:oshi-core from 6.7.0 to 6.7.1
Bumps [com.github.oshi:oshi-core](https://github.com/oshi/oshi) from 6.7.0 to 6.7.1.
- [Release notes](https://github.com/oshi/oshi/releases)
- [Changelog](https://github.com/oshi/oshi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/oshi/oshi/compare/oshi-parent-6.7.0...oshi-parent-6.7.1)

---
updated-dependencies:
- dependency-name: com.github.oshi:oshi-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-17 09:51:30 +01:00
dependabot[bot]
0134d5e5c2 build(deps): bump software.amazon.awssdk.crt:aws-crt
Bumps [software.amazon.awssdk.crt:aws-crt](https://github.com/awslabs/aws-crt-java) from 0.36.1 to 0.36.2.
- [Release notes](https://github.com/awslabs/aws-crt-java/releases)
- [Commits](https://github.com/awslabs/aws-crt-java/compare/v0.36.1...v0.36.2)

---
updated-dependencies:
- dependency-name: software.amazon.awssdk.crt:aws-crt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-17 09:51:18 +01:00
dependabot[bot]
6946c92683 build(deps): bump protobufVersion from 3.25.5 to 4.30.1
Bumps `protobufVersion` from 3.25.5 to 4.30.1.

Updates `com.google.protobuf:protobuf-java` from 3.25.5 to 4.30.1
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/compare/v3.25.5...v4.30.1)

Updates `com.google.protobuf:protobuf-java-util` from 3.25.5 to 4.30.1

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-major
- dependency-name: com.google.protobuf:protobuf-java-util
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-17 09:50:36 +01:00
dependabot[bot]
149dcac5f9 build(deps): bump software.amazon.awssdk:bom from 2.30.36 to 2.31.1
Bumps software.amazon.awssdk:bom from 2.30.36 to 2.31.1.

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-17 09:50:27 +01:00
Miloš Paunović
0af5aea287 chore(ui): create flow button on welcome page to start product tour (#7906) 2025-03-17 08:40:57 +01:00
Barthélémy Ledoux
9413d4df06 fix(ui): use container queries for dashboard (#7889) 2025-03-17 08:24:39 +01:00
Miloš Paunović
87ee267f10 chore(ui): complete refactor of yaml utils (#7888) 2025-03-17 08:14:48 +01:00
Will Russell
02336ed397 fix(docs): typos 2025-03-14 19:04:38 +00:00
Will Russell
379199e186 fix(docs): fail examples 2025-03-14 18:47:57 +00:00
Will Russell
ebec8c2121 fix(docs): schedule example 2025-03-14 18:41:36 +00:00
Malay Dewangan
564c36ceab fix: remove labels from an execution (#7256)
* fix: remove labels from an execution

* feat(test): add test for removing labels from execution

* fix(test): update test for removing labels from execution

* feat: handle system labels and add unit test for system labels

---------

Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-03-14 15:19:29 +01:00
Miloš Paunović
3d248d0d74 refactor(ui): remove the obsolete console statement (#7887) 2025-03-14 15:00:35 +01:00
Loïc Mathieu
70e6d47c1a feat(webserver): add Kestra URL to the config endpoint 2025-03-14 14:57:52 +01:00
Loïc Mathieu
1ba54cd08e feat(core): add Kestra env name and URL to the expression context 2025-03-14 14:57:52 +01:00
Florian Hussonnois
eb6b40a04b feat(cli): add new --all option to plugin install cmd (#7375)
This commit adds the option all to the plugin install command
for installing all available public plugins.

Changes:
* add new service class PluginCatalogService

close: #7375
2025-03-14 14:44:30 +01:00
Anna Geller
d314c52924 docs: kv pebble (#7886) 2025-03-14 14:31:52 +01:00
Trinaya Kantevari
71a59e7b85 Adjusted the CI to skip frontend and backend tests for commits/PRs containing the 'docs-only' text. #6729 (#7756)
* Update pull-request.yml to skip tests for PRs with 'docs-only'.

Updated the if condition in the frontend and backend jobs to skip tests if 'docs-only' is present in the Pull Request title, body, or labels.

* Update workflow-test.yml to skip tests for commits with 'docs-only'.

Updated the if condition in the frontend and backend jobs to skip tests if 'docs-only' is present in the commit message.

* Apply suggestions from code review

---------

Co-authored-by: Anna Geller <anna.m.geller@gmail.com>
2025-03-14 14:29:26 +01:00
Will Russell
ff504afd8f fix(docs): add parallel example 2025-03-14 12:37:00 +00:00
YannC
64e5b80049 fix(ui): display back core property if not in a task
close #7879
2025-03-14 11:44:22 +01:00
Loïc Mathieu
c9baaf8565 fix(core): Avoid calling Worker post construct method twice 2025-03-14 09:58:30 +01:00
Miloš Paunović
02ad2af305 chore(ui): re-organize visuals for empty state component (#7875) 2025-03-14 08:46:57 +01:00
Florian Hussonnois
bed11f1541 chore(core): add PluginVersioning interface and version for TaskRunner
Changes:
- add new interface PluginVersioning
- update interfaces TaskInterface, TriggerInterface, TaskRunner

close: kestra-io/kestra-ee#3122
2025-03-14 08:26:52 +01:00
Miloš Paunović
6d14773ea7 chore(ui): prevent node state history to be shown if there is no date present (#7867) 2025-03-13 16:26:25 +01:00
Loïc Mathieu
74577cb059 chore(deps): upgrade @kestra-io/ui-libs to 0.0.157 2025-03-13 15:50:45 +01:00
Miloš Paunović
e63b019857 chore(ui): re-organize visuals for empty state component (#7866) 2025-03-13 15:18:29 +01:00
github-actions[bot]
8684bd9481 chore(translations): localize to languages other than English (#7863) 2025-03-13 14:36:31 +01:00
Miloš Paunović
586e089ada feat(ui): add afterExecution block to no code editor (#7848) 2025-03-13 13:53:52 +01:00
Loïc Mathieu
acebfef0d1 fix(deps): move OTLP metrics lib to CLI to avoid warning in tests 2025-03-13 13:05:20 +01:00
Loïc Mathieu
f7019af9d5 feat(core): add afterExecution to the topology 2025-03-13 13:02:34 +01:00
Loïc Mathieu
1419680006 fix(core): missing afterExecution in FlowForExecution 2025-03-13 13:02:34 +01:00
Shruti Mantri
64290f5d17 feat: add finally to the flow assets in basic.md (#7857) 2025-03-13 10:27:43 +01:00
Loïc Mathieu
e2da2dfeb0 fix(core): allowedNamespace is already called in the KvStoreService 2025-03-12 16:37:16 +01:00
Will Russell
977fe222a4 fix(docs): 2025 get started video 2025-03-12 13:11:18 +00:00
ben8t
0a24a5d13f Remove pebble function references that don't exist (#7698)
* [fix] remove pebble function references that don't exist

* [fix] pebble documentation

* [fix] remove non-existing functions
2025-03-12 12:23:22 +01:00
Loïc Mathieu
ecfe925ece feat(core): redact KESTRA_JAVA_OPTS from the env available to executions 2025-03-12 11:31:53 +01:00
dependabot[bot]
76f6c93e48 build(deps): bump @babel/runtime-corejs3 from 7.26.9 to 7.26.10 in /ui (#7840)
Bumps [@babel/runtime-corejs3](https://github.com/babel/babel/tree/HEAD/packages/babel-runtime-corejs3) from 7.26.9 to 7.26.10.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.26.10/packages/babel-runtime-corejs3)

---
updated-dependencies:
- dependency-name: "@babel/runtime-corejs3"
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-12 08:43:56 +01:00
dependabot[bot]
cfb76f4279 build(deps): bump @babel/helpers from 7.26.9 to 7.26.10 in /ui (#7841)
Bumps [@babel/helpers](https://github.com/babel/babel/tree/HEAD/packages/babel-helpers) from 7.26.9 to 7.26.10.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.26.10/packages/babel-helpers)

---
updated-dependencies:
- dependency-name: "@babel/helpers"
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-12 08:43:18 +01:00
Miloš Paunović
50bbf03d53 chore(deps): regular dependency update (#7839) 2025-03-12 08:29:38 +01:00
Loïc Mathieu
2c9f09a82e feat(build): configure heap size to max 50% of available memory (#7800)
* feat(build): configure heap size to max 50% of available memory

* Apply suggestions from code review

Co-authored-by: brian-mulier-p <bmmulier@hotmail.fr>

---------

Co-authored-by: brian-mulier-p <bmmulier@hotmail.fr>
2025-03-11 16:38:51 +01:00
github-actions[bot]
c9579e5dcd chore(translations): localize to languages other than English (#7803)
Co-authored-by: GitHub Action <actions@github.com>
2025-03-11 16:34:42 +01:00
Miloš Paunović
1b13559a61 chore(ui): add the ability to change word wrap and to copy the preview output content (#7801) 2025-03-11 15:12:51 +01:00
Loïc Mathieu
c2e62d653d feat(core,jdbc): switch back to a cached thread pool for async JDBC queue
But with less threads as 2xnbCore was too much.
2025-03-11 14:25:32 +01:00
Bart Ledoux
14ff4438fe chore: update ui-libs 2025-03-11 14:06:59 +01:00
Loïc Mathieu
39b8fc1039 feat(core): afterExecution tasks
Fixes https://github.com/kestra-io/kestra-ee/issues/2792
2025-03-11 13:35:08 +01:00
Florian Hussonnois
1cb323b7aa fix(core): fix properly map MavenPluginRepositoryConfig 2025-03-11 13:17:23 +01:00
brian.mulier
4a12827684 fix(ui): remove errors from dashboard validation if it's fixed
closes #7748
2025-03-11 12:26:50 +01:00
brian.mulier
82a346b2ce fix(ui): make switch view buttons from dashboard editor the same as flow editor ones 2025-03-11 12:23:18 +01:00
dependabot[bot]
dea392a941 build(deps): bump io.micrometer:micrometer-core from 1.14.3 to 1.14.5
Bumps [io.micrometer:micrometer-core](https://github.com/micrometer-metrics/micrometer) from 1.14.3 to 1.14.5.
- [Release notes](https://github.com/micrometer-metrics/micrometer/releases)
- [Commits](https://github.com/micrometer-metrics/micrometer/compare/v1.14.3...v1.14.5)

---
updated-dependencies:
- dependency-name: io.micrometer:micrometer-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-11 11:22:43 +01:00
Loïc Mathieu
d0af5767d5 feat(core): add execution state in Pebbe context
Fixes #7155
2025-03-11 11:21:50 +01:00
brian.mulier
a064c7a956 feat(ui): introduce kv() and secret() pebble autocompletions
closes kestra-io/kestra-ee#77
2025-03-11 00:10:29 +01:00
brian.mulier
9b5b2b981f fix(webserver): add endpoint for inherited secrets 2025-03-10 23:42:33 +01:00
YannC
fac2ae813c fix(jdbc): return correct total when paginating custom dashboard chart (#7790)
close #7164
2025-03-10 18:05:44 +01:00
Nicolas K.
233a871c58 feat(core-ee): add json format to file log exporter (#7789)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-10 17:47:51 +01:00
Nicolas K.
05f24f3d65 fix(core): #7740 http configuration bearer token may change to basic because of allowFailed (#7788)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-10 16:33:15 +01:00
dependabot[bot]
66e5a7ca31 build(deps): bump software.amazon.awssdk:bom from 2.30.31 to 2.30.36
Bumps software.amazon.awssdk:bom from 2.30.31 to 2.30.36.

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 15:48:03 +01:00
dependabot[bot]
fefaa7cdbb build(deps): bump org.testcontainers:testcontainers
Bumps [org.testcontainers:testcontainers](https://github.com/testcontainers/testcontainers-java) from 1.20.5 to 1.20.6.
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.20.5...1.20.6)

---
updated-dependencies:
- dependency-name: org.testcontainers:testcontainers
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 15:47:46 +01:00
YannC
fd6b9dc065 fix(tests): increase seconds diff between dates (#7785) 2025-03-10 15:08:20 +01:00
dependabot[bot]
5935308e43 build(deps): bump software.amazon.awssdk.crt:aws-crt
Bumps [software.amazon.awssdk.crt:aws-crt](https://github.com/awslabs/aws-crt-java) from 0.34.0 to 0.36.1.
- [Release notes](https://github.com/awslabs/aws-crt-java/releases)
- [Commits](https://github.com/awslabs/aws-crt-java/compare/v0.34.0...v0.36.1)

---
updated-dependencies:
- dependency-name: software.amazon.awssdk.crt:aws-crt
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 15:02:29 +01:00
dependabot[bot]
83f06f3374 build(deps): bump org.jsoup:jsoup from 1.18.3 to 1.19.1
Bumps [org.jsoup:jsoup](https://github.com/jhy/jsoup) from 1.18.3 to 1.19.1.
- [Release notes](https://github.com/jhy/jsoup/releases)
- [Changelog](https://github.com/jhy/jsoup/blob/master/CHANGES.md)
- [Commits](https://github.com/jhy/jsoup/compare/jsoup-1.18.3...jsoup-1.19.1)

---
updated-dependencies:
- dependency-name: org.jsoup:jsoup
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 15:02:19 +01:00
dependabot[bot]
8d7bc6fdd4 build(deps): bump com.github.docker-java:docker-java from 3.4.1 to 3.4.2
Bumps [com.github.docker-java:docker-java](https://github.com/docker-java/docker-java) from 3.4.1 to 3.4.2.
- [Release notes](https://github.com/docker-java/docker-java/releases)
- [Changelog](https://github.com/docker-java/docker-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/docker-java/docker-java/compare/3.4.1...3.4.2)

---
updated-dependencies:
- dependency-name: com.github.docker-java:docker-java
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 15:01:44 +01:00
dependabot[bot]
03a44c1039 build(deps): bump org.opensearch.client:opensearch-java
Bumps [org.opensearch.client:opensearch-java](https://github.com/opensearch-project/opensearch-java) from 2.21.0 to 2.22.0.
- [Release notes](https://github.com/opensearch-project/opensearch-java/releases)
- [Changelog](https://github.com/opensearch-project/opensearch-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/opensearch-project/opensearch-java/compare/v2.21.0...v2.22.0)

---
updated-dependencies:
- dependency-name: org.opensearch.client:opensearch-java
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 15:00:59 +01:00
dependabot[bot]
5af0858445 build(deps): bump org.testcontainers:junit-jupiter from 1.20.5 to 1.20.6
Bumps [org.testcontainers:junit-jupiter](https://github.com/testcontainers/testcontainers-java) from 1.20.5 to 1.20.6.
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.20.5...1.20.6)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 15:00:31 +01:00
dependabot[bot]
0a304ff1d3 build(deps): bump com.gorylenko.gradle-git-properties
Bumps com.gorylenko.gradle-git-properties from 2.4.2 to 2.5.0.

---
updated-dependencies:
- dependency-name: com.gorylenko.gradle-git-properties
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 15:00:13 +01:00
dependabot[bot]
92418841fc build(deps): bump com.azure:azure-sdk-bom from 1.2.31 to 1.2.32
Bumps [com.azure:azure-sdk-bom](https://github.com/azure/azure-sdk-for-java) from 1.2.31 to 1.2.32.
- [Release notes](https://github.com/azure/azure-sdk-for-java/releases)
- [Commits](https://github.com/azure/azure-sdk-for-java/compare/azure-sdk-bom_1.2.31...azure-sdk-bom_1.2.32)

---
updated-dependencies:
- dependency-name: com.azure:azure-sdk-bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 14:59:56 +01:00
Florian Hussonnois
01036c829d fix(core): wait for service-manager-task thread to be stopped 2025-03-10 14:22:38 +01:00
Loïc Mathieu
5ffeee532c fix(jdbc): flaky tests JdbcServiceLivenessCoordinatorTest 2025-03-10 12:38:56 +01:00
Loïc Mathieu
c606760522 chore(core): disable OTLP metrics on tests 2025-03-10 12:38:56 +01:00
Loïc Mathieu
0870d8ebd8 fix(core): flaky test ExitTest.shouldExitAndKillTheExecution() 2025-03-10 12:38:56 +01:00
Florian Hussonnois
d2d0726f73 fix(cli): fix double shutdown warn messages
Changes:
- Avoid registering two shutdown-hooks that will both log the
shutdown message - leading to some confusion
- Some code cleanup/refactoring
2025-03-10 12:31:06 +01:00
Nicolas K.
868a232527 fix(core): failing schedule test (#7783)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-10 12:03:11 +01:00
Loïc Mathieu
3ef11044a2 feat(core): custom log filter
Since Logback 1.5.13, there is no more default implementation for the EvaluatorFilter so we need to supply our own.
See https://logback.qos.ch/manual/filters.html#evaluatorFilter
2025-03-10 10:56:13 +01:00
Miloš Paunović
2d1582f761 chore(ui): respect line numbers prop as part of editor options (#7781) 2025-03-10 10:34:15 +01:00
github-actions[bot]
94d36fdc01 chore(translations): localize to languages other than English (#7780) 2025-03-10 10:13:49 +01:00
dependabot[bot]
0388909828 build(deps): bump robinraju/release-downloader from 1.11 to 1.12
Bumps [robinraju/release-downloader](https://github.com/robinraju/release-downloader) from 1.11 to 1.12.
- [Release notes](https://github.com/robinraju/release-downloader/releases)
- [Commits](https://github.com/robinraju/release-downloader/compare/v1.11...v1.12)

---
updated-dependencies:
- dependency-name: robinraju/release-downloader
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-10 09:55:11 +01:00
Miloš Paunović
ff4f43c39b chore(ui): re-order namespace tabs (#7778) 2025-03-10 09:38:51 +01:00
Miloš Paunović
140d058beb chore(ui): remove single empty space between label key and value so it can be copied (#7774) 2025-03-10 08:08:22 +01:00
Loïc Mathieu
4276a0afd9 Revert "fix(core): ThresholdFilter is now stricly lower"
This reverts commit bd82f5e3b6.
2025-03-07 18:12:33 +01:00
Loïc Mathieu
1692cab533 feat(core): add outputs to the Flow trigger
Fixes #6848
2025-03-07 18:10:00 +01:00
Nicolas K.
3013e9dfd5 feat(core): #5467 add namespaces in the namespaceFiles parameter (#7749)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-07 17:18:45 +01:00
brian.mulier
440a942900 fix(ci): generate_translations.py is now deleting keys that are no longer in en translation to avoid ghost translations 2025-03-07 16:49:45 +01:00
Loïc Mathieu
7696d41d5f fix(jdbc): resubmit worker job to the good worker group
Fixes #7730
2025-03-07 16:44:28 +01:00
github-actions[bot]
45abaa146e chore(translations): localize to languages other than English (#7746)
Co-authored-by: GitHub Action <actions@github.com>
2025-03-07 14:57:33 +01:00
Loïc Mathieu
e9f2711cd3 feat(core): handle worker group fallback inside the scheduler 2025-03-07 14:48:02 +01:00
brian.mulier
166262209a fix(tests): reject promise with 404 instead of empty resolve if non-mocked store call in flowAutoCompletionProvider.spec.ts 2025-03-07 14:01:37 +01:00
Miloš Paunović
99bad6abb0 chore(ui): use uniformed pagination component for custom dashboard tables (#7744) 2025-03-07 13:39:40 +01:00
Barthélémy Ledoux
acc10ed638 fix(flow editor): enhance behavior when switching file tabs (#7722)
* fix(flow editor): when opening a new file and coming back to flow, validation + lowcode are broken

* fix: sync flowYaml only onEdit

* fix: files can now be saved again

* fix: make editor saving button ready for better ux

* allow for a bit of time for storybook to launch the inputs story

* add storybook for flow editor

* setup storybook
2025-03-07 11:29:54 +01:00
Loïc Mathieu
88341bb5c9 feat(webserver): mask the secret() function result from eval outputs
Fixes https://github.com/kestra-io/kestra-ee/issues/3073
2025-03-07 11:26:28 +01:00
brian-mulier-p
e8411b6b11 refacto(ui): move autocompletion to its own service (#7710)
Co-authored-by: Bart Ledoux <bledoux@kestra.io>
2025-03-07 10:46:44 +01:00
github-actions[bot]
6af105a8bf chore(translations): localize to languages other than English (#7739)
Co-authored-by: GitHub Action <actions@github.com>
2025-03-07 10:14:51 +01:00
Miloš Paunović
9a2e09cc8c chore(ui): uniforming empty state for components (#7737) 2025-03-07 09:11:16 +01:00
Nicolas K.
9b1a9d64bc feat(core): #7721 add namespace to pebble file functions (#7729)
* feat(core): #7721 add namespace to pebble file functions

* clean(core): #7721 extract tenantId only once

---------

Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-06 18:05:19 +01:00
Bart Ledoux
4678616520 fix: repair collapsed menu submenus 2025-03-06 17:12:24 +01:00
Bart Ledoux
07e4598fa4 fix(demo): on pages stop showing the docs without a button
closes #7726
2025-03-06 16:54:52 +01:00
YannC
01fe48b47a fix(ui): improved fetch of type for pluginDoc and avoid removing doc if map has "type" as property but without doc (like ENUM value) (#7727)
close #6101
close #7708
2025-03-06 16:47:04 +01:00
YannC
f7e61a46df feat(core): add new crudEventType value 2025-03-06 15:49:45 +01:00
Loïc Mathieu
dfe5552a1e feat(core): allow reading file from any namespaces
Fixes https://github.com/kestra-io/kestra-ee/issues/1934
2025-03-06 15:36:24 +01:00
YannC
593558dd22 fix(runner-memory): delete MemorySchedulerTriggerState back due to cherry-pick 2025-03-06 15:28:29 +01:00
Miloš Paunović
b22764290e chore(ui): properly sanitize markdown content before rendering (#7724) 2025-03-06 15:13:10 +01:00
YannC
78bfbf0d5e fix(): align to EE 2025-03-06 14:13:06 +01:00
ben8t
f505f29360 Add core plugin sanitychecks (#7592)
* [feature] add sanitychecks

* [feature] add dag sanitychecks

* [fix] use assert

* [fix] use assert

* [feature] add for each sanity
2025-03-06 13:43:54 +01:00
ben8t
1726347dbf change email oss auth default (#7717) 2025-03-06 10:47:24 +01:00
Miloš Paunović
bbf232ba52 fix(ui): prevent context docs open on editor custom blueprints click (#7716) 2025-03-06 09:53:54 +01:00
yuri1969
8f9fc5fe4b fix(core): stop Docker runner gracefully 2025-03-06 09:34:11 +01:00
Ludovic DEHON
9a56b763f4 chore(core): make registry unregister usable with immutable list 2025-03-06 08:48:36 +01:00
Miloš Paunović
40d37d9e42 chore(ui): auto expand first element in execution overview cascaders (#7715) 2025-03-06 08:48:22 +01:00
Florian Hussonnois
e200bbdb6b fix(cli): fix regression on CLI plugin doc cmd 2025-03-05 21:24:09 +01:00
brian.mulier
ef65623b13 fix(core): avoid duplicates in plugins subgroups + properly retrieve subgroup title 2025-03-05 18:36:45 +01:00
Nicolas K.
d8b2e92e8d feat(core-ee): #2838 add audit log shipper (#7701)
* feat(core-ee): #2838 add audit log shipper

* fix: scheduler flacky tests

* feat(core-ee): add pretty string audit log and clean PR

---------

Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-05 16:30:11 +01:00
Loïc Mathieu
2c63112a59 feat(core, jdbc): dynamic worker group key
Fixes https://github.com/kestra-io/kestra-ee/issues/3064
2025-03-05 15:37:01 +01:00
dependabot[bot]
20b87f1c9c build(deps): bump jacksonVersion from 2.18.2 to 2.18.3
Bumps `jacksonVersion` from 2.18.2 to 2.18.3.

Updates `com.fasterxml.jackson:jackson-bom` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson-bom/compare/jackson-bom-2.18.2...jackson-bom-2.18.3)

Updates `com.fasterxml.jackson.core:jackson-core` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson-core/compare/jackson-core-2.18.2...jackson-core-2.18.3)

Updates `com.fasterxml.jackson.core:jackson-databind` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson/commits)

Updates `com.fasterxml.jackson.core:jackson-annotations` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson/commits)

Updates `com.fasterxml.jackson.module:jackson-module-parameter-names` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson-modules-java8/compare/jackson-modules-java8-2.18.2...jackson-modules-java8-2.18.3)

Updates `com.fasterxml.jackson.dataformat:jackson-dataformat-yaml` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson-dataformats-text/compare/jackson-dataformats-text-2.18.2...jackson-dataformats-text-2.18.3)

Updates `com.fasterxml.jackson.dataformat:jackson-dataformat-smile` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson-dataformats-binary/compare/jackson-dataformats-binary-2.18.2...jackson-dataformats-binary-2.18.3)

Updates `com.fasterxml.jackson.dataformat:jackson-dataformat-cbor` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson-dataformats-binary/compare/jackson-dataformats-binary-2.18.2...jackson-dataformats-binary-2.18.3)

Updates `com.fasterxml.jackson.dataformat:jackson-dataformat-ion` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson-dataformat-ion/commits)

Updates `com.fasterxml.jackson.dataformat:jackson-dataformat-xml` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson-dataformat-xml/compare/jackson-dataformat-xml-2.18.2...jackson-dataformat-xml-2.18.3)

Updates `com.fasterxml.jackson.datatype:jackson-datatype-guava` from 2.18.2 to 2.18.3
- [Commits](https://github.com/FasterXML/jackson-datatypes-collections/compare/jackson-datatypes-collections-2.18.2...jackson-datatypes-collections-2.18.3)

Updates `com.fasterxml.jackson.datatype:jackson-datatype-jsr310` from 2.18.2 to 2.18.3

Updates `com.fasterxml.jackson.datatype:jackson-datatype-jdk8` from 2.18.2 to 2.18.3

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson:jackson-bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.core:jackson-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.core:jackson-databind
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.core:jackson-annotations
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.module:jackson-module-parameter-names
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-yaml
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-smile
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-cbor
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-ion
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-xml
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.datatype:jackson-datatype-guava
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.datatype:jackson-datatype-jsr310
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.fasterxml.jackson.datatype:jackson-datatype-jdk8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-05 15:36:23 +01:00
dependabot[bot]
ca4e6a4b33 build(deps): bump org.opensearch.client:opensearch-java
Bumps [org.opensearch.client:opensearch-java](https://github.com/opensearch-project/opensearch-java) from 2.20.0 to 2.21.0.
- [Release notes](https://github.com/opensearch-project/opensearch-java/releases)
- [Changelog](https://github.com/opensearch-project/opensearch-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/opensearch-project/opensearch-java/compare/v2.20.0...v2.21.0)

---
updated-dependencies:
- dependency-name: org.opensearch.client:opensearch-java
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-05 13:11:00 +01:00
dependabot[bot]
82df58d26f build(deps): bump com.google.cloud:libraries-bom from 26.54.0 to 26.56.0
Bumps [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) from 26.54.0 to 26.56.0.
- [Release notes](https://github.com/googleapis/java-cloud-bom/releases)
- [Changelog](https://github.com/googleapis/java-cloud-bom/blob/main/release-please-config.json)
- [Commits](https://github.com/googleapis/java-cloud-bom/compare/v26.54.0...v26.56.0)

---
updated-dependencies:
- dependency-name: com.google.cloud:libraries-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-05 12:28:30 +01:00
dependabot[bot]
b6f91128a1 build(deps): bump opensearchRestVersion from 2.18.0 to 2.19.1
Bumps `opensearchRestVersion` from 2.18.0 to 2.19.1.

Updates `org.opensearch.client:opensearch-rest-client` from 2.18.0 to 2.19.1
- [Release notes](https://github.com/opensearch-project/OpenSearch/releases)
- [Changelog](https://github.com/opensearch-project/OpenSearch/blob/main/CHANGELOG-3.0.md)
- [Commits](https://github.com/opensearch-project/OpenSearch/compare/2.18.0...2.19.1)

Updates `org.opensearch.client:opensearch-rest-high-level-client` from 2.18.0 to 2.19.1
- [Release notes](https://github.com/opensearch-project/OpenSearch/releases)
- [Changelog](https://github.com/opensearch-project/OpenSearch/blob/main/CHANGELOG-3.0.md)
- [Commits](https://github.com/opensearch-project/OpenSearch/compare/2.18.0...2.19.1)

---
updated-dependencies:
- dependency-name: org.opensearch.client:opensearch-rest-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.opensearch.client:opensearch-rest-high-level-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-05 12:15:10 +01:00
Miloš Paunović
a7abc17c0b chore(deps): regular dependency update (#7705) 2025-03-05 12:01:18 +01:00
Miloš Paunović
9607d0152e chore(ui): properly sanitize markdown content before rendering (#7697) 2025-03-05 08:17:02 +01:00
AJ Emerich
fc3c1a4c54 docs: fix typo in storage reverse task title (#7667) 2025-03-04 18:31:56 +01:00
Miloš Paunović
4cee4fca61 chore(ui): make sure chart stacks are following the same order every time (#7664) 2025-03-04 15:22:14 +01:00
Miloš Paunović
475573d13e feat(ui): add markdown formatting to flow run dialog (#7663) 2025-03-04 14:34:15 +01:00
Miloš Paunović
cfac9f339f chore(ui): properly sanitize markdown content before rendering (#7662) 2025-03-04 14:02:07 +01:00
Loïc Mathieu
cd97705d87 feat(core, jdbc): DashboardRepository.findAll() 2025-03-04 11:03:31 +01:00
Nicolas K.
56406c6b5c feat(#7636): add default options for HttpClient (#7650)
* feat(#7636): add default options for HttpClient

* fix(webserver): flacky execution controller tests

* fix(webserver): flacky execution controller tests

* fix(webserver): flacky execution controller tests

* fix(webserver): rework executor controller kill test

* fix(webserver): rework executor controller kill test

* fix(webserver): rework executor controller kill test

* fix(webserver): rework executor controller kill test

* fix(webserver): rework executor controller kill test

* fix(webserver): rework executor controller kill test

---------

Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-03-04 10:02:10 +01:00
YannC
38921a265a chore(deps): upgrade ui-libs version to v0.0.154 2025-03-03 22:11:53 +01:00
Shruti Mantri
be61869642 feat: add example for expression condition (#7660) 2025-03-03 18:22:49 +01:00
Shruti Mantri
1f09afe564 feat: add examples for flow conditions (#7659) 2025-03-03 17:52:56 +01:00
Shruti Mantri
35a0520dba feat: add examples for conditions - 3 (#7657)
Co-authored-by: AJ Emerich <aemerich@kestra.io>
2025-03-03 17:52:20 +01:00
YannC
cf6fad0896 fix(): avoid crash on injectDefault 2025-03-03 17:42:21 +01:00
Miloš Paunović
af0c1134e1 chore(ui): improve the topology tooltip label for adding task button (#7656) 2025-03-03 16:14:40 +01:00
Loïc Mathieu
12180457ea fix(core): MultipleCondition documentation 2025-03-03 15:14:55 +01:00
Loïc Mathieu
1eb9adf30a fix(core): validation error when timeWindow.type is null 2025-03-03 15:14:55 +01:00
dependabot[bot]
e7f2ec2aee build(deps): bump software.amazon.awssdk:bom from 2.30.16 to 2.30.31
Bumps software.amazon.awssdk:bom from 2.30.16 to 2.30.31.

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-03 15:06:43 +01:00
dependabot[bot]
94421f141e build(deps): bump org.owasp.dependencycheck from 12.0.2 to 12.1.0
Bumps org.owasp.dependencycheck from 12.0.2 to 12.1.0.

---
updated-dependencies:
- dependency-name: org.owasp.dependencycheck
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-03 14:36:41 +01:00
dependabot[bot]
65b071adfd chore(deps): bump org.testcontainers:testcontainers
Bumps [org.testcontainers:testcontainers](https://github.com/testcontainers/testcontainers-java) from 1.20.4 to 1.20.5.
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.20.4...1.20.5)

---
updated-dependencies:
- dependency-name: org.testcontainers:testcontainers
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-03 14:31:35 +01:00
Ludovic DEHON
d2bf56fecf fix(cicd): missing acls for test reports 2025-03-03 14:17:45 +01:00
dependabot[bot]
998a5cca32 chore(deps): bump flyingSaucerVersion from 9.11.3 to 9.11.4
Bumps `flyingSaucerVersion` from 9.11.3 to 9.11.4.

Updates `org.xhtmlrenderer:flying-saucer-core` from 9.11.3 to 9.11.4
- [Release notes](https://github.com/flyingsaucerproject/flyingsaucer/releases)
- [Changelog](https://github.com/flyingsaucerproject/flyingsaucer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flyingsaucerproject/flyingsaucer/compare/v9.11.3...v9.11.4)

Updates `org.xhtmlrenderer:flying-saucer-pdf` from 9.11.3 to 9.11.4
- [Release notes](https://github.com/flyingsaucerproject/flyingsaucer/releases)
- [Changelog](https://github.com/flyingsaucerproject/flyingsaucer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flyingsaucerproject/flyingsaucer/compare/v9.11.3...v9.11.4)

---
updated-dependencies:
- dependency-name: org.xhtmlrenderer:flying-saucer-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.xhtmlrenderer:flying-saucer-pdf
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-03 14:15:30 +01:00
dependabot[bot]
4b48ad597d build(deps): bump nl.basjes.gitignore:gitignore-reader
Bumps [nl.basjes.gitignore:gitignore-reader](https://github.com/nielsbasjes/codeowners) from 1.10.0 to 1.11.1.
- [Release notes](https://github.com/nielsbasjes/codeowners/releases)
- [Changelog](https://github.com/nielsbasjes/codeowners/blob/main/CHANGELOG.md)
- [Commits](https://github.com/nielsbasjes/codeowners/compare/v1.10.0...v1.11.1)

---
updated-dependencies:
- dependency-name: nl.basjes.gitignore:gitignore-reader
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-03 14:14:41 +01:00
dependabot[bot]
66fdb58f4b build(deps): bump com.github.oshi:oshi-core from 6.6.6 to 6.7.0
Bumps [com.github.oshi:oshi-core](https://github.com/oshi/oshi) from 6.6.6 to 6.7.0.
- [Release notes](https://github.com/oshi/oshi/releases)
- [Changelog](https://github.com/oshi/oshi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/oshi/oshi/compare/oshi-parent-6.6.6...oshi-parent-6.7.0)

---
updated-dependencies:
- dependency-name: com.github.oshi:oshi-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-03 14:14:27 +01:00
Loïc Mathieu
8c708e2d53 feat(core): improve merging outputs by merging them at the task level 2025-03-03 13:57:48 +01:00
Loïc Mathieu
677585213a feat(core): use HashMap.newHashMap(int) 2025-03-03 13:57:48 +01:00
Loïc Mathieu
ff3f90465d feat(core): replace new ArrayList by Collections.emptyList() 2025-03-03 13:57:48 +01:00
YannC
8f5189df49 fix(): recoverMissedSchedules behavior on long running executions (#7617)
* fix(): handle missed scheduled for long running execution

close #7549

* tests(core): add test for backfill last/none on long running execution

* fix(): review pr

* fix(): shorter test

* fix(): fix tests and add missing getStopAfter

* tests(): fix scheduler tests
2025-03-03 13:14:16 +01:00
Miloš Paunović
9446eefd94 fix(ui): make sure adding labels on flow run dialog is possible (#7652) 2025-03-03 11:29:35 +01:00
YannC
334410ab58 fix(): make subflow not found a warning instead of an error (#7649) 2025-03-03 10:38:28 +01:00
nKwiatkowski
7ccb97a963 Revert "feat(#7636): add default options for HttpClient"
This reverts commit 5b42d0adba.
2025-03-03 10:09:08 +01:00
nKwiatkowski
3957177455 Merge remote-tracking branch 'origin/develop' into develop 2025-03-03 10:05:13 +01:00
nKwiatkowski
5b42d0adba feat(#7636): add default options for HttpClient 2025-03-03 10:05:01 +01:00
Loïc Mathieu
8bd3c2fef8 fix(ci): fix vulnerability check workflow 2025-03-03 09:41:13 +01:00
Miloš Paunović
d8fc4a9ce2 fix(ui): allow sidebar theme toggle to update the editor theme (#7648) 2025-03-03 09:15:24 +01:00
Florian Hussonnois
bfd82e0b5c chore(core): eval value property once for flowable task Switch 2025-02-28 18:38:31 +01:00
Loïc Mathieu
fa07cbd3b9 feat(core): improve performance of ExecutorService.handleChildWorkerTaskResult
Searching for a retry in all parents is a costly operation, doing it only wgen we are retrying or failing avoid it most of the time.
2025-02-28 17:40:29 +01:00
github-actions[bot]
2c77a43935 chore(translations): localize to languages other than English (#7635)
Co-authored-by: GitHub Action <actions@github.com>
2025-02-28 16:09:24 +01:00
Piyush Bhaskar
60a84a49f8 chore(ui): improve empty state of the namespace files editor (#7495) 2025-02-28 15:54:39 +01:00
Piyush Bhaskar
d8295ef4ae feat(ui): allow reordering tabs in the editor (#7531) 2025-02-28 15:46:30 +01:00
Miloš Paunović
b02325a2e4 chore(ui): remove crud details from execution overview (#7634) 2025-02-28 15:39:43 +01:00
Florian Hussonnois
a098847c65 feat(core): enhance plugin management
Changes:
* add new interface PluginManager
* add new CLI for un-installing plugins
* add new option --locally to CLI plugin install
* refactor service for downloading plugins
* refactor PluginController
* move Version util class from EE to OSS
* migrate aether lib to maven-resolver (#915)

part-of: #915
2025-02-28 15:04:40 +01:00
brian.mulier
5f21eb5790 fix(ui): use watch with ref instead of accessing the value 2025-02-28 14:16:05 +01:00
MilosPaunovic
036a7cf4f7 fix(ui): improve check for text label of filters section 2025-02-28 14:00:32 +01:00
MilosPaunovic
446a034d6b fix(ui): additional check for text label of filters section 2025-02-28 12:48:38 +01:00
github-actions[bot]
3fff36881a chore(translations): localize to languages other than English (#7633)
Co-authored-by: GitHub Action <actions@github.com>
2025-02-28 12:23:36 +01:00
Anna Geller
174ac280d2 Update README.md 2025-02-28 11:12:41 +01:00
Miloš Paunović
c610ccdaee chore(ui): improve label for text search in filters section (#7631)
* chore(ui): improve label for text search in filters section

* chore(ci): prevent running translation spec on ci:cd workflow
2025-02-28 11:07:19 +01:00
Hashim Khalifa
d8ed5b8b88 feat:string filter (#7428)
* feat:string filter

* chore: include string filter in docs
2025-02-28 10:38:32 +01:00
brian.mulier
17e54134ce fix(core): no longer lowercasing PluginClassIdentifier to have proper validation upon Plugin deserialization
closes #7387
2025-02-28 10:01:57 +01:00
Anna Geller
cacbd069f9 Update README.md 2025-02-28 01:07:18 +01:00
Anna Geller
de5a0f4623 Update README.md 2025-02-28 01:05:24 +01:00
Piyush Bhaskar
0f6c8d3521 fix(ui): executions naviation based on start Date. (#7626) 2025-02-27 21:21:21 +01:00
Anna Geller
c8207b8706 fix: confusing trace log 2025-02-27 20:33:26 +01:00
brian.mulier
39cd6189ac chore(deps): bump ui-libs to 0.0.151 2025-02-27 18:47:43 +01:00
brian.mulier
9b4f3148fc fix(ui): replace alert blocks upon markdown rendering to display them properly
closes #7393
2025-02-27 16:46:23 +01:00
Shruti Mantri
ca6a7a9e16 Add full examples for dayweek, dayweekinmonth and publicholidays conditions (#7615)
* Add full examples for dayweek, dayweekinmonth and publicholidays conditions

* Update core/src/main/java/io/kestra/plugin/core/condition/DayWeekInMonth.java

---------

Co-authored-by: AJ Emerich <aemerich@kestra.io>
2025-02-27 16:39:41 +01:00
Bart Ledoux
3dc8e98ed3 fix: make sure nocode edits the right task from the topology
closes #7619
2025-02-27 15:58:17 +01:00
github-actions[bot]
ea79be9de8 chore(translations): localize to languages other than English (#7618)
Co-authored-by: GitHub Action <actions@github.com>
2025-02-27 15:17:13 +01:00
Anna Geller
3804bdc7f7 fix: improve translation prompt 2025-02-27 15:14:02 +01:00
Anna Geller
d04764814a fix: turn CI flag to a dropdown 2025-02-27 14:41:31 +01:00
Anna Geller
020d674d8c fix(translations): allow retranslating modified keys when needed 2025-02-27 14:39:17 +01:00
Miloš Paunović
ec7458dce5 feat(ui): make filter dropdown be positioned below the input caret (#7614) 2025-02-27 14:24:53 +01:00
Anna Geller
ae75ea06d2 fix: improve some wording 2025-02-27 14:19:30 +01:00
github-actions[bot]
53cd056871 chore(translations): localize to languages other than English (#7613)
Co-authored-by: GitHub Action <actions@github.com>
2025-02-27 14:14:52 +01:00
Piyush Bhaskar
9bab2d2cfa feat(ui): introduce the execution timeline section on overview tab (#7498)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-02-27 14:09:48 +01:00
Loïc Mathieu
4d87d95eac chore(deps): upgrade Micronaut core to 4.7.15 2025-02-27 14:09:22 +01:00
Anna Geller
81635ddc84 fix: clean up translations script 2025-02-27 13:11:46 +01:00
github-actions[bot]
781d1338e9 chore(translations): localize to languages other than English (#7610)
Co-authored-by: GitHub Action <actions@github.com>
2025-02-27 12:47:51 +01:00
Anna Geller
37ba85a130 Revert "chore(translations): localize to languages other than English (#7605)" (#7609)
This reverts commit 04b8a0a14c.
2025-02-27 12:41:42 +01:00
Anna Geller
7c7ac5a70d fix: only re-translate if the key is not already in the target dict (#7608)
* fix: only re-translate if the key is not already in the target dict or is empty

* Update generate_translations.py

* Update generate_translations.py
2025-02-27 12:40:46 +01:00
Shruti Mantri
f315485fc6 feat: add full examples for datetimebetween condition (#7598) 2025-02-27 11:30:59 +00:00
github-actions[bot]
04b8a0a14c chore(translations): localize to languages other than English (#7605)
Co-authored-by: GitHub Action <actions@github.com>
2025-02-27 11:53:09 +01:00
Anna Geller
8dc702ed16 fix(translations): fix translation key detection by comparing against last modifying commit (#7604) 2025-02-27 11:47:24 +01:00
Anna Geller
6dee52da16 fix: ci for translations (#7602)
* fix: ci for translations

* clean uo
2025-02-27 10:39:36 +01:00
Anna Geller
31dce9aadd Update generate-translations.yml 2025-02-27 10:23:30 +01:00
Anna Geller
cf1d98f56d chore(translations): standalone action for translations (#7597) 2025-02-27 10:22:01 +01:00
Harshit Dhaduk
3bbe65653b feat(ui): add ability to have persistent filter options (#7276)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-02-27 09:31:45 +01:00
Piyush Bhaskar
5108dd5e25 feat(ui): improve inspecting details of multiple executions (#7516) 2025-02-27 08:33:02 +01:00
Tanvir Ahmed
1ea331b8ab chore: introduce Devcontainer setup (#7507)
Co-authored-by: brian.mulier <bmmulier@hotmail.fr>
2025-02-26 19:26:55 +01:00
aeSouid
e9ceda4666 Fix issue text filter after label 2025-02-26 19:18:16 +01:00
brian.mulier
a996347de2 fix(ui): LabelInput.vue was causing UI freeze 2025-02-26 16:38:31 +01:00
Nicolas K.
d8e3a9ab8b Feat/npe on runcontext cleanup (#7585)
* fix(#7548): NPE on runContext cleanup if logger is null

* clean(#7548): unused import

---------

Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-02-26 14:06:41 +01:00
Miloš Paunović
e097bdbe53 chore(deps): regular dependency update (#7588) 2025-02-26 13:41:58 +01:00
Miloš Paunović
0090caad3f fix(ui): make sure adding labels on flow run dialog is possible (#7587) 2025-02-26 12:43:56 +01:00
Loïc Mathieu
7fbe433221 fix(core): properly render list properties 2025-02-26 12:03:55 +01:00
Miloš Paunović
b489848ab9 chore(ui): remove background and check mark from selected filters dropdown list (#7583) 2025-02-26 11:23:05 +01:00
Loïc Mathieu
fc7ef1ca38 fix(core): log can have no executionId
Fixes https://github.com/kestra-io/kestra-ee/issues/2980
2025-02-26 11:03:43 +01:00
Miloš Paunović
0dfe54baa8 fix(ui): amend inconsistencies inside the product tour (#7582) 2025-02-26 11:02:40 +01:00
Miloš Paunović
45c5592609 fix(ui): make flow deletion work as expected (#7579) 2025-02-26 09:04:18 +01:00
Miloš Paunović
3134d2abe2 chore(ui): respect default execution tab settings field when opening single execution (#7576) 2025-02-26 08:35:07 +01:00
aeSouid
2bb5ac61c2 Labels multiselect: (#7544)
combine the labels filter by operation to handle multiselect when building the query
2025-02-25 16:50:38 +01:00
Malaydewangan09
be04c168fd refactor: update http client and fix tests 2025-02-25 16:31:27 +01:00
Malaydewangan09
04c4916ac6 chore: add wiremock and update tests 2025-02-25 16:31:27 +01:00
Malaydewangan09
8d2af87db3 fix: refactor PluginSearchCommand 2025-02-25 16:31:27 +01:00
Malaydewangan09
e7950279bb feat: add plugin search command 2025-02-25 16:31:27 +01:00
Đỗ Trọng Hải
44936e9cdf fix(ui): improve coloring for task object tooltips in no code editor (#7515) 2025-02-25 15:38:57 +01:00
yuri
27755a3791 fix(ui): preselect filter comparator option (#7518) 2025-02-25 15:33:21 +01:00
brian.mulier
ff4f7abb0f fix(ui): styling enhancements for plugin doc
closes kestra-io/ui-libs#196
2025-02-25 15:21:24 +01:00
brian.mulier
48b117b351 fix(ui): plugins TOC is now handling every type of plugins 2025-02-25 15:21:24 +01:00
yuri
daca5675d0 fix(ui): amend label filter encoding after values change (#7536) 2025-02-25 14:05:12 +01:00
Miloš Paunović
b3278bf208 feat(ui): introduce topology export to image files (#7541) 2025-02-25 13:42:13 +01:00
Loïc Mathieu
8a26fdd83e chore(core): move run context property validation to the run context
This would avoid loading the Validator from the application context each time we render a property
2025-02-25 11:09:46 +01:00
Loïc Mathieu
38b8190be9 fix(core): Subflow using the old task name never ends
Fixes #7506
2025-02-25 11:06:31 +01:00
Miloš Paunović
52cb63c436 chore(ui): prevent system labels to be shown in set labels dialog (#7539)
* chore(ui): prevent system labels to be shown in set labels dialog

* chore(ui): improve the removal of labels
2025-02-25 10:42:07 +01:00
dependabot[bot]
f3cff1b8cd build(deps): bump io.micronaut.platform:micronaut-platform
Bumps [io.micronaut.platform:micronaut-platform](https://github.com/micronaut-projects/micronaut-platform) from 4.7.5 to 4.7.6.
- [Release notes](https://github.com/micronaut-projects/micronaut-platform/releases)
- [Commits](https://github.com/micronaut-projects/micronaut-platform/compare/v4.7.5...v4.7.6)

---
updated-dependencies:
- dependency-name: io.micronaut.platform:micronaut-platform
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-25 10:14:04 +01:00
Loïc Mathieu
a2c89e508c fix(core): handle space in HTTP request URI
Fixes #6936
2025-02-25 09:03:58 +01:00
aeSouid
c8aae742a2 fix(ui): amend filtering of logs (#7535) 2025-02-25 08:33:11 +01:00
aeSouid
eab7b746e5 Fix the search by labels input as single value 2025-02-24 19:31:14 +01:00
nKwiatkowski
1d5981665f fix(core): #172 add reactor into classloader blacklist 2025-02-24 18:41:15 +01:00
Ludovic DEHON
4500c976d6 fix(core): content type encoding should not be mandatory 2025-02-24 18:33:25 +01:00
Loïc Mathieu
cf10269f25 fix(jdbc): be resilient to DataException
We usually fail fast, but when a DataException is thrown it means the JDBC driver throws an exception with error code 22: data exception.
As the exception is from the data not the database or the network, there is no point of failfast, we throw a QueueException that may or may not be handled gracefully by the call site.
2025-02-24 16:56:10 +01:00
MilosPaunovic
cf635058cd chore(ui): use system namespace label from configs 2025-02-24 14:45:28 +01:00
aeSouid
bd6937a9f0 Fix the multi select status for the IAM/invitaiton page (#7528) 2025-02-24 13:36:34 +01:00
Miloš Paunović
59cc6f32d0 refactor(ui): remove obsolete chartjs-chart-treemap library (#7529) 2025-02-24 13:33:48 +01:00
Miloš Paunović
b8423ef70e chore(ui): purge empty labels on execution (#7527) 2025-02-24 12:24:50 +01:00
Miloš Paunović
0ec1b3a983 chore(ui): prevent sending random strings as child filter values (#7526)
* chore(ui): properly handle child filter

* chore(ui): prevent sending random strings as child filter values
2025-02-24 11:19:16 +01:00
Anna Geller
8cebd39f0d docs: debug expression (#7514)
* docs: debug expression

* fix: translation script

* fix: translations commit depth

* fix: comment
2025-02-24 10:22:21 +01:00
Ludovic DEHON
9fce6cfe65 fix(core): missing content type on http client 2025-02-21 23:18:27 +01:00
brian.mulier
c8c0c4e63f fix(core): camel to snake-case for app-blocks in RegisteredPlugin 2025-02-21 19:44:06 +01:00
Loïc Mathieu
25723b1acb fix(core): typo in PluginScanner 2025-02-21 19:40:16 +01:00
rajatsingh23
065d299422 chore(ui): restore automatic scroll to bottom on logs (#7365) 2025-02-21 15:08:13 +01:00
brian.mulier
9ec4d9282b fix(core): rename dashboards subgroups 2025-02-21 14:48:03 +01:00
brian.mulier
16284e5b95 fix(core): move package-info.java to proper dashboard packages 2025-02-21 14:35:36 +01:00
Ludovic DEHON
a115eb5373 fix(core): try to log message for unhandled realtime trigger exception 2025-02-21 12:50:21 +01:00
brian.mulier
ac2643c108 fix(core): add package-info.java to script + handle subgroups properly 2025-02-21 12:23:24 +01:00
Adam Hirshson
4710baec7d chore(ui): improve the scope of translations (#7505)
Co-authored-by: Yailan Bordas <bordasyailan@gmail.com>
Co-authored-by: Yailan Bordas <109761932+YAI-cs@users.noreply.github.com>
Co-authored-by: Christian Arauz <carau022@fiu.edu>
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-02-21 09:20:52 +01:00
Miloš Paunović
1efe531228 chore(ui): add translation key/value pairs (#7509) 2025-02-21 09:19:09 +01:00
brian.mulier
df92491e55 fix(core): add package-info.java to dashboard package 2025-02-20 19:05:58 +01:00
aeSouid
e7c65e020a chore(ui): improve scope type filters (#7504) 2025-02-20 14:14:08 +01:00
Miloš Paunović
90212ee7dc fix(ui): properly filter out log levels (#7503) 2025-02-20 12:18:58 +01:00
Miloš Paunović
ac1984b349 fix(ui): make flow metrics behave as expected (#7502) 2025-02-20 11:59:27 +01:00
YannC
c5767fd313 fix(core): require condition in Flow trigger (#7494)
close #971
2025-02-20 10:19:06 +01:00
Ludovic DEHON
8ee79264ed fix(tasks): remove useless format metrics on return (#7486) 2025-02-19 22:56:29 +01:00
Mathieu Gabelle
c6d21776ec fix: render before command with options in CommandsWrapper (#7496) 2025-02-19 17:02:31 +01:00
Aabhas Sao
45a9ea0cc5 chore(ui): simplify query search with spaces inside (#7404)
Signed-off-by: Aabhas Sao <aabhassao0@gmail.com>
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-02-19 16:26:47 +01:00
YannC
1f60a86739 fix(core): if subflow is disabled, raise an error (#7490) 2025-02-19 14:17:28 +01:00
Piyush Bhaskar
d0e8b691e3 chore(ui): amend margins of EE locked pages (#7446) 2025-02-19 14:11:45 +01:00
Piyush Bhaskar
0687a97eb8 feat(ui): add keyboard shortcuts for changing editor font size (#7450)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-02-19 13:54:20 +01:00
Miloš Paunović
dcf89b5db8 chore(translations): add missing key/value pairs (#7492) 2025-02-19 13:53:46 +01:00
Piyush Bhaskar
b1e29fbe31 chore(ui): improve styling of editor file tree (#7420) 2025-02-19 13:40:38 +01:00
Miloš Paunović
4b2113fb9b chore(ui): remove obsolete props (#7487) 2025-02-19 13:37:19 +01:00
Piyush Bhaskar
dc1728e420 fix(ui): fix the light theme contrast for editor. (#7438)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-02-19 13:31:47 +01:00
Miloš Paunović
8c83290d7a chore(deps): regular dependency update (#7485) 2025-02-19 12:54:49 +01:00
Miloš Paunović
212b3a82a9 fix(ui): improve handling of main filter labels on page load (#7456) 2025-02-19 12:08:53 +01:00
Miloš Paunović
1612c65ea3 fix(ui): properly cast value to boolean (#7455) 2025-02-19 11:29:07 +01:00
Miloš Paunović
6257b1508c chore(ui): disable click on search button if filtering is automatic (#7454) 2025-02-19 11:08:57 +01:00
yuri
f1670b8ac8 fix(webserver): allow special chars in label key (#7419) 2025-02-19 09:06:48 +01:00
brian.mulier
ce15ca1cac fix(ci): QEMU issue 2025-02-18 23:36:53 +01:00
Ludovic DEHON
a53395ab3f fix(core): http proxy was not passed to configuration
close #7449
2025-02-18 22:46:15 +01:00
Mathieu Gabelle
e0eaf675b2 refactor: return only command when no interpreter and no beforeCommands (#7452) 2025-02-18 18:59:04 +01:00
Ludovic DEHON
666f8a7ad9 fix(cicd): update concurrency key 2025-02-18 18:21:25 +01:00
brian.mulier
cebe8f3545 fix(tests): increase timeout on JdbcServiceLivenessCoordinatorTest.taskResubmitSkipExecution 2025-02-18 15:48:36 +01:00
brian.mulier
16e3830c9c fix(core): render delete property at the beginning in Docker task runner 2025-02-18 15:48:36 +01:00
brian.mulier
15b85ac952 fix(core): remove props with default from required in json schema to avoid validation errors
closes #7406
2025-02-18 15:48:36 +01:00
Miloš Paunović
db016a085f chore(ui): add arrows to namespace listing (#7448) 2025-02-18 13:54:07 +01:00
YannC
d5f6da10dd fix(): invert condition 2025-02-18 12:38:54 +01:00
brian.mulier
3723275817 fix(core): move back to the old worker thread pool because it was restricting it to 1 thread 2025-02-18 12:24:21 +01:00
YannC
ef29883138 fix(core): provide tenantId when looking for subflow (#7442) 2025-02-18 11:38:25 +01:00
Mathieu Gabelle
5ced9b6c4f refactor: remove rendering from Docker (#7439)
rendering is done in the CommandsWrapper so no need to render again in the task runner
2025-02-18 11:15:14 +01:00
Miloš Paunović
11f3629f38 chore(ui): add missing translations (#7444) 2025-02-18 11:11:50 +01:00
Miloš Paunović
d8c5a1e14c chore(ui): amend namespace flow creation label (#7443) 2025-02-18 11:09:55 +01:00
Miloš Paunović
cee2734817 fix(ui): improve modifying inputs from no code editor (#7440) 2025-02-18 10:36:15 +01:00
brian.mulier
13cb0fb96d fix(tests): logs are asynchronously inserted so we wait for them to be fully in 2025-02-18 10:19:58 +01:00
YannC
a020e3f3ae ci(test): force test if ref is a tag 2025-02-18 09:46:28 +01:00
brian.mulier
12fef24279 chore(deps): bump ui-libs to 0.0.141 2025-02-18 01:38:22 +01:00
rajatsingh23
4a55485cd6 fix(ui): better duration consistency on Gantt chart 2025-02-18 00:38:54 +01:00
brian.mulier
aed5734e17 chore(deps): bump ui-libs to 0.0.140 2025-02-18 00:25:39 +01:00
brian.mulier
d55ce16f57 fix(tests): wider maxDuration for retry-failed-flow-duration.yml 2025-02-18 00:05:14 +01:00
aeSouid
4abf3e9e6b Filter feat: (#7432)
Fix handle namespace column in filtering on scope
2025-02-17 14:43:03 +01:00
Bart Ledoux
fbd8934346 fix: restore red dot when there is news 2025-02-17 14:17:13 +01:00
Bart Ledoux
df6d339310 refactor: avoid en.json warning when building 2025-02-17 14:16:47 +01:00
YannC
a89ef71580 ci(publish-docker): attempts with command on qemu docker image 2025-02-17 11:15:14 +01:00
YannC
89c2d3951c ci(publish-docker): attempts with ubuntu 0.20 (#7431) 2025-02-17 10:48:10 +01:00
Mathieu Gabelle
e061a3617c refactor: introduce render in commands wrapper for property string (#7430) 2025-02-17 10:44:54 +01:00
Mathieu Gabelle
410cf0e389 fix: enable rendering of commands properties inside CommandsWrapper (#7381)
* fix: move commands to Property

migrate to Property in TaskCommands and CommandsWrapper
implement beforeCommand and interpreter
2025-02-17 09:49:49 +01:00
YannC
f783ab72b5 fix(test): attempt at making the test not flakky (#7400) 2025-02-17 09:20:26 +01:00
Anna Geller
72e78c8c31 Update ExecutionOutputs.java (#7394) 2025-02-17 09:20:10 +01:00
YannC
a716094a9d ci(): now run the file changes detection also in the workflow test 2025-02-17 09:15:26 +01:00
YannC
c75def0eac ci():allow release to be run 2025-02-17 09:10:08 +01:00
Piyush Bhaskar
c27d5ce480 chore(ui): improved Logs empty page (#7415) 2025-02-16 22:44:00 +01:00
aeSouid
49960d67ab chore(ui): improve filter parameters decoding for absolute date (#7409) 2025-02-14 14:51:13 +01:00
Loïc Mathieu
d9d2f86971 fix(core): ForEachItem inside an If task
Fixes #6801
2025-02-14 14:46:09 +01:00
Laibrez
95f5862201 chore(ui): improve execution outputs section (#7377)
Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2025-02-14 14:45:20 +01:00
AJ Emerich
e6419cf7a2 fix(ui): correct english translations (#7401)
Closes https://github.com/kestra-io/kestra-ee/issues/2913
2025-02-14 14:19:38 +01:00
Miloš Paunović
c1faeab068 chore(ui): make import flows button be a secondary one, styling-wise (#7405) 2025-02-14 12:59:44 +01:00
Aabhas Sao
dd3ff3e771 chore(ui): vertically center all elements in table rows (#7372)
Signed-off-by: Aabhas Sao <aabhassao0@gmail.com>
2025-02-14 11:52:37 +01:00
Miloš Paunović
e635338b47 chore(ui): improve colors of filter dropdown selector (#7403) 2025-02-14 11:47:10 +01:00
Loïc Mathieu
ddfed2e65c fix(core): taskrun list can be null 2025-02-14 11:45:33 +01:00
aeSouid
52fe6afae0 Filter feat: (#7389)
namespace mulitple choice in and not in
2025-02-14 11:37:57 +01:00
Miloš Paunović
a9ff2af159 fix(ui): properly handle the operation labels in filter component (#7399) 2025-02-14 10:50:16 +01:00
1120 changed files with 47527 additions and 31574 deletions

82
.devcontainer/Dockerfile Normal file
View File

@@ -0,0 +1,82 @@
FROM ubuntu:24.04
ARG BUILDPLATFORM
ARG DEBIAN_FRONTEND=noninteractive
USER root
WORKDIR /root
RUN apt update && apt install -y \
apt-transport-https ca-certificates gnupg curl wget git zip unzip less zsh net-tools iputils-ping jq lsof
ENV HOME="/root"
# --------------------------------------
# Git
# --------------------------------------
# Need to add the devcontainer workspace folder as a safe directory to enable git
# version control system to be enabled in the containers file system.
RUN git config --global --add safe.directory "/workspaces/kestra"
# --------------------------------------
# --------------------------------------
# Oh my zsh
# --------------------------------------
RUN sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" -- \
-t robbyrussell \
-p git -p node -p npm
ENV SHELL=/bin/zsh
# --------------------------------------
# --------------------------------------
# Java
# --------------------------------------
ARG OS_ARCHITECTURE
RUN mkdir -p /usr/java
RUN echo "Building on platform: $BUILDPLATFORM"
RUN case "$BUILDPLATFORM" in \
"linux/amd64") OS_ARCHITECTURE="linux-x64" ;; \
"linux/arm64") OS_ARCHITECTURE="linux-aarch64" ;; \
"darwin/amd64") OS_ARCHITECTURE="macos-x64" ;; \
"darwin/arm64") OS_ARCHITECTURE="macos-aarch64" ;; \
*) echo "Unsupported BUILDPLATFORM: $BUILDPLATFORM" && exit 1 ;; \
esac && \
wget "https://aka.ms/download-jdk/microsoft-jdk-21.0.6-$OS_ARCHITECTURE.tar.gz" && \
mv "microsoft-jdk-21.0.6-$OS_ARCHITECTURE.tar.gz" microsoft-jdk-21.0.6.tar.gz
RUN tar -xzvf microsoft-jdk-21.0.6.tar.gz && \
mv jdk-21.0.6+7 jdk-21 && \
mv jdk-21 /usr/java/
ENV JAVA_HOME=/usr/java/jdk-21
ENV PATH="$PATH:$JAVA_HOME/bin"
# Will load a custom configuration file for Micronaut
ENV MICRONAUT_ENVIRONMENTS=local,override
# Sets the path where you save plugins as Jar and is loaded during the startup process
ENV KESTRA_PLUGINS_PATH="/workspaces/kestra/local/plugins"
# --------------------------------------
# --------------------------------------
# Node.js
# --------------------------------------
RUN curl -fsSL https://deb.nodesource.com/setup_22.x -o nodesource_setup.sh \
&& bash nodesource_setup.sh && apt install -y nodejs
# Increases JavaScript heap memory to 4GB to prevent heap out of error during startup
ENV NODE_OPTIONS=--max-old-space-size=4096
# --------------------------------------
# --------------------------------------
# Python
# --------------------------------------
RUN apt install -y python3 pip python3-venv
# --------------------------------------
# --------------------------------------
# SSH
# --------------------------------------
RUN mkdir -p ~/.ssh
RUN touch ~/.ssh/config
RUN echo "Host github.com" >> ~/.ssh/config \
&& echo " IdentityFile ~/.ssh/id_ed25519" >> ~/.ssh/config
RUN touch ~/.ssh/id_ed25519
# --------------------------------------

149
.devcontainer/README.md Normal file
View File

@@ -0,0 +1,149 @@
# Kestra Devcontainer
This devcontainer provides a quick and easy setup for anyone using VSCode to get up and running quickly with this project to start development on either the frontend or backend. It bootstraps a docker container for you to develop inside of without the need to manually setup the environment.
---
## INSTRUCTIONS
### Setup:
Take a look at this guide to get an idea of what the setup is like as this devcontainer setup follows this approach: https://kestra.io/docs/getting-started/contributing
Once you have this repo cloned to your local system, you will need to install the VSCode extension [Remote Development](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack).
Then run the following command from the command palette:
`Dev Containers: Open Folder in Container...` and select your Kestra root folder.
This will then put you inside a docker container ready for development.
NOTE: you'll need to wait for the gradle build to finish and compile Java files but this process should happen automatically within VSCode.
In the meantime, you can move onto the next step...
---
### Development:
- Create a `.env.development.local` file in the `ui` folder and paste the following:
```bash
# This lets the frontend know what the backend URL is but you are free to change this to your actual server URL e.g. hosted version of Kestra.
VITE_APP_API_URL=http://localhost:8080
```
- Navigate into the `ui` folder and run `npm install` to install the dependencies for the frontend project.
- Now go to the `cli/src/main/resources` folder and create a `application-override.yml` file.
Now you have two choices:
`Local mode`:
Runs the Kestra server in local mode which uses a H2 database, so this is the only config you'd need:
```yaml
micronaut:
server:
cors:
enabled: true
configurations:
all:
allowedOrigins:
- http://localhost:5173
```
You can then open a new terminal and run the following command to start the backend server: `./gradlew runLocal`
`Standalone mode`:
Runs in standalone mode which uses Postgres. Make sure to have a local Postgres instance already running on localhost:
```yaml
kestra:
repository:
type: postgres
storage:
type: local
local:
base-path: "/app/storage"
queue:
type: postgres
tasks:
tmp-dir:
path: /tmp/kestra-wd/tmp
anonymous-usage-report:
enabled: false
server:
basic-auth:
enabled: false
datasources:
postgres:
# It is important to note that you must use the "host.docker.internal" host when connecting to a docker container outside of your devcontainer as attempting to use localhost will only point back to this devcontainer.
url: jdbc:postgresql://host.docker.internal:5432/kestra
driverClassName: org.postgresql.Driver
username: kestra
password: k3str4
flyway:
datasources:
postgres:
enabled: true
locations:
- classpath:migrations/postgres
# We must ignore missing migrations as we may delete the wrong ones or delete those that are not used anymore.
ignore-migration-patterns: "*:missing,*:future"
out-of-order: true
micronaut:
server:
cors:
enabled: true
configurations:
all:
allowedOrigins:
- http://localhost:5173
```
Then add the following settings to the `.vscode/launch.json` file:
```json
{
"version": "0.2.0",
"configurations": [
{
"type": "java",
"name": "Kestra Standalone",
"request": "launch",
"mainClass": "io.kestra.cli.App",
"projectName": "cli",
"args": "server standalone"
}
]
}
```
You can then use the VSCode `Run and Debug` extension to start the Kestra server.
Additionally, if you're doing frontend development, you can run `npm run dev` from the `ui` folder after having the above running (which will provide a backend) to access your application from `localhost:5173`. This has the benefit to watch your changes and hot-reload upon doing frontend changes.
#### Plugins
If you want your plugins to be loaded inside your devcontainer, point the `source` field to a folder containing jars of the plugins you want to embed in the following snippet in `devcontainer.json`:
```
"mounts": [
{
"source": "/absolute/path/to/your/local/jar/plugins/folder",
"target": "/workspaces/kestra/local/plugins",
"type": "bind"
}
],
```
---
### GIT
If you want to commit to GitHub, make sure to navigate to the `~/.ssh` folder and either create a new SSH key or override the existing `id_ed25519` file and paste an existing SSH key from your local machine into this file. You will then need to change the permissions of the file by running: `chmod 600 id_ed25519`. This will allow you to then push to GitHub.
---

View File

@@ -0,0 +1,46 @@
{
"name": "kestra",
"build": {
"context": ".",
"dockerfile": "Dockerfile"
},
"workspaceFolder": "/workspaces/kestra",
"forwardPorts": [5173, 8080],
"customizations": {
"vscode": {
"settings": {
"terminal.integrated.profiles.linux": {
"zsh": {
"path": "/bin/zsh"
}
},
"workbench.iconTheme": "vscode-icons",
"editor.tabSize": 4,
"editor.formatOnSave": true,
"files.insertFinalNewline": true,
"editor.defaultFormatter": "esbenp.prettier-vscode",
"telemetry.telemetryLevel": "off",
"editor.bracketPairColorization.enabled": true,
"editor.guides.bracketPairs": "active"
},
"extensions": [
"redhat.vscode-yaml",
"dbaeumer.vscode-eslint",
"vscode-icons-team.vscode-icons",
"eamodio.gitlens",
"esbenp.prettier-vscode",
"aaron-bond.better-comments",
"codeandstuff.package-json-upgrade",
"andys8.jest-snippets",
"oderwat.indent-rainbow",
"evondev.indent-rainbow-palettes",
"formulahendry.auto-rename-tag",
"IronGeek.vscode-env",
"yoavbls.pretty-ts-errors",
"github.vscode-github-actions",
"vscjava.vscode-java-pack",
"ms-azuretools.vscode-docker"
]
}
}
}

View File

@@ -37,6 +37,10 @@ The following dependencies are required to build Kestra locally:
- Docker & Docker Compose
- an IDE (Intellij IDEA, Eclipse or VS Code)
Thanks to the Kestra community, if using VSCode, you can also start development on either the frontend or backend with a bootstrapped docker container without the need to manually set up the environment.
Check out the [README](../.devcontainer/README.md) for set-up instructions and the associated [Dockerfile](../.devcontainer/Dockerfile) in the respository to get started.
To start contributing:
- [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the repository
- Clone the fork on your workstation:
@@ -46,7 +50,7 @@ git clone git@github.com:{YOUR_USERNAME}/kestra.git
cd kestra
```
#### Develop backend
#### Develop on the backend
The backend is made with [Micronaut](https://micronaut.io).
Open the cloned repository in your favorite IDE. In most of decent IDEs, Gradle build will be detected and all dependencies will be downloaded.
@@ -72,7 +76,7 @@ python3 -m pip install virtualenv
```
#### Develop frontend
#### Develop on the frontend
The frontend is made with [Vue.js](https://vuejs.org/) and located on the `/ui` folder.
- `npm install`

View File

@@ -1,57 +0,0 @@
name: Generate Translations
description: "Required the environment variable OPENAI_API_KEY to be set. This action will generate translations for the UI."
inputs:
github-token:
description: 'GitHub Token'
required: true
runs:
using: composite
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.x"
- name: Install Python dependencies
shell: bash
run: pip install gitpython openai
- name: Generate translations
shell: bash
run: python ui/src/translations/generate_translations.py
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "20.x"
- name: Check keys matching
shell: bash
run: node ui/src/translations/check.js
- name: Set up Git
shell: bash
run: |
git config --global user.name "GitHub Action"
git config --global user.email "actions@github.com"
- name: Check for changes and commit
env:
GH_TOKEN: ${{ inputs.github-token }}
shell: bash
run: |
git add ui/src/translations/*.json
if git diff --cached --quiet; then
echo "No changes to commit. Exiting with success."
exit 0
fi
git commit -m "chore(translations): auto generate values for languages other than english"
git push origin ${{ github.head_ref }}

View File

@@ -62,7 +62,7 @@ jobs:
- name: Build with Gradle
if: ${{ matrix.language == 'java' }}
run: ./gradlew testClasses -x :ui:installFrontend -x :ui:assembleFrontend
run: ./gradlew testClasses -x :ui:assembleFrontend
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)

View File

@@ -47,11 +47,11 @@ jobs:
image:
- name: "-no-plugins"
plugins: ""
packages: ""
packages: jattach
python-libs: ""
- name: ""
plugins: ${{needs.plugins.outputs.plugins}}
packages: python3 python3-venv python-is-python3 python3-pip nodejs npm curl zip unzip
packages: python3 python3-venv python-is-python3 python3-pip nodejs npm curl zip unzip jattach
python-libs: kestra
steps:
- uses: actions/checkout@v4
@@ -63,11 +63,11 @@ jobs:
if [[ "${{ inputs.release-tag }}" == "" ]]; then
TAG=${GITHUB_REF#refs/*/}
echo "tag=${TAG}" >> $GITHUB_OUTPUT
else
else
TAG="${{ inputs.release-tag }}"
echo "tag=${TAG}" >> $GITHUB_OUTPUT
fi
fi
if [[ "${{ env.PLUGIN_VERSION }}" == *"-SNAPSHOT" ]]; then
echo "plugins=--repositories=https://s01.oss.sonatype.org/content/repositories/snapshots ${{ matrix.image.plugins }}" >> $GITHUB_OUTPUT;
else
@@ -75,7 +75,7 @@ jobs:
fi
# Download release
- name: Download release
uses: robinraju/release-downloader@v1.11
uses: robinraju/release-downloader@v1.12
with:
tag: ${{steps.vars.outputs.tag}}
fileName: 'kestra-*'
@@ -89,6 +89,11 @@ jobs:
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Docker - Fix Qemu
shell: bash
run: |
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes -c yes
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

View File

@@ -0,0 +1,67 @@
name: Auto-Translate UI keys and create PR
on:
schedule:
- cron: "0 9-21 * * *" # Every hour from 9 AM to 9 PM
workflow_dispatch:
inputs:
retranslate_modified_keys:
description: "Whether to re-translate modified keys even if they already have translations."
type: choice
options:
- "false"
- "true"
default: "false"
required: false
jobs:
translations:
name: Translations
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
name: Checkout
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.x"
- name: Install Python dependencies
run: pip install gitpython openai
- name: Generate translations
run: python ui/src/translations/generate_translations.py ${{ github.event.inputs.retranslate_modified_keys }}
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "20.x"
- name: Check keys matching
run: node ui/src/translations/check.js
- name: Set up Git
run: |
git config --global user.name "GitHub Action"
git config --global user.email "actions@github.com"
- name: Commit and create PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
BRANCH_NAME="chore/update-translations-$(date +%s)"
git checkout -b $BRANCH_NAME
git add ui/src/translations/*.json
if git diff --cached --quiet; then
echo "No changes to commit. Exiting with success."
exit 0
fi
git commit -m "chore(core): localize to languages other than english" -m "Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference."
git push -u origin $BRANCH_NAME || (git push origin --delete $BRANCH_NAME && git push -u origin $BRANCH_NAME)
gh pr create --title "Translations from en.json" --body "This PR was created automatically by a GitHub Action." --base develop --head $BRANCH_NAME --assignee anna-geller --reviewer anna-geller

View File

@@ -18,7 +18,7 @@ on:
- v*
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
group: ${{ github.workflow }}-${{ github.ref }}-main
cancel-in-progress: true
jobs:
@@ -31,6 +31,7 @@ jobs:
release:
name: Release
needs: [tests]
if: "!startsWith(github.ref, 'refs/heads/releases')"
uses: ./.github/workflows/workflow-release.yml
with:
plugin-version: ${{ github.event.inputs.plugin-version != null && github.event.inputs.plugin-version || 'LATEST' }}

View File

@@ -6,11 +6,15 @@ on:
- develop
concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
group: ${{ github.workflow }}-${{ github.ref_name }}-pr
cancel-in-progress: true
jobs:
# ********************************************************************************************************************
# File changes detection
# ********************************************************************************************************************
file-changes:
if: ${{ github.event.pull_request.draft == false }}
name: File changes detection
runs-on: ubuntu-latest
timeout-minutes: 60
@@ -25,34 +29,16 @@ jobs:
filters: |
ui:
- 'ui/**'
translations:
- 'ui/src/translations/**'
backend:
- '!{ui,.github}/**'
token: ${{ secrets.GITHUB_TOKEN }}
translations:
name: 'Translations - Generate translations'
runs-on: ubuntu-latest
timeout-minutes: 60
needs: file-changes
steps:
- uses: actions/checkout@v4
name: Checkout - Current ref
if: "needs.file-changes.outputs.translations == 'true'"
- id: generate-translations
name: Translations - Generate translations
if: "needs.file-changes.outputs.translations == 'true'"
uses: ./.github/actions/generate-translations
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
# ********************************************************************************************************************
# Tests
# ********************************************************************************************************************
frontend:
name: Frontend - Tests
needs: [file-changes, translations]
needs: [file-changes]
if: "needs.file-changes.outputs.ui == 'true'"
uses: ./.github/workflows/workflow-frontend-test.yml
secrets:

View File

@@ -23,12 +23,11 @@ jobs:
exit 1
fi
CURRENT_BRANCH="{{ github.ref }}"
# Extract the major and minor versions
BASE_VERSION=$(echo "$RELEASE_VERSION" | sed -E 's/^([0-9]+\.[0-9]+)\..*/\1/')
RELEASE_BRANCH="refs/heads/releases/v${BASE_VERSION}.x"
CURRENT_BRANCH="$GITHUB_REF"
if ! [[ "$CURRENT_BRANCH" == "$RELEASE_BRANCH" ]]; then
echo "Invalid release branch. Expected $RELEASE_BRANCH, was $CURRENT_BRANCH"
exit 1

View File

@@ -8,6 +8,9 @@ on:
env:
JAVA_VERSION: '21'
permissions:
contents: read
jobs:
dependency-check:
name: Dependency Check
@@ -37,7 +40,7 @@ jobs:
- name: Npm - Install
shell: bash
working-directory: ui
run: npm install
run: npm ci
# Run OWASP dependency check plugin
- name: Gradle Dependency Check
@@ -57,6 +60,10 @@ jobs:
develop-image-check:
name: Image Check (develop)
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
actions: read
steps:
# Checkout
- uses: actions/checkout@v4
@@ -80,16 +87,28 @@ jobs:
# Run Trivy image scan for Docker vulnerabilities, see https://github.com/aquasecurity/trivy-action
- name: Docker Vulnerabilities Check
uses: aquasecurity/trivy-action@0.29.0
uses: aquasecurity/trivy-action@0.30.0
with:
image-ref: kestra/kestra:develop
format: table
format: 'template'
template: '@/contrib/sarif.tpl'
severity: 'CRITICAL,HIGH'
output: 'trivy-results.sarif'
skip-dirs: /app/plugins
scanners: vuln
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: 'trivy-results.sarif'
category: docker-
latest-image-check:
name: Image Check (latest)
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
actions: read
steps:
# Checkout
- uses: actions/checkout@v4
@@ -113,9 +132,16 @@ jobs:
# Run Trivy image scan for Docker vulnerabilities, see https://github.com/aquasecurity/trivy-action
- name: Docker Vulnerabilities Check
uses: aquasecurity/trivy-action@0.29.0
uses: aquasecurity/trivy-action@0.30.0
with:
image-ref: kestra/kestra:latest
format: table
skip-dirs: /app/plugins
scanners: vuln
scanners: vuln
severity: 'CRITICAL,HIGH'
output: 'trivy-results.sarif'
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: 'trivy-results.sarif'

View File

@@ -16,6 +16,11 @@ on:
description: 'Google Service Account'
required: true
permissions:
contents: write
checks: write
actions: read
jobs:
test:
name: Backend - Tests
@@ -26,6 +31,8 @@ jobs:
steps:
- uses: actions/checkout@v4
name: Checkout - Current ref
with:
fetch-depth: 0
# Setup build
- uses: kestra-io/actions/.github/actions/setup-build@main
@@ -54,7 +61,7 @@ jobs:
# report test
- name: Test - Publish Test Results
uses: dorny/test-reporter@v1
uses: dorny/test-reporter@v2
if: always()
with:
name: Java Tests Report
@@ -63,6 +70,7 @@ jobs:
list-suites: 'failed'
list-tests: 'failed'
fail-on-error: 'false'
token: ${{ secrets.GITHUB_AUTH_TOKEN }}
# Sonar
- name: Test - Analyze with Sonar

View File

@@ -107,6 +107,11 @@ jobs:
- name: Docker - Setup QEMU
uses: docker/setup-qemu-action@v3
- name: Docker - Fix Qemu
shell: bash
run: |
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes -c yes
- name: Docker - Setup Buildx
uses: docker/setup-buildx-action@v3

View File

@@ -19,11 +19,8 @@ jobs:
name: Frontend - Tests
runs-on: ubuntu-latest
steps:
- id: checkout
name: Checkout - Current ref
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.head_ref }}
- name: Npm - install
shell: bash
@@ -44,28 +41,15 @@ jobs:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
run: npm run build
- name: Run front-end unit tests
shell: bash
working-directory: ui
run: npm run test:cicd
- name: Storybook - Install Playwright
shell: bash
working-directory: ui
run: npx playwright install --with-deps
- name: Storybook - Build
- name: Run front-end unit tests
shell: bash
working-directory: ui
run: npm run build-storybook --quiet
- name: Storybook - Run tests
shell: bash
working-directory: ui
run: |
npx concurrently -k -s first -n "SB,TEST" -c "magenta,blue" \
"npx http-server storybook-static --port 6006 --silent" \
"npx wait-on tcp:127.0.0.1:6006 && npm run test:storybook"
run: npm run test:cicd
- name: Codecov - Upload coverage reports
uses: codecov/codecov-action@v5

View File

@@ -20,17 +20,23 @@ jobs:
name: exe
path: build/executable
# GitHub Release
- name: GitHub - Create release
id: create_github_release
uses: "marvinpinto/action-automatic-releases@latest"
if: startsWith(github.ref, 'refs/tags/v')
continue-on-error: true
# Checkout GitHub Actions
- name: Checkout - Actions
uses: actions/checkout@v4
with:
repo_token: "${{ secrets.GITHUB_TOKEN }}"
prerelease: false
files: |
build/executable/*
repository: kestra-io/actions
sparse-checkout-cone-mode: true
path: actions
sparse-checkout: |
.github/actions
# GitHub Release
- name: Create GitHub release
uses: ./actions/.github/actions/github-release
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SLACK_RELEASES_WEBHOOK_URL: ${{ secrets.SLACK_RELEASES_WEBHOOK_URL }}
# Trigger gha workflow to bump helm chart version
- name: GitHub - Trigger the Helm chart version bump

View File

@@ -1,7 +1,30 @@
name: Publish - Docker
on:
workflow_dispatch:
inputs:
plugin-version:
description: "Kestra version"
default: 'LATEST'
required: false
type: string
force-download-artifact:
description: 'Force download artifact'
required: false
type: string
default: "true"
workflow_call:
inputs:
plugin-version:
description: "Kestra version"
default: 'LATEST'
required: false
type: string
force-download-artifact:
description: 'Force download artifact'
required: false
type: string
default: "true"
secrets:
DOCKERHUB_USERNAME:
description: "The Dockerhub username."
@@ -11,26 +34,39 @@ on:
required: true
jobs:
# ********************************************************************************************************************
# Build
# ********************************************************************************************************************
build-artifacts:
name: Build - Artifacts
name: Build Artifacts
if: ${{ github.event.inputs.force-download-artifact == 'true' }}
uses: ./.github/workflows/workflow-build-artifacts.yml
with:
plugin-version: ${{ github.event.inputs.plugin-version != null && github.event.inputs.plugin-version || 'LATEST' }}
# ********************************************************************************************************************
# Docker
# ********************************************************************************************************************
publish:
name: Publish - Docker
needs: build-artifacts
runs-on: ubuntu-latest
needs: build-artifacts
if: |
always() &&
(needs.build-artifacts.result == 'success' ||
github.event.inputs.force-download-artifact != 'true')
env:
PLUGIN_VERSION: ${{ github.event.inputs.plugin-version != null && github.event.inputs.plugin-version || 'LATEST' }}
strategy:
matrix:
image:
- tag: ${{ needs.build-artifacts.outputs.docker-tag }}-no-plugins
packages: ""
- tag: -no-plugins
packages: jattach
plugins: false
python-libraries: ""
- tag: ${{ needs.build-artifacts.outputs.docker-tag }}
plugins: ${{ needs.build-artifacts.outputs.plugins }}
packages: python3 python3-venv python-is-python3 python3-pip nodejs npm curl zip unzip
- tag: ""
plugins: true
packages: python3 python3-venv python-is-python3 python3-pip nodejs npm curl zip unzip jattach
python-libraries: kestra
steps:
- name: Checkout - Current ref
@@ -40,6 +76,11 @@ jobs:
- name: Docker - Setup QEMU
uses: docker/setup-qemu-action@v3
- name: Docker - Fix Qemu
shell: bash
run: |
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes -c yes
- name: Docker - Setup Docker Buildx
uses: docker/setup-buildx-action@v3
@@ -50,17 +91,34 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
# # Get Plugins List
- name: Plugins - Get List
uses: ./.github/actions/plugins-list
id: plugins-list
if: ${{ matrix.image.plugins}}
with:
plugin-version: ${{ env.PLUGIN_VERSION }}
# Vars
- name: Docker - Set image name
- name: Docker - Set variables
shell: bash
id: vars
run: |
TAG=${GITHUB_REF#refs/*/}
if [[ $TAG = "master" || $TAG == v* ]]; then
PLUGINS="${{ matrix.image.plugins == true && steps.plugins-list.outputs.plugins || '' }}"
if [[ $TAG == v* ]]; then
TAG="${TAG}";
echo "plugins=${{ matrix.image.plugins }}" >> $GITHUB_OUTPUT
elif [[ $TAG = "develop" ]]; then
TAG="develop";
echo "plugins=--repositories=https://s01.oss.sonatype.org/content/repositories/snapshots $PLUGINS" >> $GITHUB_OUTPUT
else
echo "plugins=--repositories=https://s01.oss.sonatype.org/content/repositories/snapshots ${{ matrix.image.plugins }}" >> $GITHUB_OUTPUT
TAG="build-${{ github.run_id }}";
echo "plugins=--repositories=https://s01.oss.sonatype.org/content/repositories/snapshots $PLUGINS" >> $GITHUB_OUTPUT
fi
echo "tag=${TAG}${{ matrix.image.tag }}" >> $GITHUB_OUTPUT
# Build Docker Image
- name: Artifacts - Download executable
@@ -80,7 +138,7 @@ jobs:
with:
context: .
push: true
tags: kestra/kestra:${{ matrix.image.tag }}
tags: kestra/kestra:${{ steps.vars.outputs.tag }}
platforms: linux/amd64,linux/arm64
build-args: |
KESTRA_PLUGINS=${{ steps.vars.outputs.plugins }}

View File

@@ -1,6 +1,18 @@
name: Release
on:
workflow_dispatch:
inputs:
plugin-version:
description: "Kestra version"
default: 'LATEST'
required: false
type: string
publish-docker:
description: "Publish Docker image"
default: 'false'
required: false
type: string
workflow_call:
inputs:
plugin-version:
@@ -31,12 +43,23 @@ on:
description: "The Sonatype GPG file."
required: true
jobs:
build-artifacts:
name: Build - Artifacts
uses: ./.github/workflows/workflow-build-artifacts.yml
with:
plugin-version: ${{ github.event.inputs.plugin-version != null && github.event.inputs.plugin-version || 'LATEST' }}
Docker:
name: Publish Docker
needs: build-artifacts
uses: ./.github/workflows/workflow-publish-docker.yml
if: startsWith(github.ref, 'refs/heads/develop') || github.event.inputs.publish-docker == 'true'
with:
force-download-artifact: 'false'
plugin-version: ${{ github.event.inputs.plugin-version != null && github.event.inputs.plugin-version || 'LATEST' }}
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_PASSWORD: ${{ secrets.DOCKERHUB_PASSWORD }}
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_PASSWORD: ${{ secrets.DOCKERHUB_PASSWORD }}
Maven:
name: Publish Maven
@@ -50,6 +73,7 @@ jobs:
Github:
name: Github Release
needs: build-artifacts
if: startsWith(github.ref, 'refs/tags/v')
uses: ./.github/workflows/workflow-github-release.yml
secrets:

View File

@@ -19,8 +19,31 @@ on:
value: ${{ jobs.set-backend-status.outputs.backend_status }}
jobs:
file-changes:
name: File changes detection
runs-on: ubuntu-latest
timeout-minutes: 60
outputs:
ui: ${{ steps.changes.outputs.ui }}
backend: ${{ steps.changes.outputs.backend }}
steps:
- uses: actions/checkout@v4
if: "!startsWith(github.ref, 'refs/tags/v')"
- uses: dorny/paths-filter@v3
if: "!startsWith(github.ref, 'refs/tags/v')"
id: changes
with:
filters: |
ui:
- 'ui/**'
backend:
- '!{ui,.github}/**'
token: ${{ secrets.GITHUB_TOKEN }}
frontend:
name: Frontend - Tests
needs: file-changes
if: "needs.file-changes.outputs.ui == 'true' || startsWith(github.ref, 'refs/tags/v')"
uses: ./.github/workflows/workflow-frontend-test.yml
secrets:
GITHUB_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -29,6 +52,8 @@ jobs:
backend:
name: Backend - Tests
needs: file-changes
if: "needs.file-changes.outputs.backend == 'true' || startsWith(github.ref, 'refs/tags/v')"
uses: ./.github/workflows/workflow-backend-test.yml
secrets:
GITHUB_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

2
.gitignore vendored
View File

@@ -37,6 +37,7 @@ ui/coverage
ui/stats.html
ui/.frontend-gradle-plugin
ui/utils/CHANGELOG.md
ui/test-report.junit.xml
### Docker
/.env
@@ -57,3 +58,4 @@ core/src/main/resources/gradle.properties
**/allure-results/*
*storybook.log
storybook-static

View File

@@ -32,6 +32,7 @@
#plugin-git:io.kestra.plugin:plugin-git:LATEST
#plugin-github:io.kestra.plugin:plugin-github:LATEST
#plugin-googleworkspace:io.kestra.plugin:plugin-googleworkspace:LATEST
#plugin-graalvm:io.kestra.plugin:plugin-graalvm:LATEST
#plugin-hightouch:io.kestra.plugin:plugin-hightouch:LATEST
#plugin-hubspot:io.kestra.plugin:plugin-hubspot:LATEST
#plugin-huggingface:io.kestra.plugin:plugin-huggingface:LATEST

View File

@@ -89,7 +89,7 @@ build-docker: build-exec
--compress \
--rm \
-f ./Dockerfile \
--build-arg="APT_PACKAGES=python3 python3-venv python-is-python3 python3-pip nodejs npm curl zip unzip" \
--build-arg="APT_PACKAGES=python3 python3-venv python-is-python3 python3-pip nodejs npm curl zip unzip jattach" \
--build-arg="PYTHON_LIBRARIES=kestra" \
-t ${DOCKER_IMAGE}:${VERSION} ${DOCKER_PATH} || exit 1 ;
@@ -181,8 +181,8 @@ clone-plugins:
@echo "Using PLUGIN_GIT_DIR: $(PLUGIN_GIT_DIR)"
@mkdir -p "$(PLUGIN_GIT_DIR)"
@echo "Fetching repository list from GitHub..."
@REPOS=$(gh repo list kestra-io -L 1000 --json name | jq -r .[].name | sort | grep "^plugin-") \
for repo in $$REPOS; do \
@REPOS=$$(gh repo list kestra-io -L 1000 --json name | jq -r .[].name | sort | grep "^plugin-"); \
for repo in $$REPOS; do \
if [[ $$repo == plugin-* ]]; then \
if [ -d "$(PLUGIN_GIT_DIR)/$$repo" ]; then \
echo "Skipping: $$repo (Already cloned)"; \
@@ -194,6 +194,22 @@ clone-plugins:
done
@echo "Done!"
# Pull every plugins in main or master branch
pull-plugins:
@echo "🔍 Pulling repositories in '$(PLUGIN_GIT_DIR)'..."
@for repo in "$(PLUGIN_GIT_DIR)"/*; do \
if [ -d "$$repo/.git" ]; then \
branch=$$(git -C "$$repo" rev-parse --abbrev-ref HEAD); \
if [[ "$$branch" == "master" || "$$branch" == "main" ]]; then \
echo "🔄 Pulling: $$(basename "$$repo") (branch: $$branch)"; \
git -C "$$repo" pull; \
else \
echo "❌ Skipping: $$(basename "$$repo") (Not on master or main branch, currently on $$branch)"; \
fi; \
fi; \
done
@echo "✅ Done pulling!"
# Update all plugins jar
build-plugins:
@echo "🔍 Scanning repositories in '$(PLUGIN_GIT_DIR)'..."

View File

@@ -24,6 +24,13 @@
<a href="https://www.youtube.com/@kestra-io"><img height="25" src="https://kestra.io/youtube.svg" alt="youtube" /></a> &nbsp;
</p>
<p align="center">
<a href="https://trendshift.io/repositories/2714" target="_blank">
<img src="https://trendshift.io/api/badge/repositories/2714" alt="kestra-io%2Fkestra | Trendshift" width="250" height="55"/>
</a>
<a href="https://www.producthunt.com/posts/kestra?embed=true&utm_source=badge-top-post-badge&utm_medium=badge&utm_souce=badge-kestra" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/top-post-badge.svg?post_id=612077&theme=light&period=daily&t=1740737506162" alt="Kestra - All&#0045;in&#0045;one&#0032;automation&#0032;&#0038;&#0032;orchestration&#0032;platform | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
</p>
<p align="center">
<a href="https://go.kestra.io/video/product-overview" target="_blank">
<img src="https://kestra.io/startvideo.png" alt="Get started in 4 minutes with Kestra" width="640px" />
@@ -47,7 +54,7 @@ Kestra is an open-source, event-driven orchestration platform that makes both **
- **Structure & Resilience**: tame chaos and bring resilience to your workflows with **namespaces**, **labels**, **subflows**, **retries**, **timeout**, **error handling**, **inputs**, **outputs** that generate artifacts in the UI, **variables**, **conditional branching**, **advanced scheduling**, **event triggers**, **backfills**, **dynamic tasks**, **sequential and parallel tasks**, and skip tasks or triggers when needed by setting the flag `disabled` to `true`.
🧑‍💻 The YAML definition gets automatically adjusted any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is **always managed declaratively in code**, even if you modify your workflows in other ways (UI, CI/CD, Terraform, API calls).
🧑‍💻 The YAML definition gets automatically adjusted any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is **always managed declaratively in code**, even if you modify your workflows in other ways (UI, CI/CD, Terraform, API calls).
<p align="center">

View File

@@ -21,7 +21,7 @@ plugins {
// test
id "com.adarshr.test-logger" version "4.0.0"
id "org.sonarqube" version "6.0.1.5171"
id "org.sonarqube" version "6.1.0.5360"
id 'jacoco-report-aggregation'
// helper
@@ -33,13 +33,13 @@ plugins {
// release
id "io.github.gradle-nexus.publish-plugin" version "2.0.0"
id 'net.researchgate.release' version '3.1.0'
id "com.gorylenko.gradle-git-properties" version "2.4.2"
id "com.gorylenko.gradle-git-properties" version "2.5.0"
id 'signing'
id 'ru.vyarus.pom' version '3.0.0' apply false
id 'ru.vyarus.github-info' version '2.0.0' apply false
// OWASP dependency check
id "org.owasp.dependencycheck" version "12.0.2" apply false
id "org.owasp.dependencycheck" version "12.1.1" apply false
}
idea {
@@ -74,7 +74,7 @@ dependencies {
**********************************************************************************************************************/
allprojects {
if (it.name != 'platform') {
group "io.kestra"
group = "io.kestra"
java {
sourceCompatibility = targetJavaVersion
@@ -121,7 +121,6 @@ allprojects {
micronaut "io.micronaut:micronaut-management"
micronaut "io.micrometer:micrometer-core"
micronaut "io.micronaut.micrometer:micronaut-micrometer-registry-prometheus"
micronaut "io.micronaut.micrometer:micronaut-micrometer-registry-otlp"
micronaut "io.micronaut:micronaut-http-client"
micronaut "io.micronaut.reactor:micronaut-reactor-http-client"
micronaut "io.micronaut.tracing:micronaut-tracing-opentelemetry-http"
@@ -197,6 +196,9 @@ subprojects {
testImplementation 'org.hamcrest:hamcrest'
testImplementation 'org.hamcrest:hamcrest-library'
testImplementation 'org.exparity:hamcrest-date'
//assertj
testImplementation 'org.assertj:assertj-core'
}
test {
@@ -214,8 +216,8 @@ subprojects {
environment 'SECRET_WEBHOOK_KEY', "secretKey".bytes.encodeBase64().toString()
environment 'SECRET_NON_B64_SECRET', "some secret value"
environment 'SECRET_PASSWORD', "cGFzc3dvcmQ="
environment 'KESTRA_TEST1', "true"
environment 'KESTRA_TEST2', "Pass by env"
environment 'ENV_TEST1', "true"
environment 'ENV_TEST2', "Pass by env"
}
testlogger {
@@ -280,7 +282,7 @@ subprojects {
}
dependencies {
agent "org.aspectj:aspectjweaver:1.9.22.1"
agent "org.aspectj:aspectjweaver:1.9.24"
}
test {
@@ -597,11 +599,10 @@ release {
}
// Dynamically set properties with default values
failOnSnapshotDependencies = (project.hasProperty('release.failOnSnapshotDependencies')
? project.property('release.failOnSnapshotDependencies').toBoolean()
: true)
failOnSnapshotDependencies = providers.gradleProperty("release.failOnSnapshotDependencies")
.map(val -> Boolean.parseBoolean(val))
.getOrElse(true)
pushReleaseVersionBranch = (project.hasProperty('release.pushReleaseVersionBranch')
? project.property('release.pushReleaseVersionBranch').toString()
: null)
pushReleaseVersionBranch = providers.gradleProperty("release.pushReleaseVersionBranch")
.getOrElse(null)
}

View File

@@ -12,18 +12,9 @@ dependencies {
implementation 'ch.qos.logback.contrib:logback-json-classic'
implementation 'ch.qos.logback.contrib:logback-jackson'
// plugins
implementation 'org.eclipse.aether:aether-api'
implementation 'org.eclipse.aether:aether-spi'
implementation 'org.eclipse.aether:aether-util'
implementation 'org.eclipse.aether:aether-impl'
implementation 'org.eclipse.aether:aether-connector-basic'
implementation 'org.eclipse.aether:aether-transport-file'
implementation 'org.eclipse.aether:aether-transport-http'
implementation('org.apache.maven:maven-aether-provider') {
// sisu dependency injector is not used
exclude group: 'org.eclipse.sisu'
}
// OTLP metrics
implementation "io.micronaut.micrometer:micronaut-micrometer-registry-otlp"
// aether still use javax.inject
compileOnly 'javax.inject:javax.inject:1'
@@ -43,4 +34,7 @@ dependencies {
implementation project(":storage-local")
implementation project(":webserver")
//test
testImplementation "org.wiremock:wiremock-jetty12"
}

View File

@@ -46,8 +46,18 @@ public abstract class AbstractApiCommand extends AbstractCommand {
@Nullable
private HttpClientConfiguration httpClientConfiguration;
/**
* {@inheritDoc}
*/
protected boolean loadExternalPlugins() {
return false;
}
protected DefaultHttpClient client() throws URISyntaxException {
DefaultHttpClient defaultHttpClient = new DefaultHttpClient(server.toURI(), httpClientConfiguration != null ? httpClientConfiguration : new DefaultHttpClientConfiguration());
DefaultHttpClient defaultHttpClient = DefaultHttpClient.builder()
.uri(server.toURI())
.configuration(httpClientConfiguration != null ? httpClientConfiguration : new DefaultHttpClientConfiguration())
.build();
MessageBodyHandlerRegistry defaultHandlerRegistry = defaultHttpClient.getHandlerRegistry();
if (defaultHandlerRegistry instanceof ContextlessMessageBodyHandlerRegistry modifiableRegistry) {
modifiableRegistry.add(MediaType.TEXT_JSON_TYPE, new NettyJsonHandler<>(JsonMapper.createDefault()));

View File

@@ -4,16 +4,17 @@ import ch.qos.logback.classic.LoggerContext;
import com.google.common.collect.ImmutableMap;
import io.kestra.cli.commands.servers.ServerCommandInterface;
import io.kestra.cli.services.StartupHookInterface;
import io.kestra.core.contexts.KestraContext;
import io.kestra.core.plugins.PluginManager;
import io.kestra.core.plugins.PluginRegistry;
import io.kestra.webserver.services.FlowAutoLoaderService;
import io.micronaut.context.ApplicationContext;
import io.micronaut.context.env.yaml.YamlPropertySourceLoader;
import io.micronaut.core.annotation.Introspected;
import io.micronaut.http.uri.UriBuilder;
import io.micronaut.management.endpoint.EndpointDefaultConfiguration;
import io.micronaut.runtime.server.EmbeddedServer;
import jakarta.inject.Provider;
import lombok.extern.slf4j.Slf4j;
import org.apache.http.client.utils.URIBuilder;
import io.kestra.core.utils.Rethrow;
import picocli.CommandLine;
@@ -26,10 +27,13 @@ import java.nio.file.Paths;
import java.text.MessageFormat;
import java.time.temporal.ChronoUnit;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.Callable;
import jakarta.inject.Inject;
import picocli.CommandLine.Command;
import picocli.CommandLine.Option;
@CommandLine.Command(
@Command(
versionProvider = VersionProvider.class,
mixinStandardHelpOptions = true,
showDefaultValues = true
@@ -49,22 +53,28 @@ abstract public class AbstractCommand implements Callable<Integer> {
@Inject
private io.kestra.core.utils.VersionProvider versionProvider;
@Inject
protected Provider<PluginRegistry> pluginRegistryProvider;
@Inject
protected Provider<PluginManager> pluginManagerProvider;
private PluginRegistry pluginRegistry;
@CommandLine.Option(names = {"-v", "--verbose"}, description = "Change log level. Multiple -v options increase the verbosity.", showDefaultValue = CommandLine.Help.Visibility.NEVER)
@Option(names = {"-v", "--verbose"}, description = "Change log level. Multiple -v options increase the verbosity.", showDefaultValue = CommandLine.Help.Visibility.NEVER)
private boolean[] verbose = new boolean[0];
@CommandLine.Option(names = {"-l", "--log-level"}, description = "Change log level (values: ${COMPLETION-CANDIDATES})")
@Option(names = {"-l", "--log-level"}, description = "Change log level (values: ${COMPLETION-CANDIDATES})")
private LogLevel logLevel = LogLevel.INFO;
@CommandLine.Option(names = {"--internal-log"}, description = "Change also log level for internal log")
@Option(names = {"--internal-log"}, description = "Change also log level for internal log")
private boolean internalLog = false;
@CommandLine.Option(names = {"-c", "--config"}, description = "Path to a configuration file")
@Option(names = {"-c", "--config"}, description = "Path to a configuration file")
private Path config = Paths.get(System.getProperty("user.home"), ".kestra/config.yml");
@CommandLine.Option(names = {"-p", "--plugins"}, description = "Path to plugins directory")
protected Path pluginsPath = System.getenv("KESTRA_PLUGINS_PATH") != null ? Paths.get(System.getenv("KESTRA_PLUGINS_PATH")) : null;
@Option(names = {"-p", "--plugins"}, description = "Path to plugins directory")
protected Path pluginsPath = Optional.ofNullable(System.getenv("KESTRA_PLUGINS_PATH")).map(Paths::get).orElse(null);
public enum LogLevel {
TRACE,
@@ -76,7 +86,7 @@ abstract public class AbstractCommand implements Callable<Integer> {
@Override
public Integer call() throws Exception {
Thread.currentThread().setName(this.getClass().getDeclaredAnnotation(CommandLine.Command.class).name());
Thread.currentThread().setName(this.getClass().getDeclaredAnnotation(Command.class).name());
startLogger();
sendServerLog();
if (this.startupHook != null) {
@@ -84,8 +94,14 @@ abstract public class AbstractCommand implements Callable<Integer> {
}
if (this.pluginsPath != null && loadExternalPlugins()) {
pluginRegistry = pluginRegistry();
pluginRegistry = pluginRegistryProvider.get();
pluginRegistry.registerIfAbsent(pluginsPath);
// PluginManager mus only be initialized if a registry is also instantiated
if (isPluginManagerEnabled()) {
PluginManager manager = pluginManagerProvider.get();
manager.start();
}
}
startWebserver();
@@ -102,8 +118,15 @@ abstract public class AbstractCommand implements Callable<Integer> {
return true;
}
protected PluginRegistry pluginRegistry() {
return KestraContext.getContext().getPluginRegistry(); // Lazy init
/**
* Specifies whether the {@link PluginManager} service must be initialized.
* <p>
* This method can be overridden by concrete commands.
*
* @return {@code true} if the {@link PluginManager} service must be initialized.
*/
protected boolean isPluginManagerEnabled() {
return true;
}
private static String message(String message, Object... format) {
@@ -157,7 +180,6 @@ abstract public class AbstractCommand implements Callable<Integer> {
logger.getName().startsWith("io.kestra") &&
!logger.getName().startsWith("io.kestra.ee.runner.kafka.services"))
)
|| logger.getName().startsWith("flow")
)
.forEach(
logger -> logger.setLevel(ch.qos.logback.classic.Level.valueOf(this.logLevel.name()))
@@ -183,9 +205,9 @@ abstract public class AbstractCommand implements Callable<Integer> {
if (this.endpointConfiguration.getPort().isPresent()) {
URI endpoint = null;
try {
endpoint = new URIBuilder(server.getURL().toURI())
.setPort(this.endpointConfiguration.getPort().get())
.setPath("/health")
endpoint = UriBuilder.of(server.getURL().toURI())
.port(this.endpointConfiguration.getPort().get())
.path("/health")
.build();
} catch (URISyntaxException e) {
e.printStackTrace();
@@ -207,10 +229,12 @@ abstract public class AbstractCommand implements Callable<Integer> {
return false;
}
protected void shutdownHook(Rethrow.RunnableChecked<Exception> run) {
protected void shutdownHook(boolean logShutdown, Rethrow.RunnableChecked<Exception> run) {
Runtime.getRuntime().addShutdownHook(new Thread(
() -> {
log.warn("Receiving shutdown ! Try to graceful exit");
if (logShutdown) {
log.warn("Receiving shutdown ! Try to graceful exit");
}
try {
run.run();
} catch (Exception e) {

View File

@@ -31,6 +31,12 @@ public abstract class AbstractValidateCommand extends AbstractApiCommand {
@CommandLine.Parameters(index = "0", description = "the directory containing files to check")
protected Path directory;
/** {@inheritDoc} **/
@Override
protected boolean loadExternalPlugins() {
return local;
}
public static void handleException(ConstraintViolationException e, String resource) {
stdErr("\t@|fg(red) Unable to parse {0} due to the following error(s):|@", resource);
e.getConstraintViolations()
@@ -68,10 +74,9 @@ public abstract class AbstractValidateCommand extends AbstractApiCommand {
}
}
// bug in micronaut, we can't inject YamlFlowParser & ModelValidator, so we inject from implementation
// bug in micronaut, we can't inject ModelValidator, so we inject from implementation
public Integer call(
Class<?> cls,
YamlParser yamlParser,
ModelValidator modelValidator,
Function<Object, String> identity,
Function<Object, List<String>> warningsFunction,
@@ -88,7 +93,7 @@ public abstract class AbstractValidateCommand extends AbstractApiCommand {
.filter(YamlParser::isValidExtension)
.forEach(path -> {
try {
Object parse = yamlParser.parse(path.toFile(), cls);
Object parse = YamlParser.parse(path.toFile(), cls);
modelValidator.validate(parse);
stdOut("@|green \u2713|@ - " + identity.apply(parse));
List<String> warnings = warningsFunction.apply(parse);

View File

@@ -18,6 +18,8 @@ import picocli.CommandLine;
FlowNamespaceCommand.class,
FlowDotCommand.class,
FlowExportCommand.class,
FlowUpdateCommand.class,
FlowUpdatesCommand.class
}
)
@Slf4j

View File

@@ -29,8 +29,7 @@ public class FlowDotCommand extends AbstractCommand {
public Integer call() throws Exception {
super.call();
YamlParser parser = applicationContext.getBean(YamlParser.class);
Flow flow = parser.parse(file.toFile(), Flow.class);
Flow flow = YamlParser.parse(file.toFile(), Flow.class);
GraphCluster graph = GraphUtils.of(flow, null);

View File

@@ -20,9 +20,6 @@ public class FlowExpandCommand extends AbstractCommand {
@CommandLine.Parameters(index = "0", description = "The flow file to expand")
private Path file;
@Inject
private YamlParser yamlParser;
@Inject
private ModelValidator modelValidator;
@@ -31,7 +28,7 @@ public class FlowExpandCommand extends AbstractCommand {
super.call();
stdErr("Warning, this functionality is deprecated and will be removed at some point.");
String content = IncludeHelperExpander.expand(Files.readString(file), file.getParent());
Flow flow = yamlParser.parse(content, Flow.class);
Flow flow = YamlParser.parse(content, Flow.class);
modelValidator.validate(flow);
stdOut(content);
return 0;

View File

@@ -33,6 +33,9 @@ public class FlowUpdatesCommand extends AbstractApiCommand {
@CommandLine.Option(names = {"--delete"}, negatable = true, description = "Whether missing should be deleted")
public boolean delete = false;
@CommandLine.Option(names = {"--namespace"}, description = "The parent namespace of the flows, if not set, every namespace are allowed.")
public String namespace;
@SuppressWarnings("deprecation")
@Override
public Integer call() throws Exception {
@@ -58,8 +61,12 @@ public class FlowUpdatesCommand extends AbstractApiCommand {
body = String.join("\n---\n", flows);
}
try(DefaultHttpClient client = client()) {
String namespaceQuery = "";
if (namespace != null) {
namespaceQuery = "&namespace=" + namespace;
}
MutableHttpRequest<String> request = HttpRequest
.POST(apiUri("/flows/bulk") + "?delete=" + delete, body).contentType(MediaType.APPLICATION_YAML);
.POST(apiUri("/flows/bulk") + "?allowNamespaceChild=true&delete=" + delete + namespaceQuery, body).contentType(MediaType.APPLICATION_YAML);
List<UpdateResult> updated = client.toBlocking().retrieve(
this.requestOptions(request),
@@ -80,4 +87,9 @@ public class FlowUpdatesCommand extends AbstractApiCommand {
return 0;
}
@Override
protected boolean loadExternalPlugins() {
return false;
}
}

View File

@@ -1,9 +1,8 @@
package io.kestra.cli.commands.flows;
import io.kestra.cli.AbstractValidateCommand;
import io.kestra.core.models.flows.Flow;
import io.kestra.core.models.flows.FlowWithSource;
import io.kestra.core.models.validations.ModelValidator;
import io.kestra.core.serializers.YamlParser;
import io.kestra.core.services.FlowService;
import jakarta.inject.Inject;
import picocli.CommandLine;
@@ -16,8 +15,6 @@ import java.util.List;
description = "Validate a flow"
)
public class FlowValidateCommand extends AbstractValidateCommand {
@Inject
private YamlParser yamlParser;
@Inject
private ModelValidator modelValidator;
@@ -28,23 +25,22 @@ public class FlowValidateCommand extends AbstractValidateCommand {
@Override
public Integer call() throws Exception {
return this.call(
Flow.class,
yamlParser,
FlowWithSource.class,
modelValidator,
(Object object) -> {
Flow flow = (Flow) object;
FlowWithSource flow = (FlowWithSource) object;
return flow.getNamespace() + " / " + flow.getId();
},
(Object object) -> {
Flow flow = (Flow) object;
FlowWithSource flow = (FlowWithSource) object;
List<String> warnings = new ArrayList<>();
warnings.addAll(flowService.deprecationPaths(flow).stream().map(deprecation -> deprecation + " is deprecated").toList());
warnings.addAll(flowService.warnings(flow));
warnings.addAll(flowService.warnings(flow, this.tenantId));
return warnings;
},
(Object object) -> {
Flow flow = (Flow) object;
return flowService.relocations(flow.generateSource()).stream().map(relocation -> relocation.from() + " is replaced by " + relocation.to()).toList();
FlowWithSource flow = (FlowWithSource) object;
return flowService.relocations(flow.sourceOrGenerateIfNull()).stream().map(relocation -> relocation.from() + " is replaced by " + relocation.to()).toList();
}
);
}

View File

@@ -10,7 +10,6 @@ import io.micronaut.http.MediaType;
import io.micronaut.http.MutableHttpRequest;
import io.micronaut.http.client.exceptions.HttpClientResponseException;
import io.micronaut.http.client.netty.DefaultHttpClient;
import jakarta.inject.Inject;
import jakarta.validation.ConstraintViolationException;
import lombok.extern.slf4j.Slf4j;
import picocli.CommandLine;
@@ -27,8 +26,6 @@ import java.util.List;
)
@Slf4j
public class FlowNamespaceUpdateCommand extends AbstractServiceNamespaceUpdateCommand {
@Inject
public YamlParser yamlParser;
@CommandLine.Option(names = {"--override-namespaces"}, negatable = true, description = "Replace namespace of all flows by the one provided")
public boolean override = false;

View File

@@ -1,31 +1,37 @@
package io.kestra.cli.commands.plugins;
import io.micronaut.configuration.picocli.PicocliRunner;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import io.kestra.cli.AbstractCommand;
import io.kestra.cli.App;
import picocli.CommandLine;
import io.micronaut.configuration.picocli.PicocliRunner;
import lombok.SneakyThrows;
import picocli.CommandLine.Command;
@CommandLine.Command(
@Command(
name = "plugins",
description = "Manage plugins",
mixinStandardHelpOptions = true,
subcommands = {
PluginInstallCommand.class,
PluginUninstallCommand.class,
PluginListCommand.class,
PluginDocCommand.class
PluginDocCommand.class,
PluginSearchCommand.class
}
)
@Slf4j
public class PluginCommand extends AbstractCommand {
@SneakyThrows
@Override
public Integer call() throws Exception {
super.call();
PicocliRunner.call(App.class, "plugins", "--help");
PicocliRunner.call(App.class, "plugins", "--help");
return 0;
}
@Override
protected boolean loadExternalPlugins() {
return false;
}
}

View File

@@ -3,6 +3,7 @@ package io.kestra.cli.commands.plugins;
import com.google.common.io.Files;
import io.kestra.cli.AbstractCommand;
import io.kestra.core.docs.DocumentationGenerator;
import io.kestra.core.plugins.PluginRegistry;
import io.kestra.core.plugins.RegisteredPlugin;
import io.kestra.core.serializers.JacksonMapper;
import io.micronaut.context.ApplicationContext;
@@ -42,8 +43,10 @@ public class PluginDocCommand extends AbstractCommand {
super.call();
DocumentationGenerator documentationGenerator = applicationContext.getBean(DocumentationGenerator.class);
List<RegisteredPlugin> plugins = core ? pluginRegistry().plugins() : pluginRegistry().externalPlugins();
PluginRegistry registry = pluginRegistryProvider.get();
List<RegisteredPlugin> plugins = core ? registry.plugins() : registry.externalPlugins();
boolean hasFailures = false;
for (RegisteredPlugin registeredPlugin : plugins) {
try {
documentationGenerator
@@ -100,4 +103,10 @@ public class PluginDocCommand extends AbstractCommand {
return hasFailures ? 1 : 0;
}
/** {@inheritDoc} **/
@Override
protected boolean isPluginManagerEnabled() {
return false;
}
}

View File

@@ -1,98 +1,123 @@
package io.kestra.cli.commands.plugins;
import org.apache.commons.io.FilenameUtils;
import io.kestra.core.contexts.MavenPluginRepositoryConfig;
import io.kestra.core.plugins.LocalPluginManager;
import io.kestra.core.plugins.MavenPluginDownloader;
import io.kestra.core.plugins.PluginArtifact;
import io.kestra.core.plugins.PluginCatalogService;
import io.kestra.core.plugins.PluginManager;
import io.micronaut.http.client.HttpClient;
import io.micronaut.http.client.annotation.Client;
import io.micronaut.http.uri.UriBuilder;
import io.kestra.cli.AbstractCommand;
import io.kestra.cli.plugins.PluginDownloader;
import io.kestra.cli.plugins.RepositoryConfig;
import io.kestra.core.utils.IdUtils;
import org.apache.http.client.utils.URIBuilder;
import jakarta.inject.Provider;
import picocli.CommandLine;
import java.net.URI;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Objects;
import jakarta.inject.Inject;
import picocli.CommandLine.Command;
import picocli.CommandLine.Parameters;
import picocli.CommandLine.Option;
import picocli.CommandLine.Spec;
import static io.kestra.core.utils.Rethrow.throwConsumer;
@CommandLine.Command(
@Command(
name = "install",
description = "Install plugins"
)
public class PluginInstallCommand extends AbstractCommand {
@CommandLine.Parameters(index = "0..*", description = "Plugins to install. Represented as Maven artifact coordinates.")
@Option(names = {"--locally"}, description = "Specifies if plugins must be installed locally. If set to false the installation depends on your Kestra configuration.")
boolean locally = true;
@Option(names = {"--all"}, description = "Install all available plugins")
boolean all = false;
@Parameters(index = "0..*", description = "Plugins to install. Represented as Maven artifact coordinates (i.e., <groupId>:<artifactId>:(<version>|LATEST)")
List<String> dependencies = new ArrayList<>();
@CommandLine.Option(names = {"--repositories"}, description = "URL to additional Maven repositories")
@Option(names = {"--repositories"}, description = "URL to additional Maven repositories")
private URI[] repositories;
@CommandLine.Spec
@Spec
CommandLine.Model.CommandSpec spec;
@Inject
private PluginDownloader pluginDownloader;
Provider<MavenPluginDownloader> mavenPluginRepositoryProvider;
@Inject
@Client("api") HttpClient httpClient;
@Override
public Integer call() throws Exception {
super.call();
if (this.pluginsPath == null) {
if (this.locally && this.pluginsPath == null) {
throw new CommandLine.ParameterException(this.spec.commandLine(), "Missing required options '--plugins' " +
"or environment variable 'KESTRA_PLUGINS_PATH"
);
}
if (!pluginsPath.toFile().exists()) {
if (!pluginsPath.toFile().mkdir()) {
throw new RuntimeException("Cannot create directory: " + pluginsPath.toFile().getAbsolutePath());
}
}
List<MavenPluginRepositoryConfig> repositoryConfigs = List.of();
if (repositories != null) {
Arrays.stream(repositories)
.forEach(throwConsumer(s -> {
URIBuilder uriBuilder = new URIBuilder(s);
RepositoryConfig.RepositoryConfigBuilder builder = RepositoryConfig.builder()
repositoryConfigs = Arrays.stream(repositories)
.map(uri -> {
MavenPluginRepositoryConfig.MavenPluginRepositoryConfigBuilder builder = MavenPluginRepositoryConfig
.builder()
.id(IdUtils.create());
if (uriBuilder.getUserInfo() != null) {
int index = uriBuilder.getUserInfo().indexOf(":");
builder.basicAuth(new RepositoryConfig.BasicAuth(
uriBuilder.getUserInfo().substring(0, index),
uriBuilder.getUserInfo().substring(index + 1)
String userInfo = uri.getUserInfo();
if (userInfo != null) {
String[] userInfoParts = userInfo.split(":");
builder = builder.basicAuth(new MavenPluginRepositoryConfig.BasicAuth(
userInfoParts[0],
userInfoParts[1]
));
uriBuilder.setUserInfo(null);
}
builder.url(uriBuilder.build().toString());
pluginDownloader.addRepository(builder.build());
}));
builder.url(UriBuilder.of(uri).userInfo(null).build().toString());
return builder.build();
}).toList();
}
List<URL> resolveUrl = pluginDownloader.resolve(dependencies);
stdOut("Resolved Plugin(s) with {0}", resolveUrl);
if (all) {
PluginCatalogService service = new PluginCatalogService(httpClient, false, true);
dependencies = service.get().stream().map(Objects::toString).toList();
}
for (URL url: resolveUrl) {
Files.copy(
Paths.get(url.toURI()),
Paths.get(pluginsPath.toString(), FilenameUtils.getName(url.toString())),
StandardCopyOption.REPLACE_EXISTING
if (dependencies.isEmpty()) {
stdErr("Error: No plugin to install.");
return CommandLine.ExitCode.OK;
}
final List<PluginArtifact> pluginArtifacts;
try {
pluginArtifacts = dependencies.stream().map(PluginArtifact::fromCoordinates).toList();
} catch (IllegalArgumentException e) {
stdErr(e.getMessage());
return CommandLine.ExitCode.USAGE;
}
try (final PluginManager pluginManager = getPluginManager()) {
List<PluginArtifact> installed = pluginManager.install(
pluginArtifacts,
repositoryConfigs,
false,
pluginsPath
);
List<URI> uris = installed.stream().map(PluginArtifact::uri).toList();
stdOut("Successfully installed plugins {0} into {1}", dependencies, uris);
return CommandLine.ExitCode.OK;
}
}
stdOut("Successfully installed plugins {0} into {1}", dependencies, pluginsPath);
return 0;
private PluginManager getPluginManager() {
return locally ? new LocalPluginManager(mavenPluginRepositoryProvider.get()) : this.pluginManagerProvider.get();
}
@Override

View File

@@ -1,22 +1,31 @@
package io.kestra.cli.commands.plugins;
import io.kestra.cli.AbstractCommand;
import io.kestra.core.plugins.PluginRegistry;
import io.kestra.core.plugins.RegisteredPlugin;
import jakarta.inject.Inject;
import jakarta.inject.Provider;
import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Option;
import picocli.CommandLine.Spec;
import java.util.List;
@CommandLine.Command(
@Command(
name = "list",
description = "List all plugins already installed"
)
public class PluginListCommand extends AbstractCommand {
@CommandLine.Spec
@Spec
CommandLine.Model.CommandSpec spec;
@CommandLine.Option(names = {"--core"}, description = "Also write core tasks plugins")
@Option(names = {"--core"}, description = "Also write core tasks plugins")
private boolean core = false;
@Inject
private PluginRegistry registry;
@Override
public Integer call() throws Exception {
super.call();
@@ -27,7 +36,8 @@ public class PluginListCommand extends AbstractCommand {
);
}
List<RegisteredPlugin> plugins = core ? pluginRegistry().plugins() : pluginRegistry().externalPlugins();
List<RegisteredPlugin> plugins = core ? registry.plugins() : registry.externalPlugins();
plugins.forEach(registeredPlugin -> stdOut(registeredPlugin.toString()));
return 0;

View File

@@ -0,0 +1,149 @@
package io.kestra.cli.commands.plugins;
import io.kestra.cli.AbstractCommand;
import io.micronaut.core.type.Argument;
import io.micronaut.http.HttpRequest;
import io.micronaut.http.client.HttpClient;
import io.micronaut.http.client.annotation.Client;
import jakarta.inject.Inject;
import picocli.CommandLine.Command;
import picocli.CommandLine.Parameters;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.ArrayList;
import java.util.List;
@Command(
name = "search",
description = "Search for available Kestra plugins"
)
public class PluginSearchCommand extends AbstractCommand {
@Inject
@Client("api")
private HttpClient httpClient;
private static final ObjectMapper MAPPER = new ObjectMapper();
private static final char SPACE = ' ';
@Parameters(index = "0", description = "Search term (optional)", defaultValue = "")
private String searchTerm;
@Override
public Integer call() throws Exception {
super.call();
try {
JsonNode root = fetchPlugins();
List<PluginInfo> plugins = findPlugins(root);
printResults(plugins);
return 0;
} catch (Exception e) {
stdOut("Error processing plugins: {0}", e.getMessage());
return 1;
}
}
private JsonNode fetchPlugins() throws Exception {
String response = httpClient.toBlocking()
.retrieve(
HttpRequest.GET("/v1/plugins")
.header("Accept", "application/json")
);
return MAPPER.readTree(response);
}
private List<PluginInfo> findPlugins(JsonNode root) {
String searchTermLower = searchTerm.toLowerCase();
List<PluginInfo> plugins = new ArrayList<>();
for (JsonNode plugin : root) {
if (matchesSearch(plugin, searchTermLower)) {
plugins.add(new PluginInfo(
plugin.path("name").asText(),
plugin.path("title").asText(),
plugin.path("group").asText(),
plugin.path("version").asText("")
));
}
}
plugins.sort((p1, p2) -> p1.name.compareToIgnoreCase(p2.name));
return plugins;
}
private boolean matchesSearch(JsonNode plugin, String term) {
if (term.isEmpty()) {
return true;
}
return plugin.path("name").asText().toLowerCase().contains(term) ||
plugin.path("title").asText().toLowerCase().contains(term) ||
plugin.path("group").asText().toLowerCase().contains(term);
}
private void printResults(List<PluginInfo> plugins) {
if (plugins.isEmpty()) {
stdOut("No plugins found{0}",
searchTerm.isEmpty() ? "" : " matching '" + searchTerm + "'");
return;
}
stdOut("\nFound {0} plugins{1}",
plugins.size(),
searchTerm.isEmpty() ? "" : " matching '" + searchTerm + "'"
);
printPluginsTable(plugins);
}
private void printPluginsTable(List<PluginInfo> plugins) {
int maxName = 4, maxTitle = 5, maxGroup = 5;
for (PluginInfo plugin : plugins) {
maxName = Math.max(maxName, plugin.name.length());
maxTitle = Math.max(maxTitle, plugin.title.length());
maxGroup = Math.max(maxGroup, plugin.group.length());
}
StringBuilder namePad = new StringBuilder(maxName);
StringBuilder titlePad = new StringBuilder(maxTitle);
StringBuilder groupPad = new StringBuilder(maxGroup);
stdOut("");
printRow(namePad, titlePad, groupPad, "NAME", "TITLE", "GROUP", "VERSION",
maxName, maxTitle, maxGroup);
for (PluginInfo plugin : plugins) {
printRow(namePad, titlePad, groupPad, plugin.name, plugin.title, plugin.group, plugin.version,
maxName, maxTitle, maxGroup);
}
stdOut("");
}
private void printRow(StringBuilder namePad, StringBuilder titlePad, StringBuilder groupPad,
String name, String title, String group, String version,
int maxName, int maxTitle, int maxGroup) {
stdOut("{0} {1} {2} {3}",
pad(namePad, name, maxName),
pad(titlePad, title, maxTitle),
pad(groupPad, group, maxGroup),
version
);
}
private String pad(StringBuilder sb, String str, int length) {
sb.setLength(0);
sb.append(str);
while (sb.length() < length) {
sb.append(SPACE);
}
return sb.toString();
}
private record PluginInfo(String name, String title, String group, String version) {}
@Override
protected boolean loadExternalPlugins() {
return false;
}
}

View File

@@ -0,0 +1,69 @@
package io.kestra.cli.commands.plugins;
import io.kestra.cli.AbstractCommand;
import io.kestra.core.plugins.LocalPluginManager;
import io.kestra.core.plugins.MavenPluginDownloader;
import io.kestra.core.plugins.PluginArtifact;
import io.kestra.core.plugins.PluginManager;
import jakarta.inject.Inject;
import jakarta.inject.Provider;
import picocli.CommandLine;
import picocli.CommandLine.Parameters;
import picocli.CommandLine.Spec;
import java.net.URI;
import java.util.ArrayList;
import java.util.List;
@CommandLine.Command(
name = "uninstall",
description = "Uninstall plugins"
)
public class PluginUninstallCommand extends AbstractCommand {
@Parameters(index = "0..*", description = "The plugins to uninstall. Represented as Maven artifact coordinates (i.e., <groupId>:<artifactId>:(<version>|LATEST)")
List<String> dependencies = new ArrayList<>();
@Spec
CommandLine.Model.CommandSpec spec;
@Inject
Provider<MavenPluginDownloader> mavenPluginRepositoryProvider;
@Override
public Integer call() throws Exception {
super.call();
List<PluginArtifact> pluginArtifacts;
try {
pluginArtifacts = dependencies.stream().map(PluginArtifact::fromCoordinates).toList();
} catch (IllegalArgumentException e) {
stdErr(e.getMessage());
return CommandLine.ExitCode.USAGE;
}
final PluginManager pluginManager;
// If a PLUGIN_PATH is provided, then use the LocalPluginManager
if (pluginsPath != null) {
pluginManager = new LocalPluginManager(mavenPluginRepositoryProvider.get());
} else {
// Otherwise, we delegate to the configured plugin-manager.
pluginManager = this.pluginManagerProvider.get();
}
List<PluginArtifact> uninstalled = pluginManager.uninstall(
pluginArtifacts,
false,
pluginsPath
);
List<URI> uris = uninstalled.stream().map(PluginArtifact::uri).toList();
stdOut("Successfully uninstalled plugins {0} from {1}", dependencies, uris);
return CommandLine.ExitCode.OK;
}
@Override
protected boolean loadExternalPlugins() {
return false;
}
}

View File

@@ -1,12 +1,20 @@
package io.kestra.cli.commands.servers;
import io.kestra.cli.AbstractCommand;
import io.kestra.core.contexts.KestraContext;
import jakarta.annotation.PostConstruct;
import picocli.CommandLine;
abstract public class AbstractServerCommand extends AbstractCommand implements ServerCommandInterface {
@CommandLine.Option(names = {"--port"}, description = "The port to bind")
Integer serverPort;
@Override
public Integer call() throws Exception {
this.shutdownHook(true, () -> KestraContext.getContext().shutdown());
return super.call();
}
protected static int defaultWorkerThread() {
return Runtime.getRuntime().availableProcessors() * 4;
}

View File

@@ -1,7 +1,6 @@
package io.kestra.cli.commands.servers;
import com.google.common.collect.ImmutableMap;
import io.kestra.core.contexts.KestraContext;
import io.kestra.core.models.ServerType;
import io.kestra.core.runners.ExecutorInterface;
import io.kestra.core.services.SkipExecutionService;
@@ -9,7 +8,6 @@ import io.kestra.core.services.StartExecutorService;
import io.kestra.core.utils.Await;
import io.micronaut.context.ApplicationContext;
import jakarta.inject.Inject;
import lombok.extern.slf4j.Slf4j;
import picocli.CommandLine;
import java.util.Collections;
@@ -20,7 +18,6 @@ import java.util.Map;
name = "executor",
description = "Start the Kestra executor"
)
@Slf4j
public class ExecutorCommand extends AbstractServerCommand {
@Inject
private ApplicationContext applicationContext;
@@ -66,13 +63,10 @@ public class ExecutorCommand extends AbstractServerCommand {
this.startExecutorService.applyOptions(startExecutors, notStartExecutors);
super.call();
this.shutdownHook(() -> KestraContext.getContext().shutdown());
ExecutorInterface executorService = applicationContext.getBean(ExecutorInterface.class);
executorService.run();
log.info("Executor started");
Await.until(() -> !this.applicationContext.isRunning());
return 0;

View File

@@ -1,13 +1,11 @@
package io.kestra.cli.commands.servers;
import com.google.common.collect.ImmutableMap;
import io.kestra.core.contexts.KestraContext;
import io.kestra.core.models.ServerType;
import io.kestra.core.runners.IndexerInterface;
import io.kestra.core.utils.Await;
import io.micronaut.context.ApplicationContext;
import jakarta.inject.Inject;
import lombok.extern.slf4j.Slf4j;
import picocli.CommandLine;
import java.util.Map;
@@ -16,7 +14,6 @@ import java.util.Map;
name = "indexer",
description = "Start the Kestra indexer"
)
@Slf4j
public class IndexerCommand extends AbstractServerCommand {
@Inject
private ApplicationContext applicationContext;
@@ -31,13 +28,10 @@ public class IndexerCommand extends AbstractServerCommand {
@Override
public Integer call() throws Exception {
super.call();
this.shutdownHook(() -> KestraContext.getContext().shutdown());
IndexerInterface indexer = applicationContext.getBean(IndexerInterface.class);
indexer.run();
log.info("Indexer started");
Await.until(() -> !this.applicationContext.isRunning());
return 0;

View File

@@ -1,7 +1,6 @@
package io.kestra.cli.commands.servers;
import com.google.common.collect.ImmutableMap;
import io.kestra.core.contexts.KestraContext;
import io.kestra.core.models.ServerType;
import io.kestra.core.schedulers.AbstractScheduler;
import io.kestra.core.utils.Await;
@@ -31,12 +30,10 @@ public class SchedulerCommand extends AbstractServerCommand {
@Override
public Integer call() throws Exception {
super.call();
this.shutdownHook(() -> KestraContext.getContext().shutdown());
AbstractScheduler scheduler = applicationContext.getBean(AbstractScheduler.class);
scheduler.run();
log.info("Scheduler started");
Await.until(() -> !this.applicationContext.isRunning());
return 0;

View File

@@ -12,7 +12,6 @@ import io.kestra.core.utils.Await;
import io.micronaut.context.ApplicationContext;
import jakarta.annotation.Nullable;
import jakarta.inject.Inject;
import lombok.extern.slf4j.Slf4j;
import picocli.CommandLine;
import java.io.File;
@@ -25,7 +24,6 @@ import java.util.Map;
name = "standalone",
description = "Start the standalone all-in-one server"
)
@Slf4j
public class StandAloneCommand extends AbstractServerCommand {
@CommandLine.Spec
CommandLine.Model.CommandSpec spec;
@@ -91,11 +89,11 @@ public class StandAloneCommand extends AbstractServerCommand {
this.skipExecutionService.setSkipFlows(skipFlows);
this.skipExecutionService.setSkipNamespaces(skipNamespaces);
this.skipExecutionService.setSkipTenants(skipTenants);
this.startExecutorService.applyOptions(startExecutors, notStartExecutors);
KestraContext.getContext().injectWorkerConfigs(workerThread, null);
super.call();
this.shutdownHook(() -> KestraContext.getContext().shutdown());
if (flowPath != null) {
try {
@@ -124,8 +122,6 @@ public class StandAloneCommand extends AbstractServerCommand {
fileWatcher.startListeningFromConfig();
}
this.shutdownHook(standAloneRunner::close);
Await.until(() -> !this.applicationContext.isRunning());
return 0;

View File

@@ -1,9 +1,7 @@
package io.kestra.cli.commands.servers;
import com.google.common.collect.ImmutableMap;
import io.kestra.core.contexts.KestraContext;
import io.kestra.core.models.ServerType;
import io.kestra.core.runners.ExecutorInterface;
import io.kestra.core.runners.IndexerInterface;
import io.kestra.core.utils.Await;
import io.kestra.core.utils.ExecutorsUtils;
@@ -57,20 +55,11 @@ public class WebServerCommand extends AbstractServerCommand {
log.info("Starting an embedded indexer, this can be disabled by using `--no-indexer`.");
poolExecutor = executorsUtils.cachedThreadPool("webserver-indexer");
poolExecutor.execute(applicationContext.getBean(IndexerInterface.class));
shutdownHook(false, () -> poolExecutor.shutdown());
}
log.info("Webserver started");
this.shutdownHook(() -> {
this.close();
KestraContext.getContext().shutdown();
});
Await.until(() -> !this.applicationContext.isRunning());
return 0;
}
private void close() {
if (this.poolExecutor != null) {
this.poolExecutor.shutdown();
}
}
}

View File

@@ -7,7 +7,6 @@ import io.kestra.core.runners.Worker;
import io.kestra.core.utils.Await;
import io.micronaut.context.ApplicationContext;
import jakarta.inject.Inject;
import lombok.extern.slf4j.Slf4j;
import picocli.CommandLine;
import picocli.CommandLine.Option;
@@ -18,7 +17,6 @@ import java.util.UUID;
name = "worker",
description = "Start the Kestra worker"
)
@Slf4j
public class WorkerCommand extends AbstractServerCommand {
@Inject
@@ -39,8 +37,11 @@ public class WorkerCommand extends AbstractServerCommand {
@Override
public Integer call() throws Exception {
KestraContext.getContext().injectWorkerConfigs(thread, workerGroupKey);
super.call();
this.shutdownHook(() -> KestraContext.getContext().shutdown());
if (this.workerGroupKey != null && !this.workerGroupKey.matches("[a-zA-Z0-9_-]+")) {
throw new IllegalArgumentException("The --worker-group option must match the [a-zA-Z0-9_-]+ pattern");
}
@@ -52,13 +53,6 @@ public class WorkerCommand extends AbstractServerCommand {
worker.run();
if (this.workerGroupKey != null) {
log.info("Worker started with {} thread(s) in group '{}'", this.thread, this.workerGroupKey);
}
else {
log.info("Worker started with {} thread(s)", this.thread);
}
Await.until(() -> !this.applicationContext.isRunning());
return 0;

View File

@@ -2,6 +2,7 @@ package io.kestra.cli.commands.sys;
import io.kestra.cli.AbstractCommand;
import io.kestra.core.models.flows.Flow;
import io.kestra.core.models.flows.GenericFlow;
import io.kestra.core.repositories.FlowRepositoryInterface;
import io.micronaut.context.ApplicationContext;
import jakarta.inject.Inject;
@@ -9,6 +10,7 @@ import lombok.extern.slf4j.Slf4j;
import picocli.CommandLine;
import java.util.List;
import java.util.Objects;
@CommandLine.Command(
name = "reindex",
@@ -33,8 +35,8 @@ public class ReindexCommand extends AbstractCommand {
List<Flow> allFlow = flowRepository.findAllForAllTenants();
allFlow.stream()
.map(flow -> flowRepository.findByIdWithSource(flow.getTenantId(), flow.getNamespace(), flow.getId()).orElse(null))
.filter(flow -> flow != null)
.forEach(flow -> flowRepository.update(flow.toFlow(), flow.toFlow(), flow.getSource(), flow.toFlow()));
.filter(Objects::nonNull)
.forEach(flow -> flowRepository.update(GenericFlow.of(flow), flow));
stdOut("Successfully reindex " + allFlow.size() + " flow(s).");
}

View File

@@ -4,7 +4,6 @@ import io.kestra.cli.AbstractValidateCommand;
import io.kestra.core.models.templates.Template;
import io.kestra.core.models.templates.TemplateEnabled;
import io.kestra.core.models.validations.ModelValidator;
import io.kestra.core.serializers.YamlParser;
import jakarta.inject.Inject;
import picocli.CommandLine;
@@ -16,8 +15,6 @@ import java.util.Collections;
)
@TemplateEnabled
public class TemplateValidateCommand extends AbstractValidateCommand {
@Inject
private YamlParser yamlParser;
@Inject
private ModelValidator modelValidator;
@@ -26,7 +23,6 @@ public class TemplateValidateCommand extends AbstractValidateCommand {
public Integer call() throws Exception {
return this.call(
Template.class,
yamlParser,
modelValidator,
(Object object) -> {
Template template = (Template) object;

View File

@@ -10,7 +10,6 @@ import io.micronaut.http.HttpRequest;
import io.micronaut.http.MutableHttpRequest;
import io.micronaut.http.client.exceptions.HttpClientResponseException;
import io.micronaut.http.client.netty.DefaultHttpClient;
import jakarta.inject.Inject;
import lombok.extern.slf4j.Slf4j;
import picocli.CommandLine;
@@ -27,8 +26,6 @@ import jakarta.validation.ConstraintViolationException;
@Slf4j
@TemplateEnabled
public class TemplateNamespaceUpdateCommand extends AbstractServiceNamespaceUpdateCommand {
@Inject
public YamlParser yamlParser;
@Override
public Integer call() throws Exception {
@@ -38,7 +35,7 @@ public class TemplateNamespaceUpdateCommand extends AbstractServiceNamespaceUpda
List<Template> templates = files
.filter(Files::isRegularFile)
.filter(YamlParser::isValidExtension)
.map(path -> yamlParser.parse(path.toFile(), Template.class))
.map(path -> YamlParser.parse(path.toFile(), Template.class))
.toList();
if (templates.isEmpty()) {

View File

@@ -1,153 +0,0 @@
package io.kestra.cli.plugins;
import com.google.common.collect.ImmutableList;
import io.micronaut.context.annotation.Value;
import io.micronaut.core.annotation.Nullable;
import jakarta.inject.Inject;
import jakarta.inject.Singleton;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.FileUtils;
import org.apache.maven.repository.internal.MavenRepositorySystemUtils;
import org.eclipse.aether.DefaultRepositorySystemSession;
import org.eclipse.aether.RepositorySystem;
import org.eclipse.aether.RepositorySystemSession;
import org.eclipse.aether.artifact.Artifact;
import org.eclipse.aether.artifact.DefaultArtifact;
import org.eclipse.aether.connector.basic.BasicRepositoryConnectorFactory;
import org.eclipse.aether.impl.DefaultServiceLocator;
import org.eclipse.aether.repository.LocalRepository;
import org.eclipse.aether.repository.RemoteRepository;
import org.eclipse.aether.resolution.*;
import org.eclipse.aether.spi.connector.RepositoryConnectorFactory;
import org.eclipse.aether.spi.connector.transport.TransporterFactory;
import org.eclipse.aether.transport.file.FileTransporterFactory;
import org.eclipse.aether.transport.http.HttpTransporterFactory;
import org.eclipse.aether.util.repository.AuthenticationBuilder;
import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
@Singleton
@Slf4j
public class PluginDownloader {
private final List<RepositoryConfig> repositoryConfigs;
private final RepositorySystem system;
private final RepositorySystemSession session;
@Inject
public PluginDownloader(
List<RepositoryConfig> repositoryConfigs,
@Nullable @Value("${kestra.plugins.local-repository-path}") String localRepositoryPath
) {
this.repositoryConfigs = repositoryConfigs;
this.system = repositorySystem();
this.session = repositorySystemSession(system, localRepositoryPath);
}
public void addRepository(RepositoryConfig repositoryConfig) {
this.repositoryConfigs.add(repositoryConfig);
}
public List<URL> resolve(List<String> dependencies) throws MalformedURLException, ArtifactResolutionException, VersionRangeResolutionException {
List<RemoteRepository> repositories = remoteRepositories();
List<ArtifactResult> artifactResults = resolveArtifacts(repositories, dependencies);
List<URL> localUrls = resolveUrls(artifactResults);
log.debug("Resolved Plugin {} with {}", dependencies, localUrls);
return localUrls;
}
private List<RemoteRepository> remoteRepositories() {
return repositoryConfigs
.stream()
.map(repositoryConfig -> {
var build = new RemoteRepository.Builder(
repositoryConfig.getId(),
"default",
repositoryConfig.getUrl()
);
if (repositoryConfig.getBasicAuth() != null) {
var authenticationBuilder = new AuthenticationBuilder();
authenticationBuilder.addUsername(repositoryConfig.getBasicAuth().getUsername());
authenticationBuilder.addPassword(repositoryConfig.getBasicAuth().getPassword());
build.setAuthentication(authenticationBuilder.build());
}
return build.build();
})
.toList();
}
private static RepositorySystem repositorySystem() {
DefaultServiceLocator locator = MavenRepositorySystemUtils.newServiceLocator();
locator.addService(RepositoryConnectorFactory.class, BasicRepositoryConnectorFactory.class);
locator.addService(TransporterFactory.class, FileTransporterFactory.class);
locator.addService(TransporterFactory.class, HttpTransporterFactory.class);
return locator.getService(RepositorySystem.class);
}
private RepositorySystemSession repositorySystemSession(RepositorySystem system, String localRepositoryPath) {
DefaultRepositorySystemSession session = MavenRepositorySystemUtils.newSession();
if (localRepositoryPath == null) {
try {
final String tempDirectory = Files.createTempDirectory(this.getClass().getSimpleName().toLowerCase())
.toAbsolutePath()
.toString();
localRepositoryPath = tempDirectory;
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
try {
FileUtils.deleteDirectory(new File(tempDirectory));
} catch (IOException e) {
throw new RuntimeException(e);
}
}));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
LocalRepository localRepo = new LocalRepository(localRepositoryPath);
session.setLocalRepositoryManager(system.newLocalRepositoryManager(session, localRepo));
return session;
}
private List<ArtifactResult> resolveArtifacts(List<RemoteRepository> repositories, List<String> dependencies) throws ArtifactResolutionException, VersionRangeResolutionException {
List<ArtifactResult> results = new ArrayList<>(dependencies.size());
for (String dependency: dependencies) {
var artifact = new DefaultArtifact(dependency);
var version = system.resolveVersionRange(session, new VersionRangeRequest(artifact, repositories, null));
var artifactRequest = new ArtifactRequest(
new DefaultArtifact(artifact.getGroupId(), artifact.getArtifactId(), "jar", version.getHighestVersion().toString()),
repositories,
null
);
var artifactResult = system.resolveArtifact(session, artifactRequest);
results.add(artifactResult);
}
return results;
}
private List<URL> resolveUrls(List<ArtifactResult> artifactResults) throws MalformedURLException {
ImmutableList.Builder<URL> urls = ImmutableList.builder();
for (ArtifactResult artifactResult : artifactResults) {
URL url;
url = artifactResult.getArtifact().getFile().toPath().toUri().toURL();
urls.add(url);
}
return urls.build();
}
}

View File

@@ -1,30 +0,0 @@
package io.kestra.cli.plugins;
import io.micronaut.context.annotation.EachProperty;
import io.micronaut.context.annotation.Parameter;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Getter;
@EachProperty("kestra.plugins.repositories")
@Getter
@AllArgsConstructor
@Builder
public class RepositoryConfig {
String id;
String url;
BasicAuth basicAuth;
@Getter
@AllArgsConstructor
public static class BasicAuth {
private String username;
private String password;
}
public RepositoryConfig(@Parameter String id) {
this.id = id;
}
}

View File

@@ -1,22 +1,23 @@
package io.kestra.cli.services;
import io.kestra.core.models.flows.Flow;
import io.kestra.core.exceptions.FlowProcessingException;
import io.kestra.core.models.flows.FlowInterface;
import io.kestra.core.models.flows.FlowWithPath;
import io.kestra.core.models.flows.FlowWithSource;
import io.kestra.core.models.flows.GenericFlow;
import io.kestra.core.models.validations.ModelValidator;
import io.kestra.core.repositories.FlowRepositoryInterface;
import io.kestra.core.serializers.YamlParser;
import io.kestra.core.services.FlowListenersInterface;
import io.kestra.core.services.PluginDefaultService;
import io.micronaut.context.annotation.Requires;
import io.micronaut.context.annotation.Value;
import io.micronaut.scheduling.io.watch.FileWatchConfiguration;
import jakarta.inject.Inject;
import jakarta.annotation.Nullable;
import jakarta.inject.Singleton;
import jakarta.validation.ConstraintViolationException;
import lombok.extern.slf4j.Slf4j;
import javax.annotation.Nullable;
import java.io.IOException;
import java.nio.charset.Charset;
import java.nio.file.*;
@@ -40,9 +41,6 @@ public class FileChangedEventListener {
@Inject
private PluginDefaultService pluginDefaultService;
@Inject
private YamlParser yamlParser;
@Inject
private ModelValidator modelValidator;
@@ -59,7 +57,6 @@ public class FileChangedEventListener {
private boolean isStarted = false;
@Inject
public FileChangedEventListener(@Nullable FileWatchConfiguration fileWatchConfiguration, @Nullable WatchService watchService) {
this.fileWatchConfiguration = fileWatchConfiguration;
@@ -68,7 +65,7 @@ public class FileChangedEventListener {
public void startListeningFromConfig() throws IOException, InterruptedException {
if (fileWatchConfiguration != null && fileWatchConfiguration.isEnabled()) {
this.flowFilesManager = new LocalFlowFileWatcher(flowRepositoryInterface, pluginDefaultService);
this.flowFilesManager = new LocalFlowFileWatcher(flowRepositoryInterface);
List<Path> paths = fileWatchConfiguration.getPaths();
this.setup(paths);
@@ -76,7 +73,7 @@ public class FileChangedEventListener {
// Init existing flows not already in files
flowListeners.listen(flows -> {
if (!isStarted) {
for (FlowWithSource flow : flows) {
for (FlowInterface flow : flows) {
if (this.flows.stream().noneMatch(flowWithPath -> flowWithPath.uidWithoutRevision().equals(flow.uidWithoutRevision()))) {
flowToFile(flow, this.buildPath(flow));
this.flows.add(FlowWithPath.of(flow, this.buildPath(flow).toString()));
@@ -137,7 +134,7 @@ public class FileChangedEventListener {
try {
String content = Files.readString(filePath, Charset.defaultCharset());
Optional<Flow> flow = parseFlow(content, entry);
Optional<FlowWithSource> flow = parseFlow(content, entry);
if (flow.isPresent()) {
if (kind == StandardWatchEventKinds.ENTRY_MODIFY) {
// Check if we already have a file with the given path
@@ -156,7 +153,7 @@ public class FileChangedEventListener {
flows.add(FlowWithPath.of(flow.get(), filePath.toString()));
}
flowFilesManager.createOrUpdateFlow(flow.get(), content);
flowFilesManager.createOrUpdateFlow(GenericFlow.fromYaml(tenantId, content));
log.info("Flow {} from file {} has been created or modified", flow.get().getId(), entry);
}
@@ -207,11 +204,11 @@ public class FileChangedEventListener {
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
if (file.toString().endsWith(".yml") || file.toString().endsWith(".yaml")) {
String content = Files.readString(file, Charset.defaultCharset());
Optional<Flow> flow = parseFlow(content, file);
Optional<FlowWithSource> flow = parseFlow(content, file);
if (flow.isPresent() && flows.stream().noneMatch(flowWithPath -> flowWithPath.uidWithoutRevision().equals(flow.get().uidWithoutRevision()))) {
flows.add(FlowWithPath.of(flow.get(), file.toString()));
flowFilesManager.createOrUpdateFlow(flow.get(), content);
flowFilesManager.createOrUpdateFlow(GenericFlow.fromYaml(tenantId, content));
}
}
return FileVisitResult.CONTINUE;
@@ -223,27 +220,25 @@ public class FileChangedEventListener {
}
}
private void flowToFile(FlowWithSource flow, Path path) {
private void flowToFile(FlowInterface flow, Path path) {
Path defaultPath = path != null ? path : this.buildPath(flow);
try {
Files.writeString(defaultPath, flow.getSource());
Files.writeString(defaultPath, flow.source());
log.info("Flow {} has been written to file {}", flow.getId(), defaultPath);
} catch (IOException e) {
log.error("Error writing file: {}", defaultPath, e);
}
}
private Optional<Flow> parseFlow(String content, Path entry) {
private Optional<FlowWithSource> parseFlow(String content, Path entry) {
try {
Flow flow = yamlParser.parse(content, Flow.class);
FlowWithSource withPluginDefault = pluginDefaultService.injectDefaults(FlowWithSource.of(flow, content));
modelValidator.validate(withPluginDefault);
FlowWithSource flow = pluginDefaultService.parseFlowWithAllDefaults(tenantId, content, false);
modelValidator.validate(flow);
return Optional.of(flow);
} catch (ConstraintViolationException e) {
} catch (ConstraintViolationException | FlowProcessingException e) {
log.warn("Error while parsing flow: {}", entry, e);
}
return Optional.empty();
}
@@ -259,7 +254,7 @@ public class FileChangedEventListener {
}
}
private Path buildPath(Flow flow) {
private Path buildPath(FlowInterface flow) {
return fileWatchConfiguration.getPaths().getFirst().resolve(flow.uidWithoutRevision() + ".yml");
}
}

View File

@@ -1,11 +1,11 @@
package io.kestra.cli.services;
import io.kestra.core.models.flows.Flow;
import io.kestra.core.models.flows.FlowWithSource;
import io.kestra.core.models.flows.GenericFlow;
public interface FlowFilesManager {
FlowWithSource createOrUpdateFlow(Flow flow, String content);
FlowWithSource createOrUpdateFlow(GenericFlow flow);
void deleteFlow(FlowWithSource toDelete);

View File

@@ -1,27 +1,23 @@
package io.kestra.cli.services;
import io.kestra.core.models.flows.Flow;
import io.kestra.core.models.flows.FlowWithSource;
import io.kestra.core.models.flows.GenericFlow;
import io.kestra.core.repositories.FlowRepositoryInterface;
import io.kestra.core.services.PluginDefaultService;
import lombok.extern.slf4j.Slf4j;
@Slf4j
public class LocalFlowFileWatcher implements FlowFilesManager {
private final FlowRepositoryInterface flowRepository;
private final PluginDefaultService pluginDefaultService;
public LocalFlowFileWatcher(FlowRepositoryInterface flowRepository, PluginDefaultService pluginDefaultService) {
public LocalFlowFileWatcher(FlowRepositoryInterface flowRepository) {
this.flowRepository = flowRepository;
this.pluginDefaultService = pluginDefaultService;
}
@Override
public FlowWithSource createOrUpdateFlow(Flow flow, String content) {
FlowWithSource withDefault = pluginDefaultService.injectDefaults(FlowWithSource.of(flow, content));
public FlowWithSource createOrUpdateFlow(final GenericFlow flow) {
return flowRepository.findById(null, flow.getNamespace(), flow.getId())
.map(previous -> flowRepository.update(flow, previous, content, withDefault))
.orElseGet(() -> flowRepository.create(flow, content, withDefault));
.map(previous -> flowRepository.update(flow, previous))
.orElseGet(() -> flowRepository.create(flow));
}
@Override

View File

@@ -1,6 +1,9 @@
micronaut:
application:
name: kestra
# Disable Micronaut Open Telemetry
otel:
enabled: false
router:
static-resources:
swagger:
@@ -165,7 +168,7 @@ kestra:
values:
recoverMissedSchedules: ALL
variables:
env-vars-prefix: KESTRA_
env-vars-prefix: ENV_
cache-enabled: true
cache-size: 1000

View File

@@ -13,8 +13,7 @@ import picocli.CommandLine;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.*;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertTrue;
class AppTest {
@@ -26,7 +25,7 @@ class AppTest {
try (ApplicationContext ctx = ApplicationContext.run(Environment.CLI, Environment.TEST)) {
PicocliRunner.call(App.class, ctx, "--help");
assertThat(out.toString(), containsString("kestra"));
assertThat(out.toString()).contains("kestra");
}
}
@@ -42,7 +41,7 @@ class AppTest {
new CommandLine(App.class, new MicronautFactory(ctx)).execute(args);
assertTrue(ctx.getProperty("kestra.server-type", ServerType.class).isEmpty());
assertThat(out.toString(), startsWith("Usage: kestra server " + serverType));
assertThat(out.toString()).startsWith("Usage: kestra server " + serverType);
}
}
@@ -56,9 +55,9 @@ class AppTest {
try (ApplicationContext ctx = App.applicationContext(App.class, argsWithMissingParams)) {
new CommandLine(App.class, new MicronautFactory(ctx)).execute(argsWithMissingParams);
assertThat(out.toString(), startsWith("Missing required parameters: "));
assertThat(out.toString(), containsString("Usage: kestra flow namespace update "));
assertThat(out.toString(), not(containsString("MissingParameterException: ")));
assertThat(out.toString()).startsWith("Missing required parameters: ");
assertThat(out.toString()).contains("Usage: kestra flow namespace update ");
assertThat(out.toString()).doesNotContain("MissingParameterException: ");
}
}
}

View File

@@ -8,8 +8,7 @@ import org.junit.jupiter.api.Test;
import java.util.Map;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;
import static org.assertj.core.api.Assertions.assertThat;
class ServerCommandValidatorTest {
@@ -40,8 +39,8 @@ class ServerCommandValidatorTest {
.start()
);
final Throwable rootException = getRootException(exception);
assertThat(rootException.getClass(), is(ServerCommandValidator.ServerCommandException.class));
assertThat(rootException.getMessage(), is("Incomplete server configuration - missing required properties"));
assertThat(rootException.getClass()).isEqualTo(ServerCommandValidator.ServerCommandException.class);
assertThat(rootException.getMessage()).isEqualTo("Incomplete server configuration - missing required properties");
}
private Throwable getRootException(Throwable exception) {

View File

@@ -8,8 +8,7 @@ import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
class ConfigPropertiesCommandTest {
@Test
@@ -20,8 +19,8 @@ class ConfigPropertiesCommandTest {
try (ApplicationContext ctx = ApplicationContext.run(Environment.CLI, Environment.TEST)) {
PicocliRunner.call(ConfigPropertiesCommand.class, ctx);
assertThat(out.toString(), containsString("activeEnvironments:"));
assertThat(out.toString(), containsString("- test"));
assertThat(out.toString()).contains("activeEnvironments:");
assertThat(out.toString()).contains("- test");
}
}
}

View File

@@ -11,9 +11,7 @@ import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.net.URL;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.Is.is;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
class FlowCreateOrUpdateCommandTest {
@RetryingTest(5) // flaky on CI but cannot be reproduced even with 100 repetitions
@@ -38,7 +36,7 @@ class FlowCreateOrUpdateCommandTest {
};
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(out.toString(), containsString("4 flow(s)"));
assertThat(out.toString()).contains("4 flow(s)");
out.reset();
args = new String[]{
@@ -53,7 +51,7 @@ class FlowCreateOrUpdateCommandTest {
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
// 2 delete + 1 update
assertThat(out.toString(), containsString("4 flow(s)"));
assertThat(out.toString()).contains("4 flow(s)");
}
}
@@ -80,7 +78,7 @@ class FlowCreateOrUpdateCommandTest {
};
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(out.toString(), containsString("4 flow(s)"));
assertThat(out.toString()).contains("4 flow(s)");
out.reset();
// no "delete" arg should behave as no-delete
@@ -93,7 +91,7 @@ class FlowCreateOrUpdateCommandTest {
};
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(out.toString(), containsString("1 flow(s)"));
assertThat(out.toString()).contains("1 flow(s)");
out.reset();
args = new String[]{
@@ -106,7 +104,7 @@ class FlowCreateOrUpdateCommandTest {
};
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(out.toString(), containsString("1 flow(s)"));
assertThat(out.toString()).contains("1 flow(s)");
}
}
@@ -131,8 +129,8 @@ class FlowCreateOrUpdateCommandTest {
};
Integer call = PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("1 flow(s)"));
assertThat(call).isZero();
assertThat(out.toString()).contains("1 flow(s)");
}
}
}

View File

@@ -9,9 +9,7 @@ import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.net.URL;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.Is.is;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
class FlowDotCommandTest {
@Test
@@ -26,8 +24,8 @@ class FlowDotCommandTest {
};
Integer call = PicocliRunner.call(FlowDotCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("\"root.date\"[shape=box];"));
assertThat(call).isZero();
assertThat(out.toString()).contains("\"root.date\"[shape=box];");
}
}
}

View File

@@ -7,8 +7,7 @@ import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class FlowExpandCommandTest {
@SuppressWarnings("deprecation")
@@ -23,22 +22,20 @@ class FlowExpandCommandTest {
};
Integer call = PicocliRunner.call(FlowExpandCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), is(
"id: include\n" +
"namespace: io.kestra.cli\n" +
"\n" +
"# The list of tasks\n" +
"tasks:\n" +
"- id: t1\n" +
" type: io.kestra.plugin.core.debug.Return\n" +
" format: \"Lorem ipsum dolor sit amet\"\n" +
"- id: t2\n" +
" type: io.kestra.plugin.core.debug.Return\n" +
" format: |\n" +
" Lorem ipsum dolor sit amet\n" +
" Lorem ipsum dolor sit amet\n"
));
assertThat(call).isZero();
assertThat(out.toString()).isEqualTo("id: include\n" +
"namespace: io.kestra.cli\n" +
"\n" +
"# The list of tasks\n" +
"tasks:\n" +
"- id: t1\n" +
" type: io.kestra.plugin.core.debug.Return\n" +
" format: \"Lorem ipsum dolor sit amet\"\n" +
"- id: t2\n" +
" type: io.kestra.plugin.core.debug.Return\n" +
" format: |\n" +
" Lorem ipsum dolor sit amet\n" +
" Lorem ipsum dolor sit amet\n");
}
}
}

View File

@@ -14,10 +14,7 @@ import java.io.PrintStream;
import java.net.URL;
import java.util.zip.ZipFile;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.greaterThanOrEqualTo;
import static org.hamcrest.core.Is.is;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
class FlowExportCommandTest {
@Test
@@ -32,6 +29,8 @@ class FlowExportCommandTest {
// we use the update command to add flows to extract
String[] updateArgs = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -40,10 +39,12 @@ class FlowExportCommandTest {
directory.getPath(),
};
PicocliRunner.call(FlowNamespaceUpdateCommand.class, ctx, updateArgs);
assertThat(out.toString(), containsString("3 flow(s)"));
assertThat(out.toString()).contains("3 flow(s)");
// then we export them
String[] exportArgs = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -54,11 +55,11 @@ class FlowExportCommandTest {
};
PicocliRunner.call(FlowExportCommand.class, ctx, exportArgs);
File file = new File("/tmp/flows.zip");
assertThat(file.exists(), is(true));
assertThat(file.exists()).isTrue();
ZipFile zipFile = new ZipFile(file);
// When launching the test in a suite, there is 4 flows but when lauching individualy there is only 3
assertThat(zipFile.stream().count(), greaterThanOrEqualTo(3L));
assertThat(zipFile.stream().count()).isGreaterThanOrEqualTo(3L);
file.delete();
}

View File

@@ -10,9 +10,7 @@ import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.net.URL;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.Is.is;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
class FlowUpdatesCommandTest {
@Test
@@ -28,6 +26,8 @@ class FlowUpdatesCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -37,10 +37,12 @@ class FlowUpdatesCommandTest {
};
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(out.toString(), containsString("successfully updated !"));
assertThat(out.toString()).contains("successfully updated !");
out.reset();
args = new String[]{
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -52,7 +54,7 @@ class FlowUpdatesCommandTest {
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
// 2 delete + 1 update
assertThat(out.toString(), containsString("successfully updated !"));
assertThat(out.toString()).contains("successfully updated !");
}
}
@@ -70,6 +72,8 @@ class FlowUpdatesCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -79,11 +83,13 @@ class FlowUpdatesCommandTest {
};
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(out.toString(), containsString("4 flow(s)"));
assertThat(out.toString()).contains("4 flow(s)");
out.reset();
// no "delete" arg should behave as no-delete
args = new String[]{
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -92,10 +98,12 @@ class FlowUpdatesCommandTest {
};
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(out.toString(), containsString("1 flow(s)"));
assertThat(out.toString()).contains("1 flow(s)");
out.reset();
args = new String[]{
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -105,7 +113,36 @@ class FlowUpdatesCommandTest {
};
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(out.toString(), containsString("1 flow(s)"));
assertThat(out.toString()).contains("1 flow(s)");
}
}
@Test
void invalidWithNamespace() {
URL directory = FlowUpdatesCommandTest.class.getClassLoader().getResource("flows");
ByteArrayOutputStream out = new ByteArrayOutputStream();
System.setErr(new PrintStream(out));
try (ApplicationContext ctx = ApplicationContext.run(Environment.CLI, Environment.TEST)) {
EmbeddedServer embeddedServer = ctx.getBean(EmbeddedServer.class);
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
"myuser:pass:word",
"--namespace",
"io.kestra.cli",
"--delete",
directory.getPath(),
};
PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(out.toString()).contains("Invalid entity: flow.namespace: io.kestra.outsider_quattro_-1 - flow namespace is invalid");
}
}
@@ -121,6 +158,8 @@ class FlowUpdatesCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -130,8 +169,8 @@ class FlowUpdatesCommandTest {
};
Integer call = PicocliRunner.call(FlowUpdatesCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("1 flow(s)"));
assertThat(call).isZero();
assertThat(out.toString()).contains("1 flow(s)");
}
}
}

View File

@@ -7,9 +7,7 @@ import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.Is.is;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
class FlowValidateCommandTest {
@Test
@@ -24,8 +22,8 @@ class FlowValidateCommandTest {
};
Integer call = PicocliRunner.call(FlowValidateCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("✓ - io.kestra.cli / include"));
assertThat(call).isZero();
assertThat(out.toString()).contains("✓ - io.kestra.cli / include");
}
}
@@ -41,10 +39,10 @@ class FlowValidateCommandTest {
};
Integer call = PicocliRunner.call(FlowValidateCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("✓ - system / warning"));
assertThat(out.toString(), containsString("⚠ - tasks[0] is deprecated"));
assertThat(out.toString(), containsString(" - io.kestra.core.tasks.log.Log is replaced by io.kestra.plugin.core.log.Log"));
assertThat(call).isZero();
assertThat(out.toString()).contains("✓ - system / warning");
assertThat(out.toString()).contains("⚠ - tasks[0] is deprecated");
assertThat(out.toString()).contains(" - io.kestra.core.tasks.log.Log is replaced by io.kestra.plugin.core.log.Log");
}
}
}

View File

@@ -10,8 +10,7 @@ import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.net.URL;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
public class SingleFlowCommandsTest {
@@ -37,7 +36,7 @@ public class SingleFlowCommandsTest {
};
PicocliRunner.call(FlowDeleteCommand.class, ctx, deleteArgs);
assertThat(out.toString(), containsString("Flow successfully deleted !"));
assertThat(out.toString()).contains("Flow successfully deleted !");
out.reset();
String[] createArgs = {
@@ -49,7 +48,7 @@ public class SingleFlowCommandsTest {
};
PicocliRunner.call(FlowCreateCommand.class, ctx, createArgs);
assertThat(out.toString(), containsString("Flow successfully created !"));
assertThat(out.toString()).contains("Flow successfully created !");
out.reset();String[] updateArgs = {
@@ -63,7 +62,7 @@ public class SingleFlowCommandsTest {
};
PicocliRunner.call(FlowUpdateCommand.class, ctx, updateArgs);
assertThat(out.toString(), containsString("Flow successfully updated !"));
assertThat(out.toString()).contains("Flow successfully updated !");
out.reset();
}
}

View File

@@ -10,9 +10,7 @@ import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.net.URL;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.Is.is;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
class TemplateValidateCommandTest {
@Test
@@ -28,9 +26,9 @@ class TemplateValidateCommandTest {
};
Integer call = PicocliRunner.call(FlowValidateCommand.class, ctx, args);
assertThat(call, is(1));
assertThat(out.toString(), containsString("Unable to parse flow"));
assertThat(out.toString(), containsString("must not be empty"));
assertThat(call).isEqualTo(1);
assertThat(out.toString()).contains("Unable to parse flow");
assertThat(out.toString()).contains("must not be empty");
}
}
@@ -46,6 +44,8 @@ class TemplateValidateCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -54,9 +54,9 @@ class TemplateValidateCommandTest {
};
Integer call = PicocliRunner.call(FlowValidateCommand.class, ctx, args);
assertThat(call, is(1));
assertThat(out.toString(), containsString("Unable to parse flow"));
assertThat(out.toString(), containsString("must not be empty"));
assertThat(call).isEqualTo(1);
assertThat(out.toString()).contains("Unable to parse flow");
assertThat(out.toString()).contains("must not be empty");
}
}
}

View File

@@ -7,9 +7,7 @@ import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class FlowNamespaceCommandTest {
@Test
@@ -21,8 +19,8 @@ class FlowNamespaceCommandTest {
String[] args = {};
Integer call = PicocliRunner.call(FlowNamespaceCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("Usage: kestra flow namespace"));
assertThat(call).isZero();
assertThat(out.toString()).contains("Usage: kestra flow namespace");
}
}
}

View File

@@ -10,10 +10,7 @@ import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.net.URL;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.not;
import static org.hamcrest.core.StringContains.containsString;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class FlowNamespaceUpdateCommandTest {
@Test
@@ -39,7 +36,7 @@ class FlowNamespaceUpdateCommandTest {
};
PicocliRunner.call(FlowNamespaceUpdateCommand.class, ctx, args);
assertThat(out.toString(), containsString("namespace 'io.kestra.cli' successfully updated"));
assertThat(out.toString()).contains("namespace 'io.kestra.cli' successfully updated");
out.reset();
args = new String[]{
@@ -55,7 +52,7 @@ class FlowNamespaceUpdateCommandTest {
PicocliRunner.call(FlowNamespaceUpdateCommand.class, ctx, args);
// 2 delete + 1 update
assertThat(out.toString(), containsString("namespace 'io.kestra.cli' successfully updated"));
assertThat(out.toString()).contains("namespace 'io.kestra.cli' successfully updated");
}
}
@@ -81,9 +78,9 @@ class FlowNamespaceUpdateCommandTest {
};
Integer call = PicocliRunner.call(FlowNamespaceUpdateCommand.class, ctx, args);
assertThat(call, is(1));
assertThat(out.toString(), containsString("Unable to parse flows"));
assertThat(out.toString(), containsString("must not be empty"));
assertThat(call).isEqualTo(1);
assertThat(out.toString()).contains("Unable to parse flows");
assertThat(out.toString()).contains("must not be empty");
}
}
@@ -111,7 +108,7 @@ class FlowNamespaceUpdateCommandTest {
};
PicocliRunner.call(FlowNamespaceUpdateCommand.class, ctx, args);
assertThat(out.toString(), containsString("3 flow(s)"));
assertThat(out.toString()).contains("3 flow(s)");
out.reset();
// no "delete" arg should behave as no-delete
@@ -125,7 +122,7 @@ class FlowNamespaceUpdateCommandTest {
};
PicocliRunner.call(FlowNamespaceUpdateCommand.class, ctx, args);
assertThat(out.toString(), containsString("1 flow(s)"));
assertThat(out.toString()).contains("1 flow(s)");
out.reset();
args = new String[]{
@@ -139,7 +136,7 @@ class FlowNamespaceUpdateCommandTest {
};
PicocliRunner.call(FlowNamespaceUpdateCommand.class, ctx, args);
assertThat(out.toString(), containsString("1 flow(s)"));
assertThat(out.toString()).contains("1 flow(s)");
}
}
@@ -165,8 +162,8 @@ class FlowNamespaceUpdateCommandTest {
};
Integer call = PicocliRunner.call(FlowNamespaceUpdateCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("1 flow(s)"));
assertThat(call).isZero();
assertThat(out.toString()).contains("1 flow(s)");
}
}
@@ -195,8 +192,8 @@ class FlowNamespaceUpdateCommandTest {
};
PicocliRunner.call(FlowNamespaceUpdateCommand.class, ctx, args);
assertThat(out.toString(), containsString("io.kestra.override"));
assertThat(out.toString(), not(containsString("io.kestra.cli")));
assertThat(out.toString()).contains("io.kestra.override");
assertThat(out.toString()).doesNotContain("io.kestra.cli");
}
}

View File

@@ -7,9 +7,7 @@ import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class NamespaceCommandTest {
@Test
@@ -21,8 +19,8 @@ class NamespaceCommandTest {
String[] args = {};
Integer call = PicocliRunner.call(NamespaceCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("Usage: kestra namespace"));
assertThat(call).isZero();
assertThat(out.toString()).contains("Usage: kestra namespace");
}
}
}

View File

@@ -7,9 +7,7 @@ import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class NamespaceFilesCommandTest {
@Test
@@ -21,8 +19,8 @@ class NamespaceFilesCommandTest {
String[] args = {};
Integer call = PicocliRunner.call(NamespaceFilesCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("Usage: kestra namespace files"));
assertThat(call).isZero();
assertThat(out.toString()).contains("Usage: kestra namespace files");
}
}
}

View File

@@ -14,8 +14,8 @@ import java.net.URISyntaxException;
import java.net.URL;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.not;
import static org.hamcrest.core.StringContains.containsString;
class NamespaceFilesUpdateCommandTest {
@Test
@@ -31,6 +31,8 @@ class NamespaceFilesUpdateCommandTest {
String to = "/some/directory";
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -61,6 +63,8 @@ class NamespaceFilesUpdateCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -90,6 +94,8 @@ class NamespaceFilesUpdateCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",

View File

@@ -7,9 +7,7 @@ import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class KvCommandTest {
@Test
@@ -21,8 +19,8 @@ class KvCommandTest {
String[] args = {};
Integer call = PicocliRunner.call(KvCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("Usage: kestra namespace kv"));
assertThat(call).isZero();
assertThat(out.toString()).contains("Usage: kestra namespace kv");
}
}
}

View File

@@ -16,8 +16,7 @@ import java.io.IOException;
import java.nio.file.Files;
import java.util.Map;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;
import static org.assertj.core.api.Assertions.assertThat;
class KvUpdateCommandTest {
@Test
@@ -28,6 +27,8 @@ class KvUpdateCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -41,8 +42,8 @@ class KvUpdateCommandTest {
KVStoreService kvStoreService = ctx.getBean(KVStoreService.class);
KVStore kvStore = kvStoreService.get(null, "io.kestra.cli", null);
assertThat(kvStore.getValue("string").get(), is(new KVValue("stringValue")));
assertThat(((InternalKVStore)kvStore).getRawValue("string").get(), is("\"stringValue\""));
assertThat(kvStore.getValue("string").get()).isEqualTo(new KVValue("stringValue"));
assertThat(((InternalKVStore) kvStore).getRawValue("string").get()).isEqualTo("\"stringValue\"");
}
}
@@ -54,6 +55,8 @@ class KvUpdateCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -67,8 +70,8 @@ class KvUpdateCommandTest {
KVStoreService kvStoreService = ctx.getBean(KVStoreService.class);
KVStore kvStore = kvStoreService.get(null, "io.kestra.cli", null);
assertThat(kvStore.getValue("int").get(), is(new KVValue(1)));
assertThat(((InternalKVStore)kvStore).getRawValue("int").get(), is("1"));
assertThat(kvStore.getValue("int").get()).isEqualTo(new KVValue(1));
assertThat(((InternalKVStore) kvStore).getRawValue("int").get()).isEqualTo("1");
}
}
@@ -80,6 +83,8 @@ class KvUpdateCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -95,8 +100,8 @@ class KvUpdateCommandTest {
KVStoreService kvStoreService = ctx.getBean(KVStoreService.class);
KVStore kvStore = kvStoreService.get(null, "io.kestra.cli", null);
assertThat(kvStore.getValue("intStr").get(), is(new KVValue("1")));
assertThat(((InternalKVStore)kvStore).getRawValue("intStr").get(), is("\"1\""));
assertThat(kvStore.getValue("intStr").get()).isEqualTo(new KVValue("1"));
assertThat(((InternalKVStore) kvStore).getRawValue("intStr").get()).isEqualTo("\"1\"");
}
}
@@ -108,6 +113,8 @@ class KvUpdateCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -121,8 +128,8 @@ class KvUpdateCommandTest {
KVStoreService kvStoreService = ctx.getBean(KVStoreService.class);
KVStore kvStore = kvStoreService.get(null, "io.kestra.cli", null);
assertThat(kvStore.getValue("object").get(), is(new KVValue(Map.of("some", "json"))));
assertThat(((InternalKVStore)kvStore).getRawValue("object").get(), is("{some:\"json\"}"));
assertThat(kvStore.getValue("object").get()).isEqualTo(new KVValue(Map.of("some", "json")));
assertThat(((InternalKVStore) kvStore).getRawValue("object").get()).isEqualTo("{some:\"json\"}");
}
}
@@ -134,6 +141,8 @@ class KvUpdateCommandTest {
embeddedServer.start();
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -149,8 +158,8 @@ class KvUpdateCommandTest {
KVStoreService kvStoreService = ctx.getBean(KVStoreService.class);
KVStore kvStore = kvStoreService.get(null, "io.kestra.cli", null);
assertThat(kvStore.getValue("objectStr").get(), is(new KVValue("{\"some\":\"json\"}")));
assertThat(((InternalKVStore)kvStore).getRawValue("objectStr").get(), is("\"{\\\"some\\\":\\\"json\\\"}\""));
assertThat(kvStore.getValue("objectStr").get()).isEqualTo(new KVValue("{\"some\":\"json\"}"));
assertThat(((InternalKVStore) kvStore).getRawValue("objectStr").get()).isEqualTo("\"{\\\"some\\\":\\\"json\\\"}\"");
}
}
@@ -167,6 +176,8 @@ class KvUpdateCommandTest {
Files.write(file.toPath(), "{\"some\":\"json\",\"from\":\"file\"}".getBytes());
String[] args = {
"--plugins",
"/tmp", // pass this arg because it can cause failure
"--server",
embeddedServer.getURL().toString(),
"--user",
@@ -181,8 +192,8 @@ class KvUpdateCommandTest {
KVStoreService kvStoreService = ctx.getBean(KVStoreService.class);
KVStore kvStore = kvStoreService.get(null, "io.kestra.cli", null);
assertThat(kvStore.getValue("objectFromFile").get(), is(new KVValue(Map.of("some", "json", "from", "file"))));
assertThat(((InternalKVStore)kvStore).getRawValue("objectFromFile").get(), is("{some:\"json\",from:\"file\"}"));
assertThat(kvStore.getValue("objectFromFile").get()).isEqualTo(new KVValue(Map.of("some", "json", "from", "file")));
assertThat(((InternalKVStore) kvStore).getRawValue("objectFromFile").get()).isEqualTo("{some:\"json\",from:\"file\"}");
}
}
}

View File

@@ -0,0 +1,26 @@
package io.kestra.cli.commands.plugins;
import io.micronaut.configuration.picocli.PicocliRunner;
import io.micronaut.context.ApplicationContext;
import io.micronaut.context.env.Environment;
import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.assertj.core.api.Assertions.assertThat;
class PluginCommandTest {
@Test
void shouldGetHelps() {
ByteArrayOutputStream out = new ByteArrayOutputStream();
System.setOut(new PrintStream(out));
try (ApplicationContext ctx = ApplicationContext.run(Environment.CLI, Environment.TEST)) {
PicocliRunner.call(PluginCommand.class, ctx);
assertThat(out.toString()).contains("Usage: kestra plugins");
}
}
}

View File

@@ -17,8 +17,7 @@ import java.util.List;
import java.util.Objects;
import java.util.stream.Collectors;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.*;
import static org.assertj.core.api.Assertions.assertThat;
class PluginDocCommandTest {
@@ -44,16 +43,16 @@ class PluginDocCommandTest {
List<Path> files = Files.list(docPath).toList();
assertThat(files.size(), is(1));
assertThat(files.getFirst().getFileName().toString(), is("plugin-template-test"));
assertThat(files.size()).isEqualTo(1);
assertThat(files.getFirst().getFileName().toString()).isEqualTo("plugin-template-test");
var directory = files.getFirst().toFile();
assertThat(directory.isDirectory(), is(true));
assertThat(directory.listFiles().length, is(3));
assertThat(directory.isDirectory()).isTrue();
assertThat(directory.listFiles().length).isEqualTo(3);
var readme = directory.toPath().resolve("index.md");
var readmeContent = new String(Files.readAllBytes(readme));
assertThat(readmeContent, containsString("""
assertThat(readmeContent).contains("""
---
title: Template test
description: "Plugin template for Kestra"
@@ -61,18 +60,17 @@ class PluginDocCommandTest {
---
# Template test
"""));
""");
assertThat(readmeContent, containsString("""
assertThat(readmeContent).contains("""
Plugin template for Kestra
This is a more complex description of the plugin.
This is in markdown and will be inline inside the plugin page.
"""));
""");
assertThat(readmeContent, containsString(
"""
assertThat(readmeContent).contains("""
/> Subgroup title
Subgroup description
@@ -89,20 +87,20 @@ class PluginDocCommandTest {
\s
* [Reporting](./guides/reporting.md)
\s
"""));
""");
// check @PluginProperty from an interface
var task = directory.toPath().resolve("tasks/io.kestra.plugin.templates.ExampleTask.md");
String taskDoc = new String(Files.readAllBytes(task));
assertThat(taskDoc, containsString("""
assertThat(taskDoc).contains("""
### `example`
* **Type:** ==string==
* **Dynamic:** ✔️
* **Required:** ❌
**Example interface**
"""));
assertThat(taskDoc, containsString("""
""");
assertThat(taskDoc).contains("""
### `from`
* **Type:**
* ==string==
@@ -110,12 +108,12 @@ class PluginDocCommandTest {
* [==Example==](#io.kestra.core.models.annotations.example)
* **Dynamic:** ✔️
* **Required:** ✔️
"""));
""");
var authenticationGuide = directory.toPath().resolve("guides/authentication.md");
assertThat(new String(Files.readAllBytes(authenticationGuide)), containsString("This is how to authenticate for this plugin:"));
assertThat(new String(Files.readAllBytes(authenticationGuide))).contains("This is how to authenticate for this plugin:");
var reportingGuide = directory.toPath().resolve("guides/reporting.md");
assertThat(new String(Files.readAllBytes(reportingGuide)), containsString("This is the reporting of the plugin:"));
assertThat(new String(Files.readAllBytes(reportingGuide))).contains("This is the reporting of the plugin:");
}
}
}

View File

@@ -9,15 +9,13 @@ import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.stream.Collectors;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.*;
import static org.assertj.core.api.Assertions.assertThat;
class PluginInstallCommandTest {
@Test
void fixedVersion() throws IOException {
void shouldInstallPluginLocallyGivenFixedVersion() throws IOException {
Path pluginsPath = Files.createTempDirectory(PluginInstallCommandTest.class.getSimpleName());
pluginsPath.toFile().deleteOnExit();
@@ -27,13 +25,13 @@ class PluginInstallCommandTest {
List<Path> files = Files.list(pluginsPath).toList();
assertThat(files.size(), is(1));
assertThat(files.getFirst().getFileName().toString(), is("plugin-notifications-0.6.0.jar"));
assertThat(files.size()).isEqualTo(1);
assertThat(files.getFirst().getFileName().toString()).isEqualTo("io_kestra_plugin__plugin-notifications__0_6_0.jar");
}
}
@Test
void latestVersion() throws IOException {
void shouldInstallPluginLocallyGivenLatestVersion() throws IOException {
Path pluginsPath = Files.createTempDirectory(PluginInstallCommandTest.class.getSimpleName());
pluginsPath.toFile().deleteOnExit();
@@ -43,14 +41,14 @@ class PluginInstallCommandTest {
List<Path> files = Files.list(pluginsPath).toList();
assertThat(files.size(), is(1));
assertThat(files.getFirst().getFileName().toString(), startsWith("plugin-notifications"));
assertThat(files.getFirst().getFileName().toString(), not(containsString("LATEST")));
assertThat(files.size()).isEqualTo(1);
assertThat(files.getFirst().getFileName().toString()).startsWith("io_kestra_plugin__plugin-notifications__");
assertThat(files.getFirst().getFileName().toString()).doesNotContain("LATEST");
}
}
@Test
void rangeVersion() throws IOException {
void shouldInstallPluginLocallyGivenRangeVersion() throws IOException {
Path pluginsPath = Files.createTempDirectory(PluginInstallCommandTest.class.getSimpleName());
pluginsPath.toFile().deleteOnExit();
@@ -61,8 +59,8 @@ class PluginInstallCommandTest {
List<Path> files = Files.list(pluginsPath).toList();
assertThat(files.size(), is(1));
assertThat(files.getFirst().getFileName().toString(), is("storage-s3-0.12.1.jar"));
assertThat(files.size()).isEqualTo(1);
assertThat(files.getFirst().getFileName().toString()).isEqualTo("io_kestra_storage__storage-s3__0_12_1.jar");
}
}
}

View File

@@ -4,7 +4,6 @@ import io.micronaut.configuration.picocli.PicocliRunner;
import io.micronaut.context.ApplicationContext;
import io.micronaut.context.env.Environment;
import org.apache.commons.io.FileUtils;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
@@ -17,15 +16,14 @@ import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Objects;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
class PluginListCommandTest {
private static final String PLUGIN_TEMPLATE_TEST = "plugin-template-test-0.18.0-SNAPSHOT.jar";
@Test
void run() throws IOException, URISyntaxException {
void shouldListPluginsInstalledLocally() throws IOException, URISyntaxException {
Path pluginsPath = Files.createTempDirectory(PluginListCommandTest.class.getSimpleName());
pluginsPath.toFile().deleteOnExit();
@@ -42,7 +40,7 @@ class PluginListCommandTest {
String[] args = {"--plugins", pluginsPath.toAbsolutePath().toString()};
PicocliRunner.call(PluginListCommand.class, ctx, args);
assertThat(out.toString(), containsString("io.kestra.plugin.templates.Example"));
assertThat(out.toString()).contains("io.kestra.plugin.templates.Example");
}
}
}

View File

@@ -0,0 +1,104 @@
package io.kestra.cli.commands.plugins;
import com.github.tomakehurst.wiremock.junit5.WireMockTest;
import io.micronaut.configuration.picocli.PicocliRunner;
import io.micronaut.context.ApplicationContext;
import io.micronaut.context.env.Environment;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.util.Map;
import static com.github.tomakehurst.wiremock.client.WireMock.*;
import static org.assertj.core.api.Assertions.assertThat;
@WireMockTest(httpPort = 28181)
class PluginSearchCommandTest {
private ByteArrayOutputStream outputStreamCaptor;
private final PrintStream originalOut = System.out;
@BeforeEach
void setUp() {
outputStreamCaptor = new ByteArrayOutputStream();
System.setOut(new PrintStream(outputStreamCaptor));
}
@AfterEach
void tearDown() {
System.setOut(originalOut);
}
@Test
void searchWithExactMatch() {
stubFor(get(urlEqualTo("/v1/plugins"))
.willReturn(aResponse()
.withHeader("Content-Type", "application/json")
.withBody("""
[
{
"name": "plugin-notifications",
"title": "Notifications",
"group": "io.kestra.plugin",
"version": "0.6.0"
},
{
"name": "plugin-scripts",
"title": "Scripts",
"group": "io.kestra.plugin",
"version": "0.5.0"
}
]
""")));
try (ApplicationContext ctx = ApplicationContext.builder(Environment.CLI, Environment.TEST)
.properties(Map.of("micronaut.http.services.api.url", "http://localhost:28181"))
.start()) {
String[] args = {"notifications"};
PicocliRunner.call(PluginSearchCommand.class, ctx, args);
String output = outputStreamCaptor.toString().trim();
assertThat(output).contains("Found 1 plugins matching 'notifications'");
assertThat(output).contains("plugin-notifications");
assertThat(output).doesNotContain("plugin-scripts");
}
}
@Test
void searchWithEmptyQuery() {
stubFor(get(urlEqualTo("/v1/plugins"))
.willReturn(aResponse()
.withHeader("Content-Type", "application/json")
.withBody("""
[
{
"name": "plugin-notifications",
"title": "Notifications",
"group": "io.kestra.plugin",
"version": "0.6.0"
},
{
"name": "plugin-scripts",
"title": "Scripts",
"group": "io.kestra.plugin",
"version": "0.5.0"
}
]
""")));
try (ApplicationContext ctx = ApplicationContext.builder(Environment.CLI, Environment.TEST)
.properties(Map.of("micronaut.http.services.api.url", "http://localhost:28181"))
.start()) {
String[] args = {""};
PicocliRunner.call(PluginSearchCommand.class, ctx, args);
String output = outputStreamCaptor.toString().trim();
assertThat(output).contains("Found 2 plugins");
assertThat(output).contains("plugin-notifications");
assertThat(output).contains("plugin-scripts");
}
}
}

View File

@@ -11,9 +11,7 @@ import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.net.URL;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.Is.is;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
class ReindexCommandTest {
@Test
@@ -36,7 +34,7 @@ class ReindexCommandTest {
directory.getPath(),
};
PicocliRunner.call(FlowNamespaceUpdateCommand.class, ctx, updateArgs);
assertThat(out.toString(), containsString("3 flow(s)"));
assertThat(out.toString()).contains("3 flow(s)");
// then we reindex them
String[] reindexArgs = {
@@ -44,9 +42,9 @@ class ReindexCommandTest {
"flow",
};
Integer call = PicocliRunner.call(ReindexCommand.class, ctx, reindexArgs);
assertThat(call, is(0));
assertThat(call).isZero();
// in local it reindex 3 flows and in CI 4 for an unknown reason
assertThat(out.toString(), containsString("Successfully reindex"));
assertThat(out.toString()).contains("Successfully reindex");
}
}
}

View File

@@ -7,9 +7,7 @@ import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class DatabaseCommandTest {
@Test
@@ -21,8 +19,8 @@ class DatabaseCommandTest {
String[] args = {};
Integer call = PicocliRunner.call(DatabaseCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("Usage: kestra sys database"));
assertThat(call).isZero();
assertThat(out.toString()).contains("Usage: kestra sys database");
}
}
}

View File

@@ -8,9 +8,7 @@ import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class StateStoreCommandTest {
@Test
@@ -22,8 +20,8 @@ class StateStoreCommandTest {
String[] args = {};
Integer call = PicocliRunner.call(StateStoreCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("Usage: kestra sys state-store"));
assertThat(call).isZero();
assertThat(out.toString()).contains("Usage: kestra sys state-store");
}
}
}

View File

@@ -1,16 +1,15 @@
package io.kestra.cli.commands.sys.statestore;
import com.devskiller.friendly_id.FriendlyId;
import io.kestra.core.exceptions.MigrationRequiredException;
import io.kestra.core.exceptions.ResourceExpiredException;
import io.kestra.core.models.flows.Flow;
import io.kestra.core.models.flows.GenericFlow;
import io.kestra.core.repositories.FlowRepositoryInterface;
import io.kestra.core.runners.RunContext;
import io.kestra.core.runners.RunContextFactory;
import io.kestra.core.storages.StateStore;
import io.kestra.core.storages.StorageInterface;
import io.kestra.core.utils.Hashing;
import io.kestra.core.utils.IdUtils;
import io.kestra.core.utils.Slugify;
import io.kestra.plugin.core.log.Log;
import io.micronaut.configuration.picocli.PicocliRunner;
@@ -26,9 +25,7 @@ import java.net.URI;
import java.util.List;
import java.util.Map;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class StateStoreMigrateCommandTest {
@Test
@@ -45,7 +42,7 @@ class StateStoreMigrateCommandTest {
.namespace("some.valid.namespace." + ((int) (Math.random() * 1000000)))
.tasks(List.of(Log.builder().id("log").type(Log.class.getName()).message("logging").build()))
.build();
flowRepository.create(flow, flow.generateSource(), flow);
flowRepository.create(GenericFlow.of(flow));
StorageInterface storage = ctx.getBean(StorageInterface.class);
String tenantId = flow.getTenantId();
@@ -56,10 +53,7 @@ class StateStoreMigrateCommandTest {
oldStateStoreUri,
new ByteArrayInputStream("my-value".getBytes())
);
assertThat(
storage.exists(tenantId, flow.getNamespace(), oldStateStoreUri),
is(true)
);
assertThat(storage.exists(tenantId, flow.getNamespace(), oldStateStoreUri)).isTrue();
RunContext runContext = ctx.getBean(RunContextFactory.class).of(flow, Map.of("flow", Map.of(
"tenantId", tenantId,
@@ -72,13 +66,10 @@ class StateStoreMigrateCommandTest {
String[] args = {};
Integer call = PicocliRunner.call(StateStoreMigrateCommand.class, ctx, args);
assertThat(new String(stateStore.getState(true, "my-state", "sub-name", "my-taskrun-value").readAllBytes()), is("my-value"));
assertThat(
storage.exists(tenantId, flow.getNamespace(), oldStateStoreUri),
is(false)
);
assertThat(new String(stateStore.getState(true, "my-state", "sub-name", "my-taskrun-value").readAllBytes())).isEqualTo("my-value");
assertThat(storage.exists(tenantId, flow.getNamespace(), oldStateStoreUri)).isFalse();
assertThat(call, is(0));
assertThat(call).isZero();
}
}
}

View File

@@ -15,9 +15,7 @@ import java.net.URL;
import java.util.Map;
import java.util.zip.ZipFile;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.StringContains.containsString;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class TemplateExportCommandTest {
@Test
@@ -42,7 +40,7 @@ class TemplateExportCommandTest {
};
PicocliRunner.call(TemplateNamespaceUpdateCommand.class, ctx, args);
assertThat(out.toString(), containsString("3 template(s)"));
assertThat(out.toString()).contains("3 template(s)");
// then we export them
String[] exportArgs = {
@@ -56,9 +54,9 @@ class TemplateExportCommandTest {
};
PicocliRunner.call(TemplateExportCommand.class, ctx, exportArgs);
File file = new File("/tmp/templates.zip");
assertThat(file.exists(), is(true));
assertThat(file.exists()).isTrue();
ZipFile zipFile = new ZipFile(file);
assertThat(zipFile.stream().count(), is(3L));
assertThat(zipFile.stream().count()).isEqualTo(3L);
file.delete();
}

View File

@@ -11,11 +11,9 @@ import java.io.PrintStream;
import java.net.URL;
import java.util.Map;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.Is.is;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
public class TemplateValidateCommandTest {
class TemplateValidateCommandTest {
@Test
void runLocal() {
URL directory = TemplateValidateCommandTest.class.getClassLoader().getResource("invalidsTemplates/template.yml");
@@ -29,9 +27,9 @@ public class TemplateValidateCommandTest {
};
Integer call = PicocliRunner.call(TemplateValidateCommand.class, ctx, args);
assertThat(call, is(1));
assertThat(out.toString(), containsString("Unable to parse template"));
assertThat(out.toString(), containsString("must not be empty"));
assertThat(call).isEqualTo(1);
assertThat(out.toString()).contains("Unable to parse template");
assertThat(out.toString()).contains("must not be empty");
}
}
@@ -55,9 +53,9 @@ public class TemplateValidateCommandTest {
};
Integer call = PicocliRunner.call(TemplateValidateCommand.class, ctx, args);
assertThat(call, is(1));
assertThat(out.toString(), containsString("Unable to parse template"));
assertThat(out.toString(), containsString("must not be empty"));
assertThat(call).isEqualTo(1);
assertThat(out.toString()).contains("Unable to parse template");
assertThat(out.toString()).contains("must not be empty");
}
}
}

View File

@@ -7,9 +7,7 @@ import org.junit.jupiter.api.Test;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.core.Is.is;
import static org.assertj.core.api.Assertions.assertThat;
class TemplateNamespaceCommandTest {
@Test
@@ -21,8 +19,8 @@ class TemplateNamespaceCommandTest {
String[] args = {};
Integer call = PicocliRunner.call(TemplateNamespaceCommand.class, ctx, args);
assertThat(call, is(0));
assertThat(out.toString(), containsString("Usage: kestra template namespace"));
assertThat(call).isZero();
assertThat(out.toString()).contains("Usage: kestra template namespace");
}
}
}

View File

@@ -11,8 +11,7 @@ import java.io.PrintStream;
import java.net.URL;
import java.util.Map;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.StringContains.containsString;
import static org.assertj.core.api.Assertions.assertThat;
class TemplateNamespaceUpdateCommandTest {
@Test
@@ -37,7 +36,7 @@ class TemplateNamespaceUpdateCommandTest {
};
PicocliRunner.call(TemplateNamespaceUpdateCommand.class, ctx, args);
assertThat(out.toString(), containsString("3 template(s)"));
assertThat(out.toString()).contains("3 template(s)");
}
}
@@ -64,8 +63,8 @@ class TemplateNamespaceUpdateCommandTest {
Integer call = PicocliRunner.call(TemplateNamespaceUpdateCommand.class, ctx, args);
// assertThat(call, is(1));
assertThat(out.toString(), containsString("Unable to parse templates"));
assertThat(out.toString(), containsString("must not be empty"));
assertThat(out.toString()).contains("Unable to parse templates");
assertThat(out.toString()).contains("must not be empty");
}
}
@@ -93,7 +92,7 @@ class TemplateNamespaceUpdateCommandTest {
};
PicocliRunner.call(TemplateNamespaceUpdateCommand.class, ctx, args);
assertThat(out.toString(), containsString("3 template(s)"));
assertThat(out.toString()).contains("3 template(s)");
String[] newArgs = {
"--server",
@@ -107,7 +106,7 @@ class TemplateNamespaceUpdateCommandTest {
};
PicocliRunner.call(TemplateNamespaceUpdateCommand.class, ctx, newArgs);
assertThat(out.toString(), containsString("1 template(s)"));
assertThat(out.toString()).contains("1 template(s)");
}
}
}

View File

@@ -10,8 +10,7 @@ import java.io.IOException;
import java.nio.file.Files;
import java.util.Map;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;
import static org.assertj.core.api.Assertions.assertThat;
class DeleteConfigurationApplicationListenersTest {
@@ -28,7 +27,7 @@ class DeleteConfigurationApplicationListenersTest {
);
try (ApplicationContext ctx = ApplicationContext.run(mapPropertySource, Environment.CLI, Environment.TEST)) {
assertThat(tempFile.exists(), is(false));
assertThat(tempFile.exists()).isFalse();
}
}
}

View File

@@ -19,8 +19,7 @@ import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicBoolean;
import static io.kestra.core.utils.Rethrow.throwRunnable;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.*;
import static org.assertj.core.api.Assertions.assertThat;
@MicronautTest(environments = {"test", "file-watch"}, transactional = false)
class FileChangedEventListenerTest {
@@ -77,9 +76,9 @@ class FileChangedEventListenerTest {
Duration.ofSeconds(10)
);
Flow myflow = flowRepository.findById(null, "io.kestra.tests.watch", "myflow").orElseThrow();
assertThat(myflow.getTasks(), hasSize(1));
assertThat(myflow.getTasks().getFirst().getId(), is("hello"));
assertThat(myflow.getTasks().getFirst().getType(), is("io.kestra.plugin.core.log.Log"));
assertThat(myflow.getTasks()).hasSize(1);
assertThat(myflow.getTasks().getFirst().getId()).isEqualTo("hello");
assertThat(myflow.getTasks().getFirst().getType()).isEqualTo("io.kestra.plugin.core.log.Log");
// delete the flow
Files.delete(Path.of(FILE_WATCH + "/myflow.yaml"));
@@ -116,9 +115,9 @@ class FileChangedEventListenerTest {
Duration.ofSeconds(10)
);
Flow pluginDefaultFlow = flowRepository.findById(null, "io.kestra.tests.watch", "pluginDefault").orElseThrow();
assertThat(pluginDefaultFlow.getTasks(), hasSize(1));
assertThat(pluginDefaultFlow.getTasks().getFirst().getId(), is("helloWithDefault"));
assertThat(pluginDefaultFlow.getTasks().getFirst().getType(), is("io.kestra.plugin.core.log.Log"));
assertThat(pluginDefaultFlow.getTasks()).hasSize(1);
assertThat(pluginDefaultFlow.getTasks().getFirst().getId()).isEqualTo("helloWithDefault");
assertThat(pluginDefaultFlow.getTasks().getFirst().getType()).isEqualTo("io.kestra.plugin.core.log.Log");
// delete both files
Files.delete(Path.of(FILE_WATCH + "/plugin-default.yaml"));

View File

@@ -38,6 +38,13 @@ dependencies {
implementation group: 'dev.failsafe', name: 'failsafe'
api 'org.apache.httpcomponents.client5:httpclient5'
// plugins
implementation 'org.apache.maven.resolver:maven-resolver-impl'
implementation 'org.apache.maven.resolver:maven-resolver-supplier'
implementation 'org.apache.maven.resolver:maven-resolver-connector-basic'
implementation 'org.apache.maven.resolver:maven-resolver-transport-file'
implementation 'org.apache.maven.resolver:maven-resolver-transport-http'
// scheduler
implementation group: 'com.cronutils', name: 'cron-utils'
@@ -66,7 +73,7 @@ dependencies {
testImplementation "io.micronaut:micronaut-http-server-netty"
testImplementation "io.micronaut:micronaut-management"
testImplementation "org.testcontainers:testcontainers:1.20.4"
testImplementation "org.testcontainers:junit-jupiter:1.20.4"
testImplementation "org.testcontainers:testcontainers:1.20.6"
testImplementation "org.testcontainers:junit-jupiter:1.20.6"
testImplementation "org.bouncycastle:bcpkix-jdk18on:1.80"
}

View File

@@ -3,7 +3,6 @@ package io.kestra.core.contexts;
import io.kestra.core.exceptions.KestraRuntimeException;
import io.kestra.core.plugins.DefaultPluginRegistry;
import io.kestra.core.plugins.PluginRegistry;
import io.kestra.core.plugins.serdes.PluginDeserializer;
import io.kestra.core.storages.StorageInterface;
import io.kestra.core.storages.StorageInterfaceFactory;
import io.micronaut.context.annotation.Bean;
@@ -34,7 +33,7 @@ public class KestraBeansFactory {
StorageConfig storageConfig;
@Value("${kestra.storage.type}")
Optional<String> storageType;
protected Optional<String> storageType;
@Requires(missingBeans = PluginRegistry.class)
@Singleton
@@ -42,16 +41,25 @@ public class KestraBeansFactory {
return DefaultPluginRegistry.getOrCreate();
}
@Singleton
public StorageInterfaceFactory storageInterfaceFactory(final PluginRegistry pluginRegistry){
return new StorageInterfaceFactory(pluginRegistry, validator);
}
@Requires(missingBeans = StorageInterface.class)
@Singleton
@Bean(preDestroy = "close")
public StorageInterface storageInterface(final PluginRegistry pluginRegistry) throws IOException {
String pluginId = storageType.orElseThrow(() -> new KestraRuntimeException(String.format(
public StorageInterface storageInterface(final StorageInterfaceFactory storageInterfaceFactory) throws IOException {
String pluginId = getStoragePluginId(storageInterfaceFactory);
return storageInterfaceFactory.make(null, pluginId, storageConfig.getStorageConfig(pluginId));
}
public String getStoragePluginId(StorageInterfaceFactory storageInterfaceFactory) {
return storageType.orElseThrow(() -> new KestraRuntimeException(String.format(
"No storage configured through the application property '%s'. Supported types are: %s"
, KESTRA_STORAGE_TYPE_CONFIG,
StorageInterfaceFactory.getLoggableStorageIds(pluginRegistry)
storageInterfaceFactory.getLoggableStorageIds()
)));
return StorageInterfaceFactory.make(pluginRegistry, pluginId, storageConfig.getStorageConfig(pluginId), validator);
}
@ConfigurationProperties("kestra")
@@ -67,7 +75,7 @@ public class KestraBeansFactory {
* @return the configuration.
*/
@SuppressWarnings("unchecked")
private Map<String, Object> getStorageConfig(String type) {
public Map<String, Object> getStorageConfig(String type) {
return (Map<String, Object>) storage.get(StringConvention.CAMEL_CASE.format(type));
}
}

View File

@@ -10,6 +10,8 @@ import io.micronaut.context.env.Environment;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.HashMap;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicReference;
@@ -25,7 +27,11 @@ public abstract class KestraContext {
private static final AtomicReference<KestraContext> INSTANCE = new AtomicReference<>();
// Properties
private static final String KESTRA_SERVER_TYPE = "kestra.server-type";
public static final String KESTRA_SERVER_TYPE = "kestra.server-type";
// Those properties are injected bases on the CLI args.
private static final String KESTRA_WORKER_MAX_NUM_THREADS = "kestra.worker.max-num-threads";
private static final String KESTRA_WORKER_GROUP_KEY = "kestra.worker.group-key";
/**
* Gets the current {@link KestraContext}.
@@ -54,6 +60,12 @@ public abstract class KestraContext {
*/
public abstract ServerType getServerType();
public abstract Optional<Integer> getWorkerMaxNumThreads();
public abstract Optional<String> getWorkerGroupKey();
public abstract void injectWorkerConfigs(Integer maxNumThreads, String workerGroupKey);
/**
* Returns the Kestra Version.
*
@@ -110,6 +122,34 @@ public abstract class KestraContext {
.orElse(ServerType.STANDALONE);
}
/** {@inheritDoc} **/
@Override
public Optional<Integer> getWorkerMaxNumThreads() {
return Optional.ofNullable(environment)
.flatMap(env -> env.getProperty(KESTRA_WORKER_MAX_NUM_THREADS, Integer.class));
}
/** {@inheritDoc} **/
@Override
public Optional<String> getWorkerGroupKey() {
return Optional.ofNullable(environment)
.flatMap(env -> env.getProperty(KESTRA_WORKER_GROUP_KEY, String.class));
}
/** {@inheritDoc} **/
@Override
public void injectWorkerConfigs(Integer maxNumThreads, String workerGroupKey) {
final Map<String, Object> configs = new HashMap<>();
Optional.ofNullable(maxNumThreads)
.ifPresent(val -> configs.put(KESTRA_WORKER_MAX_NUM_THREADS, val));
Optional.ofNullable(workerGroupKey)
.ifPresent(val -> configs.put(KESTRA_WORKER_GROUP_KEY, val));
if (!configs.isEmpty()) {
environment.addPropertySource("kestra-runtime", configs);
}
}
/** {@inheritDoc} **/
@Override
public void shutdown() {

View File

@@ -0,0 +1,27 @@
package io.kestra.core.contexts;
import io.micronaut.context.annotation.ConfigurationProperties;
import io.micronaut.context.annotation.EachProperty;
import io.micronaut.context.annotation.Parameter;
import io.micronaut.core.annotation.Nullable;
import lombok.Builder;
@Builder
@EachProperty("kestra.plugins.repositories")
public record MavenPluginRepositoryConfig(
@Parameter
String id,
String url,
@Nullable
BasicAuth basicAuth
) {
@Builder
@ConfigurationProperties("basic-auth")
public record BasicAuth(
String username,
String password
) {
}
}

View File

@@ -1,6 +1,7 @@
package io.kestra.core.docs;
import com.google.common.base.CaseFormat;
import io.kestra.core.models.Plugin;
import io.kestra.core.models.tasks.retrys.AbstractRetry;
import io.kestra.core.models.tasks.runners.TaskRunner;
import lombok.AllArgsConstructor;

View File

@@ -1,10 +1,9 @@
package io.kestra.core.docs;
import io.kestra.core.plugins.RegisteredPlugin;
import io.kestra.core.plugins.PluginClassAndMetadata;
import lombok.*;
import java.util.*;
import java.util.stream.Collectors;
@Getter
@EqualsAndHashCode
@@ -21,16 +20,18 @@ public class ClassPluginDocumentation<T> extends AbstractClassDocumentation<T> {
private Map<String, Object> outputsSchema;
@SuppressWarnings("unchecked")
private ClassPluginDocumentation(JsonSchemaGenerator jsonSchemaGenerator, RegisteredPlugin plugin, Class<? extends T> cls, Class<T> baseCls, String alias) {
super(jsonSchemaGenerator, cls, baseCls);
private ClassPluginDocumentation(JsonSchemaGenerator jsonSchemaGenerator, PluginClassAndMetadata<T> plugin, boolean allProperties) {
super(jsonSchemaGenerator, plugin.type(), allProperties ? null : plugin.baseClass());
// plugins metadata
this.cls = alias == null ? cls.getName() : alias;
Class<? extends T> cls = plugin.type();
this.cls = plugin.alias() == null ? cls.getName() : plugin.alias();
this.group = plugin.group();
this.docLicense = plugin.license();
this.pluginTitle = plugin.title();
this.icon = plugin.icon(cls);
if (alias != null) {
this.icon = plugin.icon();
if (plugin.alias() != null) {
replacement = cls.getName();
}
@@ -38,10 +39,10 @@ public class ClassPluginDocumentation<T> extends AbstractClassDocumentation<T> {
this.subGroup = cls.getPackageName().substring(this.group.length() + 1);
}
this.shortName = alias == null ? cls.getSimpleName() : alias.substring(alias.lastIndexOf('.') + 1);
this.shortName = plugin.alias() == null ? cls.getSimpleName() : plugin.alias().substring(plugin.alias().lastIndexOf('.') + 1);
// outputs
this.outputsSchema = jsonSchemaGenerator.outputs(baseCls, cls);
this.outputsSchema = jsonSchemaGenerator.outputs(allProperties ? null : plugin.baseClass(), cls);
if (this.outputsSchema.containsKey("$defs")) {
this.defs.putAll((Map<String, Object>) this.outputsSchema.get("$defs"));
@@ -67,17 +68,13 @@ public class ClassPluginDocumentation<T> extends AbstractClassDocumentation<T> {
.toList();
}
if (alias != null) {
if (plugin.alias() != null) {
this.deprecated = true;
}
}
public static <T> ClassPluginDocumentation<T> of(JsonSchemaGenerator jsonSchemaGenerator, RegisteredPlugin plugin, Class<? extends T> cls, Class<T> baseCls) {
return new ClassPluginDocumentation<>(jsonSchemaGenerator, plugin, cls, baseCls, null);
}
public static <T> ClassPluginDocumentation<T> of(JsonSchemaGenerator jsonSchemaGenerator, RegisteredPlugin plugin, Class<? extends T> cls, Class<T> baseCls, String alias) {
return new ClassPluginDocumentation<>(jsonSchemaGenerator, plugin, cls, baseCls, alias);
public static <T> ClassPluginDocumentation<T> of(JsonSchemaGenerator jsonSchemaGenerator, PluginClassAndMetadata<T> plugin, boolean allProperties) {
return new ClassPluginDocumentation<>(jsonSchemaGenerator, plugin, allProperties);
}
@AllArgsConstructor

View File

@@ -7,6 +7,7 @@ import io.kestra.core.models.tasks.logs.LogExporter;
import io.kestra.core.models.tasks.runners.TaskRunner;
import io.kestra.core.models.tasks.Task;
import io.kestra.core.models.triggers.AbstractTrigger;
import io.kestra.core.plugins.PluginClassAndMetadata;
import io.kestra.core.plugins.RegisteredPlugin;
import io.kestra.core.runners.pebble.Extension;
import io.kestra.core.runners.pebble.JsonWriter;
@@ -217,7 +218,15 @@ public class DocumentationGenerator {
private <T> List<Document> generate(RegisteredPlugin registeredPlugin, List<Class<? extends T>> cls, Class<T> baseCls, String type) {
return cls
.stream()
.map(r -> ClassPluginDocumentation.of(jsonSchemaGenerator, registeredPlugin, r, baseCls))
.map(pluginClass -> {
PluginClassAndMetadata<T> metadata = PluginClassAndMetadata.create(
registeredPlugin,
pluginClass,
baseCls,
null
);
return ClassPluginDocumentation.of(jsonSchemaGenerator, metadata, true);
})
.map(pluginDocumentation -> {
try {
return new Document(
@@ -247,15 +256,15 @@ public class DocumentationGenerator {
classPluginDocumentation.getCls() + ".md";
}
public static <T> String render(ClassPluginDocumentation<T> classPluginDocumentation) throws IOException {
public static String render(ClassPluginDocumentation<?> classPluginDocumentation) throws IOException {
return render("task", JacksonMapper.toMap(classPluginDocumentation));
}
public static <T> String render(AbstractClassDocumentation<T> classInputDocumentation) throws IOException {
public static String render(AbstractClassDocumentation classInputDocumentation) throws IOException {
return render("task", JacksonMapper.toMap(classInputDocumentation));
}
public static <T> String render(String templateName, Map<String, Object> vars) throws IOException {
public static String render(String templateName, Map<String, Object> vars) throws IOException {
String pebbleTemplate = IOUtils.toString(
Objects.requireNonNull(DocumentationGenerator.class.getClassLoader().getResourceAsStream("docs/" + templateName + ".peb")),
StandardCharsets.UTF_8

View File

@@ -0,0 +1,65 @@
package io.kestra.core.docs;
import io.kestra.core.models.dashboards.Dashboard;
import io.kestra.core.models.flows.Flow;
import io.kestra.core.models.flows.PluginDefault;
import io.kestra.core.models.tasks.Task;
import io.kestra.core.models.templates.Template;
import io.kestra.core.models.triggers.AbstractTrigger;
import jakarta.inject.Singleton;
import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
/**
* Service for getting schemas.
*/
@Singleton
public class JsonSchemaCache {
private final JsonSchemaGenerator jsonSchemaGenerator;
private final ConcurrentMap<CacheKey, Map<String, Object>> schemaCache = new ConcurrentHashMap<>();
private final Map<SchemaType, Class<?>> classesBySchemaType = new HashMap<>();
/**
* Creates a new {@link JsonSchemaCache} instance.
*
* @param jsonSchemaGenerator The {@link JsonSchemaGenerator}.
*/
public JsonSchemaCache(final JsonSchemaGenerator jsonSchemaGenerator) {
this.jsonSchemaGenerator = Objects.requireNonNull(jsonSchemaGenerator, "JsonSchemaGenerator cannot be null");
registerClassForType(SchemaType.FLOW, Flow.class);
registerClassForType(SchemaType.TEMPLATE, Template.class);
registerClassForType(SchemaType.TASK, Task.class);
registerClassForType(SchemaType.TRIGGER, AbstractTrigger.class);
registerClassForType(SchemaType.PLUGINDEFAULT, PluginDefault.class);
registerClassForType(SchemaType.DASHBOARD, Dashboard.class);
}
public Map<String, Object> getSchemaForType(final SchemaType type,
final boolean arrayOf) {
return schemaCache.computeIfAbsent(new CacheKey(type, arrayOf), (key) -> {
Class<?> cls = Optional.ofNullable(classesBySchemaType.get(type))
.orElseThrow(() -> new IllegalArgumentException("Cannot found schema for type '" + type + "'"));
return jsonSchemaGenerator.schemas(cls, arrayOf);
});
}
public void registerClassForType(final SchemaType type, final Class<?> clazz) {
classesBySchemaType.put(type, clazz);
}
public void clear() {
schemaCache.clear();
}
private record CacheKey(SchemaType type, boolean arrayOf) {
}
}

View File

@@ -4,6 +4,8 @@ import com.fasterxml.classmate.ResolvedType;
import com.fasterxml.classmate.members.HierarchicType;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.fasterxml.jackson.databind.node.TextNode;
@@ -47,10 +49,18 @@ import java.util.function.Predicate;
import java.util.stream.Collectors;
import java.util.stream.StreamSupport;
import static io.kestra.core.serializers.JacksonMapper.MAP_TYPE_REFERENCE;
@Singleton
public class JsonSchemaGenerator {
private static final List<Class<?>> TYPES_RESOLVED_AS_STRING = List.of(Duration.class, LocalTime.class, LocalDate.class, LocalDateTime.class, ZonedDateTime.class, OffsetDateTime.class, OffsetTime.class);
private static final ObjectMapper MAPPER = JacksonMapper.ofJson().copy()
.configure(SerializationFeature.WRITE_DURATIONS_AS_TIMESTAMPS, false);
private static final ObjectMapper YAML_MAPPER = JacksonMapper.ofYaml().copy()
.configure(SerializationFeature.WRITE_DURATIONS_AS_TIMESTAMPS, false);
private final PluginRegistry pluginRegistry;
@Inject
@@ -64,13 +74,21 @@ public class JsonSchemaGenerator {
return this.schemas(cls, false);
}
private void replaceOneOfWithAnyOf(ObjectNode objectNode) {
objectNode.findParents("oneOf").forEach(jsonNode -> {
if (jsonNode instanceof ObjectNode oNode) {
oNode.set("anyOf", oNode.remove("oneOf"));
}
});
}
public <T> Map<String, Object> schemas(Class<? extends T> cls, boolean arrayOf) {
SchemaGeneratorConfigBuilder builder = new SchemaGeneratorConfigBuilder(
SchemaVersion.DRAFT_7,
OptionPreset.PLAIN_JSON
);
this.build(builder,true);
this.build(builder, true);
SchemaGeneratorConfig schemaGeneratorConfig = builder.build();
@@ -80,42 +98,69 @@ public class JsonSchemaGenerator {
if (arrayOf) {
objectNode.put("type", "array");
}
replaceAnyOfWithOneOf(objectNode);
pullOfDefaultFromOneOf(objectNode);
replaceOneOfWithAnyOf(objectNode);
pullDocumentationAndDefaultFromAnyOf(objectNode);
removeRequiredOnPropsWithDefaults(objectNode);
return JacksonMapper.toMap(objectNode);
return MAPPER.convertValue(objectNode, MAP_TYPE_REFERENCE);
} catch (IllegalArgumentException e) {
throw new IllegalArgumentException("Unable to generate jsonschema for '" + cls.getName() + "'", e);
}
}
private void replaceAnyOfWithOneOf(ObjectNode objectNode) {
objectNode.findParents("anyOf").forEach(jsonNode -> {
if (jsonNode instanceof ObjectNode oNode) {
oNode.set("oneOf", oNode.remove("anyOf"));
private void removeRequiredOnPropsWithDefaults(ObjectNode objectNode) {
objectNode.findParents("required").forEach(jsonNode -> {
if (jsonNode instanceof ObjectNode clazzSchema && clazzSchema.get("required") instanceof ArrayNode requiredPropsNode && clazzSchema.get("properties") instanceof ObjectNode properties) {
List<String> requiredFieldValues = StreamSupport.stream(requiredPropsNode.spliterator(), false)
.map(JsonNode::asText)
.toList();
properties.fields().forEachRemaining(e -> {
int indexInRequiredArray = requiredFieldValues.indexOf(e.getKey());
if (indexInRequiredArray != -1 && e.getValue() instanceof ObjectNode valueNode && valueNode.has("default")) {
requiredPropsNode.remove(indexInRequiredArray);
}
});
if (requiredPropsNode.isEmpty()) {
clazzSchema.remove("required");
}
}
});
}
// This hack exists because for Property we generate a oneOf for properties that are not strings.
// By default, the 'default' is in each oneOf which Monaco editor didn't take into account.
// So, we pull off the 'default' from any of the oneOf to the parent.
private void pullOfDefaultFromOneOf(ObjectNode objectNode) {
objectNode.findParents("oneOf").forEach(jsonNode -> {
// This hack exists because for Property we generate a anyOf for properties that are not strings.
// By default, the 'default' is in each anyOf which Monaco editor didn't take into account.
// So, we pull off the 'default' from any of the anyOf to the parent.
// same thing for documentation fields: 'title', 'description', '$deprecated'
private void pullDocumentationAndDefaultFromAnyOf(ObjectNode objectNode) {
objectNode.findParents("anyOf").forEach(jsonNode -> {
if (jsonNode instanceof ObjectNode oNode) {
JsonNode oneOf = oNode.get("oneOf");
if (oneOf instanceof ArrayNode arrayNode) {
JsonNode anyOf = oNode.get("anyOf");
if (anyOf instanceof ArrayNode arrayNode) {
Iterator<JsonNode> it = arrayNode.elements();
JsonNode defaultNode = null;
while (it.hasNext() && defaultNode == null) {
var nodesToPullUp = new HashMap<String, Optional<JsonNode>>(Map.ofEntries(
Map.entry("default", Optional.empty()),
Map.entry("title", Optional.empty()),
Map.entry("description", Optional.empty()),
Map.entry("$deprecated", Optional.empty())
));
// find nodes to pull up
while (it.hasNext() && nodesToPullUp.containsValue(Optional.<JsonNode>empty())) {
JsonNode next = it.next();
if (next instanceof ObjectNode nextAsObj) {
defaultNode = nextAsObj.get("default");
nodesToPullUp.entrySet().stream()
.filter(node -> node.getValue().isEmpty())
.forEach(node -> node
.setValue(Optional.ofNullable(
nextAsObj.get(node.getKey())
)));
}
}
if (defaultNode != null) {
oNode.set("default", defaultNode);
}
// create nodes on parent
nodesToPullUp.entrySet().stream()
.filter(node -> node.getValue().isPresent())
.forEach(node -> oNode.set(node.getKey(), node.getValue().get()));
}
}
});
@@ -141,7 +186,7 @@ public class JsonSchemaGenerator {
try {
sb.append("Default value is : `")
.append(JacksonMapper.ofYaml().writeValueAsString(collectedTypeAttributes.get("default")).trim())
.append(YAML_MAPPER.writeValueAsString(collectedTypeAttributes.get("default")).trim())
.append("`");
} catch (JsonProcessingException ignored) {
@@ -181,6 +226,7 @@ public class JsonSchemaGenerator {
}
protected void build(SchemaGeneratorConfigBuilder builder, boolean draft7) {
// builder.withObjectMapper(builder.getObjectMapper().configure(SerializationFeature.WRITE_DURATIONS_AS_TIMESTAMPS, false));
builder
.with(new JakartaValidationModule(
JakartaValidationOption.NOT_NULLABLE_METHOD_IS_REQUIRED,
@@ -252,11 +298,11 @@ public class JsonSchemaGenerator {
TypeContext context = target.getContext();
Class<?> erasedType = javaType.getTypeParameters().getFirst().getErasedType();
if(String.class.isAssignableFrom(erasedType)) {
if (String.class.isAssignableFrom(erasedType)) {
return List.of(
context.resolve(String.class)
);
} else if(Object.class.equals(erasedType)) {
} else if (Object.class.equals(erasedType)) {
return List.of(
context.resolve(Object.class)
);
@@ -296,6 +342,9 @@ public class JsonSchemaGenerator {
if (pluginPropertyAnnotation.beta()) {
memberAttributes.put("$beta", true);
}
if (pluginPropertyAnnotation.internalStorageURI()) {
memberAttributes.put("$internalStorageURI", true);
}
}
Schema schema = member.getAnnotationConsideringFieldAndGetter(Schema.class);
@@ -363,7 +412,7 @@ public class JsonSchemaGenerator {
// handle deprecated tasks
Schema schema = scope.getType().getErasedType().getAnnotation(Schema.class);
Deprecated deprecated = scope.getType().getErasedType().getAnnotation(Deprecated.class);
if ((schema != null && schema.deprecated()) || deprecated != null ) {
if ((schema != null && schema.deprecated()) || deprecated != null) {
collectedTypeAttributes.put("$deprecated", "true");
}
});
@@ -388,7 +437,7 @@ public class JsonSchemaGenerator {
});
// Subtype resolver for all plugins
if(builder.build().getSchemaVersion() != SchemaVersion.DRAFT_2019_09) {
if (builder.build().getSchemaVersion() != SchemaVersion.DRAFT_2019_09) {
builder.forTypesInGeneral()
.withSubtypeResolver((declaredType, context) -> {
TypeContext typeContext = context.getTypeContext();
@@ -577,7 +626,7 @@ public class JsonSchemaGenerator {
if (property.has("allOf")) {
for (Iterator<JsonNode> it = property.get("allOf").elements(); it.hasNext(); ) {
JsonNode child = it.next();
if(child.has("default")) {
if (child.has("default")) {
return true;
}
}
@@ -591,7 +640,7 @@ public class JsonSchemaGenerator {
OptionPreset.PLAIN_JSON
);
this.build(builder,false);
this.build(builder, false);
// we don't return base properties unless specified with @PluginProperty
builder
@@ -603,10 +652,11 @@ public class JsonSchemaGenerator {
SchemaGenerator generator = new SchemaGenerator(schemaGeneratorConfig);
try {
ObjectNode objectNode = generator.generateSchema(cls);
replaceAnyOfWithOneOf(objectNode);
pullOfDefaultFromOneOf(objectNode);
replaceOneOfWithAnyOf(objectNode);
pullDocumentationAndDefaultFromAnyOf(objectNode);
removeRequiredOnPropsWithDefaults(objectNode);
return JacksonMapper.toMap(extractMainRef(objectNode));
return MAPPER.convertValue(extractMainRef(objectNode), MAP_TYPE_REFERENCE);
} catch (IllegalArgumentException e) {
throw new IllegalArgumentException("Unable to generate jsonschema for '" + cls.getName() + "'", e);
}
@@ -714,7 +764,8 @@ public class JsonSchemaGenerator {
field.setAccessible(true);
return field.invoke(instance);
} catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException | IllegalArgumentException ignored) {
} catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException |
IllegalArgumentException ignored) {
}
@@ -723,7 +774,8 @@ public class JsonSchemaGenerator {
field.setAccessible(true);
return field.invoke(instance);
} catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException | IllegalArgumentException ignored) {
} catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException |
IllegalArgumentException ignored) {
}

View File

@@ -7,6 +7,7 @@ import lombok.Data;
import lombok.NoArgsConstructor;
import java.util.*;
import java.util.function.Predicate;
import java.util.stream.Collectors;
import static java.util.function.Predicate.not;
@@ -50,9 +51,12 @@ public class Plugin {
if (subgroup == null) {
plugin.title = registeredPlugin.title();
} else {
subGroupInfos = registeredPlugin.allClass().stream().filter(c -> c.getName().contains(subgroup)).map(clazz -> clazz.getPackage().getDeclaredAnnotation(PluginSubGroup.class)).toList().getFirst();
plugin.title = !subGroupInfos.title().isEmpty() ? subGroupInfos.title() : subgroup.substring(subgroup.lastIndexOf('.') + 1);;
subGroupInfos = registeredPlugin.allClass().stream()
.filter(c -> c.getPackageName().contains(subgroup))
.min(Comparator.comparingInt(a -> a.getPackageName().length()))
.map(clazz -> clazz.getPackage().getDeclaredAnnotation(PluginSubGroup.class))
.orElseThrow();
plugin.title = !subGroupInfos.title().isEmpty() ? subGroupInfos.title() : subgroup.substring(subgroup.lastIndexOf('.') + 1);
}
plugin.group = registeredPlugin.group();
plugin.description = subGroupInfos != null && !subGroupInfos.description().isEmpty() ? subGroupInfos.description() : registeredPlugin.description();
@@ -74,27 +78,28 @@ public class Plugin {
plugin.categories = subGroupInfos != null ?
Arrays.stream(subGroupInfos.categories()).toList() :
registeredPlugin
.allClass()
.stream()
.map(clazz -> clazz.getPackage().getDeclaredAnnotation(PluginSubGroup.class))
.filter(Objects::nonNull)
.flatMap(r -> Arrays.stream(r.categories()))
.distinct()
.toList();
.allClass()
.stream()
.map(clazz -> clazz.getPackage().getDeclaredAnnotation(PluginSubGroup.class))
.filter(Objects::nonNull)
.flatMap(r -> Arrays.stream(r.categories()))
.distinct()
.toList();
plugin.subGroup = subgroup;
plugin.tasks = filterAndGetClassName(registeredPlugin.getTasks(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
plugin.triggers = filterAndGetClassName(registeredPlugin.getTriggers(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
plugin.conditions = filterAndGetClassName(registeredPlugin.getConditions(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
plugin.storages = filterAndGetClassName(registeredPlugin.getStorages(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
plugin.secrets = filterAndGetClassName(registeredPlugin.getSecrets(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
plugin.taskRunners = filterAndGetClassName(registeredPlugin.getTaskRunners(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
plugin.apps = filterAndGetClassName(registeredPlugin.getApps(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
plugin.appBlocks = filterAndGetClassName(registeredPlugin.getAppBlocks(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
plugin.charts = filterAndGetClassName(registeredPlugin.getCharts(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
plugin.dataFilters = filterAndGetClassName(registeredPlugin.getDataFilters(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
plugin.logExporters = filterAndGetClassName(registeredPlugin.getLogExporters(), includeDeprecated).stream().filter(c -> subgroup == null || c.startsWith(subgroup)).toList();
Predicate<Class<?>> packagePredicate = c -> subgroup == null || c.getPackageName().equals(subgroup);
plugin.tasks = filterAndGetClassName(registeredPlugin.getTasks(), includeDeprecated, packagePredicate).stream().toList();
plugin.triggers = filterAndGetClassName(registeredPlugin.getTriggers(), includeDeprecated, packagePredicate).stream().toList();
plugin.conditions = filterAndGetClassName(registeredPlugin.getConditions(), includeDeprecated, packagePredicate).stream().toList();
plugin.storages = filterAndGetClassName(registeredPlugin.getStorages(), includeDeprecated, packagePredicate).stream().toList();
plugin.secrets = filterAndGetClassName(registeredPlugin.getSecrets(), includeDeprecated, packagePredicate).stream().toList();
plugin.taskRunners = filterAndGetClassName(registeredPlugin.getTaskRunners(), includeDeprecated, packagePredicate).stream().toList();
plugin.apps = filterAndGetClassName(registeredPlugin.getApps(), includeDeprecated, packagePredicate).stream().toList();
plugin.appBlocks = filterAndGetClassName(registeredPlugin.getAppBlocks(), includeDeprecated, packagePredicate).stream().toList();
plugin.charts = filterAndGetClassName(registeredPlugin.getCharts(), includeDeprecated, packagePredicate).stream().toList();
plugin.dataFilters = filterAndGetClassName(registeredPlugin.getDataFilters(), includeDeprecated, packagePredicate).stream().toList();
plugin.logExporters = filterAndGetClassName(registeredPlugin.getLogExporters(), includeDeprecated, packagePredicate).stream().toList();
return plugin;
}
@@ -103,15 +108,16 @@ public class Plugin {
* Filters the given list of class all internal Plugin, as well as, all legacy org.kestra classes.
* Those classes are only filtered from the documentation to ensure backward compatibility.
*
* @param list The list of classes?
* @param list The list of classes?
* @param includeDeprecated whether to include deprecated plugins or not
* @return a filtered streams.
* @return a filtered streams.
*/
private static List<String> filterAndGetClassName(final List<? extends Class<?>> list, boolean includeDeprecated) {
private static List<String> filterAndGetClassName(final List<? extends Class<?>> list, boolean includeDeprecated, Predicate<Class<?>> clazzFilter) {
return list
.stream()
.filter(not(io.kestra.core.models.Plugin::isInternal))
.filter(p -> includeDeprecated || !io.kestra.core.models.Plugin.isDeprecated(p))
.filter(clazzFilter)
.map(Class::getName)
.filter(c -> !c.startsWith("org.kestra."))
.toList();

View File

@@ -1,11 +1,20 @@
package io.kestra.core.docs;
import com.fasterxml.jackson.annotation.JsonCreator;
import io.kestra.core.utils.Enums;
public enum SchemaType {
flow,
template,
task,
trigger,
plugindefault,
apps,
dashboard
FLOW,
TEMPLATE,
TASK,
TRIGGER,
PLUGINDEFAULT,
APPS,
DASHBOARD;
@JsonCreator
public static SchemaType fromString(final String value) {
return Enums.getForNameIgnoreCase(value, SchemaType.class);
}
}

Some files were not shown because too many files have changed in this diff Show More