Compare commits

..

19 Commits

Author SHA1 Message Date
Miloš Paunović
cb31e48f4f chore(core): improve file organization in .github folder (#13836) 2025-12-25 16:00:11 +01:00
Miloš Paunović
a3f96a2741 chore(core): make left menu appear as overlay on small screens (#13834)
Closes https://github.com/kestra-io/kestra/issues/1425.
Closes https://github.com/kestra-io/kestra/issues/13502.
Closes https://github.com/kestra-io/kestra/issues/13503.
Closes https://github.com/kestra-io/kestra/issues/13504.
Closes https://github.com/kestra-io/kestra/issues/13505.
Closes https://github.com/kestra-io/kestra/issues/13506.
Closes https://github.com/kestra-io/kestra/issues/13507.
Closes https://github.com/kestra-io/kestra/issues/13508.
Closes https://github.com/kestra-io/kestra/issues/13510.
Closes https://github.com/kestra-io/kestra/issues/13511.
Closes https://github.com/kestra-io/kestra/issues/13512.
Closes https://github.com/kestra-io/kestra/issues/13513.
Closes https://github.com/kestra-io/kestra/issues/13514.
Closes https://github.com/kestra-io/kestra/issues/13516.
2025-12-25 14:29:56 +01:00
Miloš Paunović
5ca6fa8d77 fix(secrets): mark secret field required during creation (#13833)
Closes https://github.com/kestra-io/kestra-ee/issues/6209.
2025-12-25 10:54:55 +01:00
Miloš Paunović
a3a206f3c4 chore(core): polishing of templated blueprints ui (#13806)
Related to https://github.com/kestra-io/kestra-ee/pull/6201.

Closes https://github.com/kestra-io/kestra-ee/issues/6179.
2025-12-25 09:06:49 +01:00
aflahaa
31f1e505e3 refactor(core): remove usage of unnecessary i18n composable (#13826)
Closes https://github.com/kestra-io/kestra/issues/13350.

Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2025-12-24 12:56:51 +01:00
YannC
75e0c1d11f fix: updated the package-lock.json after dependencies upgrade (#13825) 2025-12-24 10:43:09 +01:00
dependabot[bot]
4cf883877d build(deps): bump the minor group in /ui with 4 updates (#13821)
Bumps the minor group in /ui with 4 updates: [element-plus](https://github.com/element-plus/element-plus), [posthog-js](https://github.com/PostHog/posthog-js), [rolldown-vite](https://github.com/vitejs/rolldown-vite/tree/HEAD/packages/vite) and [vue-tsc](https://github.com/vuejs/language-tools/tree/HEAD/packages/tsc).


Updates `element-plus` from 2.12.0 to 2.13.0
- [Release notes](https://github.com/element-plus/element-plus/releases)
- [Changelog](https://github.com/element-plus/element-plus/blob/dev/CHANGELOG.en-US.md)
- [Commits](https://github.com/element-plus/element-plus/compare/2.12.0...2.13.0)

Updates `posthog-js` from 1.308.0 to 1.310.1
- [Release notes](https://github.com/PostHog/posthog-js/releases)
- [Changelog](https://github.com/PostHog/posthog-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/PostHog/posthog-js/compare/posthog-js@1.308.0...posthog-js@1.310.1)

Updates `rolldown-vite` from 7.2.11 to 7.3.0
- [Release notes](https://github.com/vitejs/rolldown-vite/releases)
- [Changelog](https://github.com/vitejs/rolldown-vite/blob/rolldown-vite/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/rolldown-vite/commits/v7.3.0/packages/vite)

Updates `vue-tsc` from 3.1.8 to 3.2.1
- [Release notes](https://github.com/vuejs/language-tools/releases)
- [Changelog](https://github.com/vuejs/language-tools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vuejs/language-tools/commits/v3.2.1/packages/tsc)

---
updated-dependencies:
- dependency-name: element-plus
  dependency-version: 2.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: posthog-js
  dependency-version: 1.310.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: rolldown-vite
  dependency-version: 7.3.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: vue-tsc
  dependency-version: 3.2.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-12-24 08:53:14 +01:00
dependabot[bot]
87b1e8fb01 build(deps): bump the patch group in /ui with 7 updates (#13822)
Bumps the patch group in /ui with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [@vue-flow/core](https://github.com/bcakmakoglu/vue-flow/tree/HEAD/packages/core) | `1.48.0` | `1.48.1` |
| [vue](https://github.com/vuejs/core) | `3.5.25` | `3.5.26` |
| [vue-i18n](https://github.com/intlify/vue-i18n/tree/HEAD/packages/vue-i18n) | `11.2.2` | `11.2.7` |
| [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) | `8.50.0` | `8.50.1` |
| [@vitejs/plugin-vue-jsx](https://github.com/vitejs/vite-plugin-vue/tree/HEAD/packages/plugin-vue-jsx) | `5.1.2` | `5.1.3` |
| [sass](https://github.com/sass/dart-sass) | `1.97.0` | `1.97.1` |
| [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint) | `8.50.0` | `8.50.1` |


Updates `@vue-flow/core` from 1.48.0 to 1.48.1
- [Release notes](https://github.com/bcakmakoglu/vue-flow/releases)
- [Changelog](https://github.com/bcakmakoglu/vue-flow/blob/master/packages/core/CHANGELOG.md)
- [Commits](https://github.com/bcakmakoglu/vue-flow/commits/@vue-flow/core@1.48.1/packages/core)

Updates `vue` from 3.5.25 to 3.5.26
- [Release notes](https://github.com/vuejs/core/releases)
- [Changelog](https://github.com/vuejs/core/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vuejs/core/compare/v3.5.25...v3.5.26)

Updates `vue-i18n` from 11.2.2 to 11.2.7
- [Release notes](https://github.com/intlify/vue-i18n/releases)
- [Changelog](https://github.com/intlify/vue-i18n/blob/master/CHANGELOG.md)
- [Commits](https://github.com/intlify/vue-i18n/commits/v11.2.7/packages/vue-i18n)

Updates `@typescript-eslint/parser` from 8.50.0 to 8.50.1
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.50.1/packages/parser)

Updates `@vitejs/plugin-vue-jsx` from 5.1.2 to 5.1.3
- [Release notes](https://github.com/vitejs/vite-plugin-vue/releases)
- [Changelog](https://github.com/vitejs/vite-plugin-vue/blob/main/packages/plugin-vue-jsx/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite-plugin-vue/commits/plugin-vue@5.1.3/packages/plugin-vue-jsx)

Updates `sass` from 1.97.0 to 1.97.1
- [Release notes](https://github.com/sass/dart-sass/releases)
- [Changelog](https://github.com/sass/dart-sass/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sass/dart-sass/compare/1.97.0...1.97.1)

Updates `typescript-eslint` from 8.50.0 to 8.50.1
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.50.1/packages/typescript-eslint)

---
updated-dependencies:
- dependency-name: "@vue-flow/core"
  dependency-version: 1.48.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch
- dependency-name: vue
  dependency-version: 3.5.26
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch
- dependency-name: vue-i18n
  dependency-version: 11.2.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch
- dependency-name: "@typescript-eslint/parser"
  dependency-version: 8.50.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: patch
- dependency-name: "@vitejs/plugin-vue-jsx"
  dependency-version: 5.1.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: patch
- dependency-name: sass
  dependency-version: 1.97.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: patch
- dependency-name: typescript-eslint
  dependency-version: 8.50.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-24 08:51:19 +01:00
dependabot[bot]
b5c6101090 build(deps): bump the build group in /ui with 6 updates (#13820)
---
updated-dependencies:
- dependency-name: "@rollup/rollup-darwin-arm64"
  dependency-version: 4.54.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: build
- dependency-name: "@rollup/rollup-darwin-x64"
  dependency-version: 4.54.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: build
- dependency-name: "@rollup/rollup-linux-x64-gnu"
  dependency-version: 4.54.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: build
- dependency-name: "@swc/core-darwin-arm64"
  dependency-version: 1.15.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: build
- dependency-name: "@swc/core-darwin-x64"
  dependency-version: 1.15.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: build
- dependency-name: "@swc/core-linux-x64-gnu"
  dependency-version: 1.15.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: build
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-12-24 08:45:45 +01:00
dependabot[bot]
e4323728d6 build(deps-dev): bump storybook from 9.1.16 to 9.1.17 in /ui (#13761)
Bumps [storybook](https://github.com/storybookjs/storybook/tree/HEAD/code/core) from 9.1.16 to 9.1.17.
- [Release notes](https://github.com/storybookjs/storybook/releases)
- [Changelog](https://github.com/storybookjs/storybook/blob/next/CHANGELOG.md)
- [Commits](https://github.com/storybookjs/storybook/commits/v9.1.17/code/core)

---
updated-dependencies:
- dependency-name: storybook
  dependency-version: 9.1.17
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-24 08:44:29 +01:00
Roman Acevedo
5495971ecf test: add test env by default to be marked as test in ThreadUncaughtExceptionHandler.uncaughtException() 2025-12-23 22:18:48 +01:00
Roman Acevedo
dec1ee4272 test(cli): try to unflaky PluginDocCommandTest.run 2025-12-23 20:23:30 +01:00
Roman Acevedo
69cc6b2715 build(gradle): insure test report is always generated 2025-12-23 20:23:30 +01:00
Barthélémy Ledoux
431b4ccdb9 fix: make reorder of tests work (#13774)
* rename function

* fix: make reorder of tests work

---------

Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-12-23 17:11:54 +01:00
Georg Traar
e9207a6f53 feat(ui): update onboarding CTA to product tour (#13803)
* feat(ui): update onboarding CTA to product tour

Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.

Co-authored-by: Georg Traar <gtraar@kestra.io>
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-12-23 16:13:48 +01:00
Alankar Jagtap
419c1041d5 refactor(core): import moment as a library directly into the component (#13800)
Closes https://github.com/kestra-io/kestra/issues/12953.

Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2025-12-23 13:46:45 +01:00
Karan Suresh
02cd5efb05 refactor(core): remove usage of unnecessary i18n composable (#13804)
Closes https://github.com/kestra-io/kestra/issues/13261.

Signed-off-by: Iam-karan-suresh <karansuresh.info@gmail.com>
Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2025-12-23 13:37:15 +01:00
github-actions[bot]
2d549940c4 chore(core): localize to languages other than english (#13809)
Extended localization support by adding translations for multiple languages using English as the base. This enhances accessibility and usability for non-English-speaking users while keeping English as the source reference.

Co-authored-by: GitHub Action <actions@github.com>
2025-12-23 13:24:58 +01:00
Barthélémy Ledoux
97e138fbae fix: use injection instead of store (#13777)
Co-authored-by: Miloš Paunović <paun992@hotmail.com>
2025-12-23 13:23:14 +01:00
84 changed files with 1925 additions and 3242 deletions

View File

@@ -63,9 +63,9 @@ You can also build it from a terminal using `./gradlew build`, the Gradle wrappe
- Configure the following environment variables:
- `MICRONAUT_ENVIRONMENTS`: can be set to any string and will load a custom configuration file in `cli/src/main/resources/application-{env}.yml`.
- `KESTRA_PLUGINS_PATH`: is the path where you will save plugins as Jar and will be load on startup.
- See the screenshot below for an example: ![Intellij IDEA Configuration ](run-app.png)
- See the screenshot below for an example: ![Intellij IDEA Configuration ](./assets/run-app.png)
- If you encounter **JavaScript memory heap out** error during startup, configure `NODE_OPTIONS` environment variable with some large value.
- Example `NODE_OPTIONS: --max-old-space-size=4096` or `NODE_OPTIONS: --max-old-space-size=8192` ![Intellij IDEA Configuration ](node_option_env_var.png)
- Example `NODE_OPTIONS: --max-old-space-size=4096` or `NODE_OPTIONS: --max-old-space-size=8192` ![Intellij IDEA Configuration ](./assets/node_option_env_var.png)
- The server starts by default on port 8080 and is reachable on `http://localhost:8080`
If you want to launch all tests, you need Python and some packages installed on your machine, on Ubuntu you can install them with:

View File

Before

Width:  |  Height:  |  Size: 130 KiB

After

Width:  |  Height:  |  Size: 130 KiB

View File

Before

Width:  |  Height:  |  Size: 210 KiB

After

Width:  |  Height:  |  Size: 210 KiB

View File

@@ -222,6 +222,7 @@ subprojects {subProj ->
def commonTestConfig = { Test t ->
t.ignoreFailures = true
t.finalizedBy jacocoTestReport
// set Xmx for test workers
t.maxHeapSize = '4g'

View File

@@ -4,7 +4,6 @@ import io.micronaut.configuration.picocli.PicocliRunner;
import io.micronaut.context.ApplicationContext;
import io.micronaut.context.env.Environment;
import org.apache.commons.io.FileUtils;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import java.io.File;
@@ -15,7 +14,6 @@ import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.Objects;
import java.util.stream.Collectors;
import static org.assertj.core.api.Assertions.assertThat;
@@ -25,7 +23,8 @@ class PluginDocCommandTest {
@Test
void run() throws IOException, URISyntaxException {
Path pluginsPath = Files.createTempDirectory(PluginListCommandTest.class.getSimpleName());
var testDirectoryName = PluginListCommandTest.class.getSimpleName();
Path pluginsPath = Files.createTempDirectory(testDirectoryName + "_pluginsPath_");
pluginsPath.toFile().deleteOnExit();
FileUtils.copyFile(
@@ -34,7 +33,7 @@ class PluginDocCommandTest {
new File(URI.create("file://" + pluginsPath.toAbsolutePath() + "/" + PLUGIN_TEMPLATE_TEST))
);
Path docPath = Files.createTempDirectory(PluginInstallCommandTest.class.getSimpleName());
Path docPath = Files.createTempDirectory(testDirectoryName + "_docPath_");
docPath.toFile().deleteOnExit();
try (ApplicationContext ctx = ApplicationContext.run(Environment.CLI, Environment.TEST)) {
@@ -43,9 +42,9 @@ class PluginDocCommandTest {
List<Path> files = Files.list(docPath).toList();
assertThat(files.size()).isEqualTo(1);
assertThat(files.getFirst().getFileName().toString()).isEqualTo("plugin-template-test");
var directory = files.getFirst().toFile();
assertThat(files.stream().map(path -> path.getFileName().toString())).contains("plugin-template-test");
// don't know why, but sometimes there is an addition "plugin-notifications" directory present
var directory = files.stream().filter(path -> "plugin-template-test".equals(path.getFileName().toString())).findFirst().get().toFile();
assertThat(directory.isDirectory()).isTrue();
assertThat(directory.listFiles().length).isEqualTo(3);

View File

@@ -24,9 +24,6 @@ dependencies {
// reactor
api "io.projectreactor:reactor-core"
// awaitility
api "org.awaitility:awaitility"
// micronaut
api "io.micronaut.data:micronaut-data-model"
implementation "io.micronaut:micronaut-http-server-netty"

View File

@@ -2,7 +2,6 @@ package io.kestra.core.docs;
import com.fasterxml.classmate.ResolvedType;
import com.fasterxml.classmate.members.HierarchicType;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
@@ -23,8 +22,6 @@ import com.github.victools.jsonschema.module.swagger2.Swagger2Module;
import com.google.common.collect.ImmutableMap;
import io.kestra.core.models.annotations.Plugin;
import io.kestra.core.models.annotations.PluginProperty;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.assets.CustomAsset;
import io.kestra.core.models.conditions.Condition;
import io.kestra.core.models.conditions.ScheduleCondition;
import io.kestra.core.models.dashboards.DataFilter;
@@ -66,7 +63,7 @@ import static io.kestra.core.serializers.JacksonMapper.MAP_TYPE_REFERENCE;
@Singleton
@Slf4j
public class JsonSchemaGenerator {
private static final List<Class<?>> TYPES_RESOLVED_AS_STRING = List.of(Duration.class, LocalTime.class, LocalDate.class, LocalDateTime.class, ZonedDateTime.class, OffsetDateTime.class, OffsetTime.class);
private static final List<Class<?>> SUBTYPE_RESOLUTION_EXCLUSION_FOR_PLUGIN_SCHEMA = List.of(Task.class, AbstractTrigger.class);
@@ -279,10 +276,10 @@ public class JsonSchemaGenerator {
.with(Option.DEFINITION_FOR_MAIN_SCHEMA)
.with(Option.PLAIN_DEFINITION_KEYS)
.with(Option.ALLOF_CLEANUP_AT_THE_END);
// HACK: Registered a custom JsonUnwrappedDefinitionProvider prior to the JacksonModule
// to be able to return an CustomDefinition with an empty node when the ResolvedType can't be found.
builder.forTypesInGeneral().withCustomDefinitionProvider(new JsonUnwrappedDefinitionProvider() {
builder.forTypesInGeneral().withCustomDefinitionProvider(new JsonUnwrappedDefinitionProvider(){
@Override
public CustomDefinition provideCustomSchemaDefinition(ResolvedType javaType, SchemaGenerationContext context) {
try {
@@ -324,7 +321,7 @@ public class JsonSchemaGenerator {
// inline some type
builder.forTypesInGeneral()
.withCustomDefinitionProvider(new CustomDefinitionProviderV2() {
@Override
public CustomDefinition provideCustomSchemaDefinition(ResolvedType javaType, SchemaGenerationContext context) {
if (javaType.isInstanceOf(Map.class) || javaType.isInstanceOf(Enum.class)) {
@@ -592,31 +589,11 @@ public class JsonSchemaGenerator {
// The `const` property is used by editors for auto-completion based on that schema.
builder.forTypesInGeneral().withTypeAttributeOverride((collectedTypeAttributes, scope, context) -> {
final Class<?> pluginType = scope.getType().getErasedType();
Plugin pluginAnnotation = pluginType.getAnnotation(Plugin.class);
if (pluginAnnotation != null) {
if (pluginType.getAnnotation(Plugin.class) != null) {
ObjectNode properties = (ObjectNode) collectedTypeAttributes.get("properties");
if (properties != null) {
String typeConst = pluginType.getName();
// This is needed so that assets can have arbitrary types while still being able to be identified as assets.
if (pluginType == CustomAsset.class) {
properties.set("type", context.getGeneratorConfig().createObjectNode()
.put("type", "string")
);
return;
}
if (Asset.class.isAssignableFrom(pluginType)) {
// For Asset types, we want to be able to use a simple-string type. Convention is that first alias is that string type.
typeConst = pluginAnnotation.aliases().length > 0 ? pluginAnnotation.aliases()[0] : pluginType.getName();
Arrays.stream(pluginType.getDeclaredMethods())
.filter(m -> m.isAnnotationPresent(JsonProperty.class))
.forEach(m -> properties.set(m.getAnnotation(JsonProperty.class).value(), context.getGeneratorConfig().createObjectNode()
.put("type", "string")
));
}
properties.set("type", context.getGeneratorConfig().createObjectNode()
.put("const", typeConst)
.put("const", pluginType.getName())
);
}
}
@@ -787,14 +764,6 @@ public class JsonSchemaGenerator {
consumer.accept(typeContext.resolve(clz));
}
}).toList();
} else if (declaredType.getErasedType() == Asset.class) {
return getRegisteredPlugins()
.stream()
.flatMap(registeredPlugin -> registeredPlugin.getAssets().stream())
.filter(p -> allowedPluginTypes.isEmpty() || allowedPluginTypes.contains(p.getName()))
.filter(Predicate.not(io.kestra.core.models.Plugin::isInternal))
.map(typeContext::resolve)
.toList();
}
return null;

View File

@@ -103,48 +103,12 @@ public record QueryFilter(
return List.of(Op.EQUALS, Op.NOT_EQUALS, Op.IN, Op.NOT_IN, Op.CONTAINS);
}
},
METADATA("metadata") {
@Override
public List<Op> supportedOp() {
return List.of(Op.EQUALS, Op.NOT_EQUALS, Op.IN, Op.NOT_IN, Op.CONTAINS);
}
},
FLOW_ID("flowId") {
@Override
public List<Op> supportedOp() {
return List.of(Op.EQUALS, Op.NOT_EQUALS, Op.CONTAINS, Op.STARTS_WITH, Op.ENDS_WITH, Op.REGEX, Op.IN, Op.NOT_IN, Op.PREFIX);
}
},
FLOW_REVISION("flowRevision") {
@Override
public List<Op> supportedOp() {
return List.of(Op.EQUALS, Op.NOT_EQUALS, Op.IN, Op.NOT_IN);
}
},
ID("id") {
@Override
public List<Op> supportedOp() {
return List.of(Op.EQUALS, Op.NOT_EQUALS, Op.CONTAINS, Op.STARTS_WITH, Op.ENDS_WITH, Op.REGEX);
}
},
ASSET_ID("assetId") {
@Override
public List<Op> supportedOp() {
return List.of(Op.EQUALS, Op.NOT_EQUALS, Op.CONTAINS, Op.STARTS_WITH, Op.ENDS_WITH, Op.REGEX);
}
},
TYPE("type") {
@Override
public List<Op> supportedOp() {
return List.of(Op.EQUALS, Op.NOT_EQUALS, Op.CONTAINS, Op.STARTS_WITH, Op.ENDS_WITH, Op.REGEX);
}
},
CREATED("created") {
@Override
public List<Op> supportedOp() {
return List.of(Op.GREATER_THAN_OR_EQUAL_TO, Op.GREATER_THAN, Op.LESS_THAN_OR_EQUAL_TO, Op.LESS_THAN, Op.EQUALS, Op.NOT_EQUALS);
}
},
UPDATED("updated") {
@Override
public List<Op> supportedOp() {
@@ -199,18 +163,6 @@ public record QueryFilter(
return List.of(Op.EQUALS, Op.NOT_EQUALS, Op.CONTAINS, Op.STARTS_WITH, Op.ENDS_WITH, Op.IN, Op.NOT_IN);
}
},
TASK_ID("taskId") {
@Override
public List<Op> supportedOp() {
return List.of(Op.EQUALS, Op.NOT_EQUALS, Op.CONTAINS, Op.STARTS_WITH, Op.ENDS_WITH, Op.IN, Op.NOT_IN);
}
},
TASK_RUN_ID("taskRunId") {
@Override
public List<Op> supportedOp() {
return List.of(Op.EQUALS, Op.NOT_EQUALS, Op.CONTAINS, Op.STARTS_WITH, Op.ENDS_WITH, Op.IN, Op.NOT_IN);
}
},
CHILD_FILTER("childFilter") {
@Override
public List<Op> supportedOp() {
@@ -360,34 +312,6 @@ public record QueryFilter(
Field.UPDATED
);
}
},
ASSET {
@Override
public List<Field> supportedField() {
return List.of(
Field.QUERY,
Field.ID,
Field.TYPE,
Field.NAMESPACE,
Field.METADATA,
Field.UPDATED
);
}
},
ASSET_USAGE {
@Override
public List<Field> supportedField() {
return List.of(
Field.ASSET_ID,
Field.NAMESPACE,
Field.FLOW_ID,
Field.FLOW_REVISION,
Field.EXECUTION_ID,
Field.TASK_ID,
Field.TASK_RUN_ID,
Field.CREATED
);
}
};
public abstract List<Field> supportedField();

View File

@@ -1,111 +0,0 @@
package io.kestra.core.models.assets;
import com.fasterxml.jackson.annotation.JsonAnySetter;
import io.kestra.core.models.DeletedInterface;
import io.kestra.core.models.HasUID;
import io.kestra.core.models.Plugin;
import io.kestra.core.utils.IdUtils;
import io.swagger.v3.oas.annotations.Hidden;
import jakarta.annotation.Nullable;
import jakarta.validation.constraints.NotBlank;
import jakarta.validation.constraints.Pattern;
import jakarta.validation.constraints.Size;
import lombok.Getter;
import lombok.NoArgsConstructor;
import java.time.Instant;
import java.util.*;
@Getter
@NoArgsConstructor
public abstract class Asset implements HasUID, DeletedInterface, Plugin {
@Hidden
@Pattern(regexp = "^[a-z0-9][a-z0-9_-]*")
protected String tenantId;
@Pattern(regexp = "^[a-z0-9][a-z0-9._-]*")
@Size(min = 1, max = 150)
protected String namespace;
@NotBlank
@Pattern(regexp = "^[a-zA-Z0-9][a-zA-Z0-9._-]*")
@Size(min = 1, max = 150)
protected String id;
@NotBlank
protected String type;
protected String displayName;
protected String description;
protected Map<String, Object> metadata;
@Nullable
@Hidden
private Instant created;
@Nullable
@Hidden
private Instant updated;
@Hidden
private boolean deleted;
public Asset(
String tenantId,
String namespace,
String id,
String type,
String displayName,
String description,
Map<String, Object> metadata,
Instant created,
Instant updated,
boolean deleted
) {
this.tenantId = tenantId;
this.namespace = namespace;
this.id = id;
this.type = type;
this.displayName = displayName;
this.description = description;
this.metadata = Optional.ofNullable(metadata).map(HashMap::new).orElse(new HashMap<>());
Instant now = Instant.now();
this.created = Optional.ofNullable(created).orElse(now);
this.updated = Optional.ofNullable(updated).orElse(now);
this.deleted = deleted;
}
public <T extends Asset> T toUpdated() {
if (this.created == null) {
this.created = Instant.now();
}
this.updated = Instant.now();
return (T) this;
}
public Asset toDeleted() {
this.deleted = true;
return this;
}
@JsonAnySetter
public void setMetadata(String name, Object value) {
metadata.put(name, value);
}
@Override
public String uid() {
return Asset.uid(tenantId, id);
}
public static String uid(String tenantId, String id) {
return IdUtils.fromParts(tenantId, id);
}
public Asset withTenantId(String tenantId) {
this.tenantId = tenantId;
return this;
}
}

View File

@@ -1,19 +0,0 @@
package io.kestra.core.models.assets;
import io.kestra.core.utils.IdUtils;
import io.swagger.v3.oas.annotations.Hidden;
public record AssetIdentifier(@Hidden String tenantId, @Hidden String namespace, String id){
public AssetIdentifier withTenantId(String tenantId) {
return new AssetIdentifier(tenantId, this.namespace, this.id);
}
public String uid() {
return IdUtils.fromParts(tenantId, id);
}
public static AssetIdentifier of(Asset asset) {
return new AssetIdentifier(asset.getTenantId(), asset.getNamespace(), asset.getId());
}
}

View File

@@ -1,18 +0,0 @@
package io.kestra.core.models.assets;
import io.kestra.core.models.HasUID;
import io.kestra.core.models.flows.FlowId;
import io.kestra.core.utils.IdUtils;
/**
* Represents an entity that used an asset
*/
public record AssetUser(String tenantId, String namespace, String flowId, Integer flowRevision, String executionId, String taskId, String taskRunId) implements HasUID {
public String uid() {
return IdUtils.fromParts(tenantId, namespace, flowId, String.valueOf(flowRevision), executionId, taskRunId);
}
public FlowId toFlowId() {
return FlowId.of(tenantId, namespace, flowId, flowRevision);
}
}

View File

@@ -1,22 +0,0 @@
package io.kestra.core.models.assets;
import com.fasterxml.jackson.annotation.JsonCreator;
import io.micronaut.core.annotation.Introspected;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import java.util.List;
import java.util.Optional;
@Getter
public class AssetsDeclaration extends AssetsInOut {
private boolean enableAuto;
@JsonCreator
public AssetsDeclaration(Boolean enableAuto, List<AssetIdentifier> inputs, List<Asset> outputs) {
super(inputs, outputs);
this.enableAuto = Optional.ofNullable(enableAuto).orElse(false);
}
}

View File

@@ -1,21 +0,0 @@
package io.kestra.core.models.assets;
import com.fasterxml.jackson.annotation.JsonCreator;
import lombok.Getter;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
@Getter
public class AssetsInOut {
private List<AssetIdentifier> inputs;
private List<Asset> outputs;
@JsonCreator
public AssetsInOut(List<AssetIdentifier> inputs, List<Asset> outputs) {
this.inputs = Optional.ofNullable(inputs).orElse(Collections.emptyList());
this.outputs = Optional.ofNullable(outputs).orElse(Collections.emptyList());
}
}

View File

@@ -1,30 +0,0 @@
package io.kestra.core.models.assets;
import com.fasterxml.jackson.annotation.JsonCreator;
import io.kestra.core.models.annotations.Plugin;
import lombok.Builder;
import lombok.NoArgsConstructor;
import java.time.Instant;
import java.util.Map;
@NoArgsConstructor
@Plugin
public class CustomAsset extends Asset {
@Builder
@JsonCreator
public CustomAsset(
String tenantId,
String namespace,
String id,
String type,
String displayName,
String description,
Map<String, Object> metadata,
Instant created,
Instant updated,
boolean deleted
) {
super(tenantId, namespace, id, type, displayName, description, metadata, created, updated, deleted);
}
}

View File

@@ -1,73 +0,0 @@
package io.kestra.core.models.assets;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import io.kestra.core.models.annotations.Plugin;
import lombok.Builder;
import lombok.NoArgsConstructor;
import java.time.Instant;
import java.util.Map;
import java.util.Optional;
@NoArgsConstructor
@Plugin(aliases = DatasetAsset.ASSET_TYPE)
public class DatasetAsset extends Asset {
public static final String ASSET_TYPE = "DATASET";
@Builder
@JsonCreator
public DatasetAsset(
String tenantId,
String namespace,
String id,
String displayName,
String description,
String system,
String location,
String format,
Map<String, Object> metadata,
Instant created,
Instant updated,
boolean deleted
) {
super(tenantId, namespace, id, ASSET_TYPE, displayName, description, metadata, created, updated, deleted);
this.setSystem(system);
this.setLocation(location);
this.setFormat(format);
}
@JsonProperty("system")
public String getSystem() {
return Optional.ofNullable(metadata.get("system")).map(Object::toString).orElse(null);
}
@JsonProperty("location")
public String getLocation() {
return Optional.ofNullable(metadata.get("location")).map(Object::toString).orElse(null);
}
@JsonProperty("format")
public String getFormat() {
return Optional.ofNullable(metadata.get("format")).map(Object::toString).orElse(null);
}
public void setSystem(String system) {
if (system != null) {
metadata.put("system", system);
}
}
public void setLocation(String location) {
if (location != null) {
metadata.put("location", location);
}
}
public void setFormat(String format) {
if (format != null) {
metadata.put("format", format);
}
}
}

View File

@@ -1,31 +0,0 @@
package io.kestra.core.models.assets;
import com.fasterxml.jackson.annotation.JsonCreator;
import io.kestra.core.models.annotations.Plugin;
import lombok.Builder;
import lombok.NoArgsConstructor;
import java.time.Instant;
import java.util.Map;
@NoArgsConstructor
@Plugin(aliases = ExternalAsset.ASSET_TYPE)
public class ExternalAsset extends Asset {
public static final String ASSET_TYPE = "EXTERNAL";
@Builder
@JsonCreator
public ExternalAsset(
String tenantId,
String namespace,
String id,
String displayName,
String description,
Map<String, Object> metadata,
Instant created,
Instant updated,
boolean deleted
) {
super(tenantId, namespace, id, ASSET_TYPE, displayName, description, metadata, created, updated, deleted);
}
}

View File

@@ -1,60 +0,0 @@
package io.kestra.core.models.assets;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import io.kestra.core.models.annotations.Plugin;
import lombok.Builder;
import lombok.NoArgsConstructor;
import java.time.Instant;
import java.util.Map;
import java.util.Optional;
@NoArgsConstructor
@Plugin(aliases = FileAsset.ASSET_TYPE)
public class FileAsset extends Asset {
public static final String ASSET_TYPE = "FILE";
@Builder
@JsonCreator
public FileAsset(
String tenantId,
String namespace,
String id,
String displayName,
String description,
String system,
String path,
Map<String, Object> metadata,
Instant created,
Instant updated,
boolean deleted
) {
super(tenantId, namespace, id, ASSET_TYPE, displayName, description, metadata, created, updated, deleted);
this.setSystem(system);
this.setPath(path);
}
@JsonProperty("system")
public String getSystem() {
return Optional.ofNullable(metadata.get("system")).map(Object::toString).orElse(null);
}
@JsonProperty("path")
public String getPath() {
return Optional.ofNullable(metadata.get("path")).map(Object::toString).orElse(null);
}
public void setSystem(String system) {
if (system != null) {
metadata.put("system", system);
}
}
public void setPath(String path) {
if (path != null) {
metadata.put("path", path);
}
}
}

View File

@@ -1,86 +0,0 @@
package io.kestra.core.models.assets;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import io.kestra.core.models.annotations.Plugin;
import lombok.Builder;
import lombok.NoArgsConstructor;
import java.time.Instant;
import java.util.Map;
import java.util.Optional;
@NoArgsConstructor
@Plugin(aliases = TableAsset.ASSET_TYPE)
public class TableAsset extends Asset {
public static final String ASSET_TYPE = "TABLE";
@Builder
@JsonCreator
public TableAsset(
String tenantId,
String namespace,
String id,
String displayName,
String description,
String system,
String database,
String schema,
String name,
Map<String, Object> metadata,
Instant created,
Instant updated,
boolean deleted
) {
super(tenantId, namespace, id, ASSET_TYPE, displayName, description, metadata, created, updated, deleted);
this.setSystem(system);
this.setDatabase(database);
this.setSchema(schema);
this.setName(name);
}
@JsonProperty("system")
public String getSystem() {
return Optional.ofNullable(metadata.get("system")).map(Object::toString).orElse(null);
}
@JsonProperty("database")
public String getDatabase() {
return Optional.ofNullable(metadata.get("database")).map(Object::toString).orElse(null);
}
@JsonProperty("schema")
public String getSchema() {
return Optional.ofNullable(metadata.get("schema")).map(Object::toString).orElse(null);
}
@JsonProperty("name")
public String getName() {
return Optional.ofNullable(metadata.get("name")).map(Object::toString).orElse(null);
}
public void setSystem(String system) {
if (system != null) {
metadata.put("system", system);
}
}
public void setDatabase(String database) {
if (database != null) {
metadata.put("database", database);
}
}
public void setSchema(String schema) {
if (schema != null) {
metadata.put("schema", schema);
}
}
public void setName(String name) {
if (name != null) {
metadata.put("name", name);
}
}
}

View File

@@ -1,73 +0,0 @@
package io.kestra.core.models.assets;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import io.kestra.core.models.annotations.Plugin;
import lombok.Builder;
import lombok.NoArgsConstructor;
import java.time.Instant;
import java.util.Map;
import java.util.Optional;
@NoArgsConstructor
@Plugin(aliases = VmAsset.ASSET_TYPE)
public class VmAsset extends Asset {
public static final String ASSET_TYPE = "VM";
@Builder
@JsonCreator
public VmAsset(
String tenantId,
String namespace,
String id,
String displayName,
String description,
String provider,
String region,
String state,
Map<String, Object> metadata,
Instant created,
Instant updated,
boolean deleted
) {
super(tenantId, namespace, id, ASSET_TYPE, displayName, description, metadata, created, updated, deleted);
this.setProvider(provider);
this.setRegion(region);
this.setState(state);
}
@JsonProperty("provider")
public String getProvider() {
return Optional.ofNullable(metadata.get("provider")).map(Object::toString).orElse(null);
}
@JsonProperty("region")
public String getRegion() {
return Optional.ofNullable(metadata.get("region")).map(Object::toString).orElse(null);
}
@JsonProperty("state")
public String getState() {
return Optional.ofNullable(metadata.get("state")).map(Object::toString).orElse(null);
}
public void setProvider(String provider) {
if (provider != null) {
metadata.put("provider", provider);
}
}
public void setRegion(String region) {
if (region != null) {
metadata.put("region", region);
}
}
public void setState(String state) {
if (state != null) {
metadata.put("state", state);
}
}
}

View File

@@ -2,7 +2,6 @@ package io.kestra.core.models.executions;
import com.fasterxml.jackson.annotation.JsonInclude;
import io.kestra.core.models.TenantInterface;
import io.kestra.core.models.assets.AssetsInOut;
import io.kestra.core.models.flows.State;
import io.kestra.core.models.tasks.ResolvedTask;
import io.kestra.core.models.tasks.retrys.AbstractRetry;
@@ -58,10 +57,6 @@ public class TaskRun implements TenantInterface {
@Schema(implementation = Object.class)
Variables outputs;
@With
@Nullable
AssetsInOut assets;
@NotNull
State state;
@@ -92,7 +87,6 @@ public class TaskRun implements TenantInterface {
this.value,
this.attempts,
this.outputs,
this.assets,
this.state.withState(state),
this.iteration,
this.dynamic,
@@ -120,7 +114,6 @@ public class TaskRun implements TenantInterface {
this.value,
newAttempts,
this.outputs,
this.assets,
this.state.withState(state),
this.iteration,
this.dynamic,
@@ -144,7 +137,6 @@ public class TaskRun implements TenantInterface {
this.value,
newAttempts,
this.outputs,
this.assets,
this.state.withState(State.Type.FAILED),
this.iteration,
this.dynamic,
@@ -164,7 +156,6 @@ public class TaskRun implements TenantInterface {
.value(this.getValue())
.attempts(this.getAttempts())
.outputs(this.getOutputs())
.assets(this.getAssets())
.state(state == null ? this.getState() : state)
.iteration(this.getIteration())
.build();
@@ -251,7 +242,6 @@ public class TaskRun implements TenantInterface {
", parentTaskRunId=" + this.getParentTaskRunId() +
", state=" + this.getState().getCurrent().toString() +
", outputs=" + this.getOutputs() +
", assets=" + this.getAssets() +
", attempts=" + this.getAttempts() +
")";
}

View File

@@ -5,13 +5,11 @@ import com.fasterxml.jackson.annotation.JsonInclude;
import io.kestra.core.exceptions.IllegalVariableEvaluationException;
import io.kestra.core.models.annotations.Plugin;
import io.kestra.core.models.annotations.PluginProperty;
import io.kestra.core.models.assets.AssetsDeclaration;
import io.kestra.core.models.executions.TaskRun;
import io.kestra.core.models.property.Property;
import io.kestra.core.models.tasks.retrys.AbstractRetry;
import io.kestra.core.runners.RunContext;
import io.kestra.plugin.core.flow.WorkingDirectory;
import jakarta.annotation.Nullable;
import jakarta.validation.Valid;
import jakarta.validation.constraints.Size;
import lombok.Builder;
@@ -80,11 +78,6 @@ abstract public class Task implements TaskInterface {
@Valid
private Cache taskCache;
@PluginProperty(hidden = true, group = PluginProperty.CORE_GROUP)
@Valid
@Nullable
private Property<AssetsDeclaration> assets;
public Optional<Task> findById(String id) {
if (this.getId().equals(id)) {
return Optional.of(this);

View File

@@ -1,13 +1,10 @@
package io.kestra.core.models.tasks.runners;
import com.fasterxml.jackson.databind.ObjectMapper;
import io.kestra.core.exceptions.IllegalVariableEvaluationException;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.executions.AbstractMetricEntry;
import io.kestra.core.queues.QueueException;
import io.kestra.core.runners.AssetEmitter;
import io.kestra.core.runners.RunContext;
import io.kestra.core.serializers.JacksonMapper;
import jakarta.inject.Singleton;
import org.slf4j.Logger;
import org.slf4j.event.Level;
import org.slf4j.spi.LoggingEventBuilder;
@@ -21,7 +18,6 @@ import java.util.regex.Matcher;
import java.util.regex.Pattern;
import static io.kestra.core.runners.RunContextLogger.ORIGINAL_TIMESTAMP_KEY;
import static io.kestra.core.utils.Rethrow.throwConsumer;
/**
* Service for matching and capturing structured data from task execution logs.
@@ -80,18 +76,6 @@ public class TaskLogLineMatcher {
}
});
}
if (match.assets() != null) {
try {
AssetEmitter assetEmitter = runContext.assets();
match.assets().forEach(throwConsumer(assetEmitter::upsert));
} catch (IllegalVariableEvaluationException e) {
logger.warn("Unable to get asset emitter for log '{}'", data, e);
} catch (QueueException e) {
logger.warn("Unable to emit asset for log '{}'", data, e);
}
}
return match;
}
@@ -110,9 +94,8 @@ public class TaskLogLineMatcher {
public record TaskLogMatch(
Map<String, Object> outputs,
List<AbstractMetricEntry<?>> metrics,
List<LogLine> logs,
List<Asset> assets
) {
List<LogLine> logs
) {
@Override
public Map<String, Object> outputs() {
return Optional.ofNullable(outputs).orElse(Map.of());

View File

@@ -6,10 +6,8 @@ import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import io.kestra.core.models.Label;
import io.kestra.core.models.annotations.Plugin;
import io.kestra.core.models.annotations.PluginProperty;
import io.kestra.core.models.assets.AssetsDeclaration;
import io.kestra.core.models.conditions.Condition;
import io.kestra.core.models.flows.State;
import io.kestra.core.models.property.Property;
import io.kestra.core.models.tasks.WorkerGroup;
import io.kestra.core.serializers.ListOrMapOfLabelDeserializer;
import io.kestra.core.serializers.ListOrMapOfLabelSerializer;
@@ -90,9 +88,6 @@ abstract public class AbstractTrigger implements TriggerInterface {
)
private boolean allowConcurrent = false;
@PluginProperty(hidden = true, group = PluginProperty.CORE_GROUP)
private Property<AssetsDeclaration> assets;
/**
* For backward compatibility: we rename minLogLevel to logLevel.
* @deprecated use {@link #logLevel} instead

View File

@@ -1,7 +1,6 @@
package io.kestra.core.plugins;
import io.kestra.core.models.Plugin;
import io.kestra.core.models.assets.Asset;
import jakarta.annotation.Nullable;
import jakarta.validation.constraints.NotNull;
import org.slf4j.Logger;

View File

@@ -2,7 +2,6 @@ package io.kestra.core.plugins;
import com.fasterxml.jackson.databind.module.SimpleModule;
import io.kestra.core.app.AppPluginInterface;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.conditions.Condition;
import io.kestra.core.models.dashboards.DataFilter;
import io.kestra.core.models.dashboards.DataFilterKPI;
@@ -12,7 +11,6 @@ import io.kestra.core.models.tasks.Task;
import io.kestra.core.models.tasks.logs.LogExporter;
import io.kestra.core.models.tasks.runners.TaskRunner;
import io.kestra.core.models.triggers.AbstractTrigger;
import io.kestra.core.plugins.serdes.AssetDeserializer;
import io.kestra.core.plugins.serdes.PluginDeserializer;
import io.kestra.core.secret.SecretPluginInterface;
import io.kestra.core.storages.StorageInterface;
@@ -47,6 +45,5 @@ public class PluginModule extends SimpleModule {
addDeserializer(SecretPluginInterface.class, new PluginDeserializer<>());
addDeserializer(AppPluginInterface.class, new PluginDeserializer<>());
addDeserializer(LogExporter.class, new PluginDeserializer<>());
addDeserializer(Asset.class, new AssetDeserializer());
}
}

View File

@@ -3,7 +3,6 @@ package io.kestra.core.plugins;
import io.kestra.core.app.AppBlockInterface;
import io.kestra.core.app.AppPluginInterface;
import io.kestra.core.models.Plugin;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.conditions.Condition;
import io.kestra.core.models.dashboards.DataFilter;
import io.kestra.core.models.dashboards.DataFilterKPI;
@@ -109,7 +108,6 @@ public class PluginScanner {
List<Class<? extends StorageInterface>> storages = new ArrayList<>();
List<Class<? extends SecretPluginInterface>> secrets = new ArrayList<>();
List<Class<? extends TaskRunner<?>>> taskRunners = new ArrayList<>();
List<Class<? extends Asset>> assets = new ArrayList<>();
List<Class<? extends AppPluginInterface>> apps = new ArrayList<>();
List<Class<? extends AppBlockInterface>> appBlocks = new ArrayList<>();
List<Class<? extends Chart<?>>> charts = new ArrayList<>();
@@ -157,10 +155,6 @@ public class PluginScanner {
//noinspection unchecked
taskRunners.add((Class<? extends TaskRunner<?>>) runner.getClass());
}
case Asset asset -> {
log.debug("Loading Asset plugin: '{}'", plugin.getClass());
assets.add(asset.getClass());
}
case AppPluginInterface app -> {
log.debug("Loading App plugin: '{}'", plugin.getClass());
apps.add(app.getClass());
@@ -229,7 +223,6 @@ public class PluginScanner {
.conditions(conditions)
.storages(storages)
.secrets(secrets)
.assets(assets)
.apps(apps)
.appBlocks(appBlocks)
.taskRunners(taskRunners)

View File

@@ -3,7 +3,6 @@ package io.kestra.core.plugins;
import io.kestra.core.app.AppBlockInterface;
import io.kestra.core.app.AppPluginInterface;
import io.kestra.core.models.annotations.PluginSubGroup;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.conditions.Condition;
import io.kestra.core.models.dashboards.DataFilter;
import io.kestra.core.models.dashboards.DataFilterKPI;
@@ -40,7 +39,6 @@ public class RegisteredPlugin {
public static final String STORAGES_GROUP_NAME = "storages";
public static final String SECRETS_GROUP_NAME = "secrets";
public static final String TASK_RUNNERS_GROUP_NAME = "task-runners";
public static final String ASSETS_GROUP_NAME = "assets";
public static final String APPS_GROUP_NAME = "apps";
public static final String APP_BLOCKS_GROUP_NAME = "app-blocks";
public static final String CHARTS_GROUP_NAME = "charts";
@@ -58,7 +56,6 @@ public class RegisteredPlugin {
private final List<Class<? extends StorageInterface>> storages;
private final List<Class<? extends SecretPluginInterface>> secrets;
private final List<Class<? extends TaskRunner<?>>> taskRunners;
private final List<Class<? extends Asset>> assets;
private final List<Class<? extends AppPluginInterface>> apps;
private final List<Class<? extends AppBlockInterface>> appBlocks;
private final List<Class<? extends Chart<?>>> charts;
@@ -77,7 +74,6 @@ public class RegisteredPlugin {
!storages.isEmpty() ||
!secrets.isEmpty() ||
!taskRunners.isEmpty() ||
!assets.isEmpty() ||
!apps.isEmpty() ||
!appBlocks.isEmpty() ||
!charts.isEmpty() ||
@@ -149,10 +145,6 @@ public class RegisteredPlugin {
return AppPluginInterface.class;
}
if (this.getAssets().stream().anyMatch(r -> r.getName().equals(cls))) {
return Asset.class;
}
if (this.getLogExporters().stream().anyMatch(r -> r.getName().equals(cls))) {
return LogExporter.class;
}
@@ -188,7 +180,6 @@ public class RegisteredPlugin {
result.put(STORAGES_GROUP_NAME, Arrays.asList(this.getStorages().toArray(Class[]::new)));
result.put(SECRETS_GROUP_NAME, Arrays.asList(this.getSecrets().toArray(Class[]::new)));
result.put(TASK_RUNNERS_GROUP_NAME, Arrays.asList(this.getTaskRunners().toArray(Class[]::new)));
result.put(ASSETS_GROUP_NAME, Arrays.asList(this.getAssets().toArray(Class[]::new)));
result.put(APPS_GROUP_NAME, Arrays.asList(this.getApps().toArray(Class[]::new)));
result.put(APP_BLOCKS_GROUP_NAME, Arrays.asList(this.getAppBlocks().toArray(Class[]::new)));
result.put(CHARTS_GROUP_NAME, Arrays.asList(this.getCharts().toArray(Class[]::new)));
@@ -368,12 +359,6 @@ public class RegisteredPlugin {
b.append("] ");
}
if (!this.getAssets().isEmpty()) {
b.append("[Assets: ");
b.append(this.getAssets().stream().map(Class::getName).collect(Collectors.joining(", ")));
b.append("] ");
}
if (!this.getApps().isEmpty()) {
b.append("[Apps: ");
b.append(this.getApps().stream().map(Class::getName).collect(Collectors.joining(", ")));

View File

@@ -1,16 +0,0 @@
package io.kestra.core.plugins.serdes;
import com.fasterxml.jackson.databind.JsonDeserializer;
import io.kestra.core.models.Plugin;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.assets.CustomAsset;
/**
* Specific {@link JsonDeserializer} for deserializing {@link Asset}.
*/
public final class AssetDeserializer extends PluginDeserializer<Asset> {
@Override
protected Class<? extends Plugin> fallbackClass() {
return CustomAsset.class;
}
}

View File

@@ -29,7 +29,7 @@ import java.util.Optional;
* The {@link PluginDeserializer} uses the {@link PluginRegistry} to found the plugin class corresponding to
* a plugin type.
*/
public class PluginDeserializer<T extends Plugin> extends JsonDeserializer<T> {
public final class PluginDeserializer<T extends Plugin> extends JsonDeserializer<T> {
private static final Logger log = LoggerFactory.getLogger(PluginDeserializer.class);
@@ -93,10 +93,6 @@ public class PluginDeserializer<T extends Plugin> extends JsonDeserializer<T> {
identifier
);
pluginType = pluginRegistry.findClassByIdentifier(identifier);
if (pluginType == null) {
pluginType = fallbackClass();
}
}
if (pluginType == null) {
@@ -157,8 +153,4 @@ public class PluginDeserializer<T extends Plugin> extends JsonDeserializer<T> {
return isVersioningSupported && version != null && !version.isEmpty() ? type + ":" + version : type;
}
protected Class<? extends Plugin> fallbackClass() {
return null;
}
}

View File

@@ -1,12 +0,0 @@
package io.kestra.core.runners;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.queues.QueueException;
import java.util.List;
public interface AssetEmitter {
void upsert(Asset asset) throws QueueException;
List<Asset> outputs();
}

View File

@@ -6,13 +6,11 @@ import com.google.common.base.CaseFormat;
import com.google.common.collect.ImmutableMap;
import io.kestra.core.exceptions.IllegalVariableEvaluationException;
import io.kestra.core.metrics.MetricRegistry;
import io.kestra.core.models.assets.AssetsDeclaration;
import io.kestra.core.models.Plugin;
import io.kestra.core.models.executions.AbstractMetricEntry;
import io.kestra.core.models.property.Property;
import io.kestra.core.models.tasks.Task;
import io.kestra.core.models.triggers.AbstractTrigger;
import io.kestra.core.services.AssetManagerFactory;
import io.kestra.core.plugins.PluginConfigurations;
import io.kestra.core.services.KVStoreService;
import io.kestra.core.storages.Storage;
@@ -56,7 +54,6 @@ public class DefaultRunContext extends RunContext {
private MetricRegistry meterRegistry;
private VersionProvider version;
private KVStoreService kvStoreService;
private AssetManagerFactory assetManagerFactory;
private Optional<String> secretKey;
private WorkingDir workingDir;
private Validator validator;
@@ -76,8 +73,6 @@ public class DefaultRunContext extends RunContext {
private Task task;
private AbstractTrigger trigger;
private volatile AssetEmitter assetEmitter;
private final AtomicBoolean isInitialized = new AtomicBoolean(false);
@@ -166,7 +161,6 @@ public class DefaultRunContext extends RunContext {
this.secretKey = applicationContext.getProperty("kestra.encryption.secret-key", String.class);
this.validator = applicationContext.getBean(Validator.class);
this.localPath = applicationContext.getBean(LocalPathFactory.class).createLocalPath(this);
this.assetManagerFactory = applicationContext.getBean(AssetManagerFactory.class);
}
}
@@ -543,23 +537,6 @@ public class DefaultRunContext extends RunContext {
return flow != null ? flow.get("tenantId") : null;
}
/**
* {@inheritDoc}
*/
@Override
public TaskRunInfo taskRunInfo() {
Optional<Map<String, Object>> maybeTaskRunMap = Optional.ofNullable(this.getVariables().get("taskrun"))
.map(Map.class::cast);
return new TaskRunInfo(
(String) this.getVariables().get("executionId"),
(String) this.getVariables().get("taskId"),
maybeTaskRunMap.map(m -> (String) m.get("id"))
.orElse(null),
maybeTaskRunMap.map(m -> (String) m.get("value"))
.orElse(null)
);
}
/**
* {@inheritDoc}
*/
@@ -568,7 +545,12 @@ public class DefaultRunContext extends RunContext {
public FlowInfo flowInfo() {
Map<String, Object> flow = (Map<String, Object>) this.getVariables().get("flow");
// normally only tests should not have the flow variable
return flow == null ? new FlowInfo(null, null, null, null) : FlowInfo.from(flow);
return flow == null ? new FlowInfo(null, null, null, null) : new FlowInfo(
(String) flow.get("tenantId"),
(String) flow.get("namespace"),
(String) flow.get("id"),
(Integer) flow.get("revision")
);
}
/**
@@ -612,25 +594,6 @@ public class DefaultRunContext extends RunContext {
return new AclCheckerImpl(this.applicationContext, flowInfo());
}
@Override
public AssetEmitter assets() throws IllegalVariableEvaluationException {
if (this.assetEmitter == null) {
synchronized (this) {
if (this.assetEmitter == null) {
this.assetEmitter = assetManagerFactory.of(
Optional.ofNullable(task).map(Task::getAssets)
.or(() -> Optional.ofNullable(trigger).map(AbstractTrigger::getAssets))
.flatMap(throwFunction(asset -> this.render(asset).as(AssetsDeclaration.class)))
.map(AssetsDeclaration::isEnableAuto)
.orElse(false)
);
}
}
}
return this.assetEmitter;
}
@Override
public LocalPath localPath() {
return localPath;

View File

@@ -143,8 +143,6 @@ public abstract class RunContext implements PropertyContext {
@Deprecated(forRemoval = true)
public abstract String tenantId();
public abstract TaskRunInfo taskRunInfo();
public abstract FlowInfo flowInfo();
/**
@@ -192,19 +190,7 @@ public abstract class RunContext implements PropertyContext {
*/
public abstract LocalPath localPath();
public record TaskRunInfo(String executionId, String taskId, String taskRunId, Object value) {
}
public record FlowInfo(String tenantId, String namespace, String id, Integer revision) {
public static FlowInfo from(Map<String, Object> flowInfoMap) {
return new FlowInfo(
(String) flowInfoMap.get("tenantId"),
(String) flowInfoMap.get("namespace"),
(String) flowInfoMap.get("id"),
(Integer) flowInfoMap.get("revision")
);
}
}
/**
@@ -220,11 +206,6 @@ public abstract class RunContext implements PropertyContext {
*/
public abstract AclChecker acl();
/**
* Get access to the Assets handler.
*/
public abstract AssetEmitter assets() throws IllegalVariableEvaluationException;
/**
* Clone this run context for a specific plugin.
* @return a new run context with the plugin configuration of the given plugin.

View File

@@ -1,7 +1,6 @@
package io.kestra.core.runners;
import io.kestra.core.models.HasUID;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.executions.TaskRun;
import lombok.AllArgsConstructor;
import lombok.Builder;
@@ -11,7 +10,6 @@ import java.util.ArrayList;
import java.util.List;
import jakarta.validation.constraints.NotNull;
import lombok.With;
@Value
@AllArgsConstructor
@@ -23,7 +21,8 @@ public class WorkerTaskResult implements HasUID {
List<TaskRun> dynamicTaskRuns;
public WorkerTaskResult(TaskRun taskRun) {
this(taskRun, new ArrayList<>());
this.taskRun = taskRun;
this.dynamicTaskRuns = new ArrayList<>();
}
/**

View File

@@ -1,25 +0,0 @@
package io.kestra.core.services;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.runners.AssetEmitter;
import jakarta.inject.Singleton;
import java.util.ArrayList;
import java.util.List;
@Singleton
public class AssetManagerFactory {
public AssetEmitter of(boolean enabled) {
return new AssetEmitter() {
@Override
public void upsert(Asset asset) {
throw new UnsupportedOperationException();
}
@Override
public List<Asset> outputs() {
return new ArrayList<>();
}
};
}
}

View File

@@ -1,32 +0,0 @@
package io.kestra.core.services;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.assets.AssetIdentifier;
import io.kestra.core.models.assets.AssetUser;
import io.kestra.core.queues.QueueException;
import jakarta.annotation.PostConstruct;
import jakarta.inject.Named;
import jakarta.inject.Singleton;
import java.util.Collections;
import java.util.List;
@Singleton
public class AssetService implements Runnable {
@PostConstruct
public void start() {
this.run();
}
public void asyncUpsert(AssetUser assetUser, Asset asset) throws QueueException {
// No-op
}
public void assetLineage(AssetUser assetUser, List<AssetIdentifier> inputs, List<AssetIdentifier> outputs) throws QueueException {
// No-op
}
public void run() {
// No-op
}
}

View File

@@ -1,6 +1,5 @@
package io.kestra.core.test.flow;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.flows.State;
import io.kestra.core.models.property.Property;
import jakarta.validation.constraints.NotNull;
@@ -9,7 +8,6 @@ import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;
import java.util.List;
import java.util.Map;
@Getter
@@ -27,7 +25,5 @@ public class TaskFixture {
private Map<String, Object> outputs;
private List<Asset> assets;
private Property<String> description;
}

View File

@@ -19,7 +19,15 @@ import jakarta.inject.Inject;
import jakarta.inject.Singleton;
import java.lang.reflect.Field;
import java.util.*;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
@@ -147,8 +155,6 @@ public class FlowValidator implements ConstraintValidator<FlowValidation, Flow>
.map(task -> task.getId())
.collect(Collectors.toList());
violations.addAll(assetsViolations(allTasks));
if (!invalidTasks.isEmpty()) {
violations.add("Invalid output reference: use outputs[key-name] instead of outputs.key-name — keys with dashes require bracket notation, offending tasks:" +
" [" + String.join(", ", invalidTasks) + "]");
@@ -175,12 +181,6 @@ public class FlowValidator implements ConstraintValidator<FlowValidation, Flow>
}
}
protected List<String> assetsViolations(List<Task> allTasks) {
return allTasks.stream().filter(task -> task.getAssets() != null)
.map(taskWithAssets -> "Task '" + taskWithAssets.getId() + "' can't have any `assets` because assets are only available in Enterprise Edition.")
.toList();
}
private static boolean checkObjectFieldsWithPatterns(Object object, List<Pattern> patterns) {
if (object == null) {
return true;

View File

@@ -1,268 +0,0 @@
package io.kestra.assets.assets;
import com.fasterxml.jackson.core.JsonProcessingException;
import io.kestra.core.junit.annotations.KestraTest;
import io.kestra.core.models.assets.*;
import io.kestra.core.serializers.JacksonMapper;
import io.kestra.core.utils.TestsUtils;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
@MicronautTest
public class AssetTest {
@Test
void custom() throws JsonProcessingException {
String namespace = TestsUtils.randomNamespace();
String id = TestsUtils.randomString();
String type = "MY_OWN_ASSET_TYPE";
String displayName = "My own asset";
String description = "This is my asset";
String metadataKey = "owner";
String metadataValue = "data-team";
Asset asset = JacksonMapper.ofYaml().readValue("""
namespace: %s
id: %s
type: %s
displayName: %s
description: %s
metadata:
%s: %s""".formatted(
namespace,
id,
type,
displayName,
description,
metadataKey,
metadataValue
), Asset.class);
assertThat(asset).isInstanceOf(CustomAsset.class);
assertThat(asset.getNamespace()).isEqualTo(namespace);
assertThat(asset.getId()).isEqualTo(id);
assertThat(asset.getType()).isEqualTo(type);
assertThat(asset.getDisplayName()).isEqualTo(displayName);
assertThat(asset.getDescription()).isEqualTo(description);
assertThat(asset.getMetadata().get(metadataKey)).isEqualTo(metadataValue);
}
@Test
void external() throws JsonProcessingException {
String namespace = TestsUtils.randomNamespace();
String id = TestsUtils.randomString();
String type = "EXTERNAL";
String displayName = "External asset";
String description = "This is an external asset";
String metadataKey = "owner";
String metadataValue = "external-team";
Asset asset = JacksonMapper.ofYaml().readValue("""
namespace: %s
id: %s
type: %s
displayName: %s
description: %s
metadata:
%s: %s""".formatted(
namespace,
id,
type,
displayName,
description,
metadataKey,
metadataValue
), Asset.class);
assertThat(asset).isInstanceOf(ExternalAsset.class);
assertThat(asset.getNamespace()).isEqualTo(namespace);
assertThat(asset.getId()).isEqualTo(id);
assertThat(asset.getType()).isEqualTo(type);
assertThat(asset.getDisplayName()).isEqualTo(displayName);
assertThat(asset.getDescription()).isEqualTo(description);
assertThat(asset.getMetadata().get(metadataKey)).isEqualTo(metadataValue);
}
@Test
void dataset() throws JsonProcessingException {
String namespace = TestsUtils.randomNamespace();
String id = TestsUtils.randomString();
String displayName = "My Dataset";
String description = "This is my dataset";
String system = "S3";
String location = "s3://my-bucket/my-dataset";
String format = "parquet";
String metadataKey = "owner";
String metadataValue = "data-team";
Asset asset = JacksonMapper.ofYaml().readValue("""
namespace: %s
id: %s
type: %s
displayName: %s
description: %s
system: %s
location: %s
format: %s
metadata:
%s: %s""".formatted(
namespace,
id,
DatasetAsset.ASSET_TYPE,
displayName,
description,
system,
location,
format,
metadataKey,
metadataValue
), Asset.class);
assertThat(asset).isInstanceOf(DatasetAsset.class);
DatasetAsset datasetAsset = (DatasetAsset) asset;
assertThat(datasetAsset.getNamespace()).isEqualTo(namespace);
assertThat(datasetAsset.getId()).isEqualTo(id);
assertThat(datasetAsset.getDisplayName()).isEqualTo(displayName);
assertThat(datasetAsset.getDescription()).isEqualTo(description);
assertThat(datasetAsset.getSystem()).isEqualTo(system);
assertThat(datasetAsset.getLocation()).isEqualTo(location);
assertThat(datasetAsset.getFormat()).isEqualTo(format);
assertThat(datasetAsset.getMetadata().get(metadataKey)).isEqualTo(metadataValue);
}
@Test
void file() throws JsonProcessingException {
String namespace = TestsUtils.randomNamespace();
String id = TestsUtils.randomString();
String displayName = "My File";
String description = "This is my file";
String system = "local";
String path = "/data/my-file.txt";
String metadataKey = "owner";
String metadataValue = "file-team";
Asset asset = JacksonMapper.ofYaml().readValue("""
namespace: %s
id: %s
type: %s
displayName: %s
description: %s
system: %s
path: %s
metadata:
%s: %s""".formatted(
namespace,
id,
FileAsset.ASSET_TYPE,
displayName,
description,
system,
path,
metadataKey,
metadataValue
), Asset.class);
assertThat(asset).isInstanceOf(FileAsset.class);
FileAsset fileAsset = (FileAsset) asset;
assertThat(fileAsset.getNamespace()).isEqualTo(namespace);
assertThat(fileAsset.getId()).isEqualTo(id);
assertThat(fileAsset.getDisplayName()).isEqualTo(displayName);
assertThat(fileAsset.getDescription()).isEqualTo(description);
assertThat(fileAsset.getSystem()).isEqualTo(system);
assertThat(fileAsset.getPath()).isEqualTo(path);
assertThat(fileAsset.getMetadata().get(metadataKey)).isEqualTo(metadataValue);
}
@Test
void table() throws JsonProcessingException {
String namespace = TestsUtils.randomNamespace();
String id = TestsUtils.randomString();
String displayName = "My Table";
String description = "This is my table";
String system = "postgres";
String database = "mydb";
String schema = "my_schema";
String name = "mytable";
String metadataKey = "owner";
String metadataValue = "table-team";
Asset asset = JacksonMapper.ofYaml().readValue("""
namespace: %s
id: %s
type: %s
displayName: %s
description: %s
system: %s
database: %s
schema: %s
name: %s
metadata:
%s: %s""".formatted(
namespace,
id,
TableAsset.ASSET_TYPE,
displayName,
description,
system,
database,
schema,
name,
metadataKey,
metadataValue
), Asset.class);
assertThat(asset).isInstanceOf(TableAsset.class);
TableAsset tableAsset = (TableAsset) asset;
assertThat(tableAsset.getNamespace()).isEqualTo(namespace);
assertThat(tableAsset.getId()).isEqualTo(id);
assertThat(tableAsset.getDisplayName()).isEqualTo(displayName);
assertThat(tableAsset.getDescription()).isEqualTo(description);
assertThat(tableAsset.getSystem()).isEqualTo(system);
assertThat(tableAsset.getDatabase()).isEqualTo(database);
assertThat(tableAsset.getSchema()).isEqualTo(schema);
assertThat(tableAsset.getName()).isEqualTo(name);
assertThat(tableAsset.getMetadata().get(metadataKey)).isEqualTo(metadataValue);
}
@Test
void vm() throws JsonProcessingException {
String namespace = TestsUtils.randomNamespace();
String id = TestsUtils.randomString();
String displayName = "My VM";
String description = "This is my vm";
String provider = "aws";
String region = "us-east-1";
String state = "running";
String metadataKey = "owner";
String metadataValue = "vm-team";
Asset asset = JacksonMapper.ofYaml().readValue("""
namespace: %s
id: %s
type: %s
displayName: %s
description: %s
provider: %s
region: %s
state: %s
metadata:
%s: %s""".formatted(
namespace,
id,
VmAsset.ASSET_TYPE,
displayName,
description,
provider,
region,
state,
metadataKey,
metadataValue
), Asset.class);
assertThat(asset).isInstanceOf(VmAsset.class);
VmAsset vmAsset = (VmAsset) asset;
assertThat(vmAsset.getNamespace()).isEqualTo(namespace);
assertThat(vmAsset.getId()).isEqualTo(id);
assertThat(vmAsset.getDisplayName()).isEqualTo(displayName);
assertThat(vmAsset.getDescription()).isEqualTo(description);
assertThat(vmAsset.getProvider()).isEqualTo(provider);
assertThat(vmAsset.getRegion()).isEqualTo(region);
assertThat(vmAsset.getState()).isEqualTo(state);
assertThat(vmAsset.getMetadata().get(metadataKey)).isEqualTo(metadataValue);
}
}

View File

@@ -95,4 +95,4 @@ class PluginDeserializerTest {
public record TestPlugin(String type) implements Plugin {
}
}
}

View File

@@ -81,9 +81,6 @@ public abstract class AbstractRunnerTest {
@Inject
private AfterExecutionTestCase afterExecutionTestCase;
@Inject
private AssetTestCase assetTestCase;
@Test
@ExecuteFlow("flows/valids/full.yaml")
void full(Execution execution) {
@@ -561,10 +558,4 @@ public abstract class AbstractRunnerTest {
public void shouldCallTasksAfterListener(Execution execution) {
afterExecutionTestCase.shouldCallTasksAfterListener(execution);
}
@Test
@LoadFlows(value = "flows/valids/assets.yaml", tenantId = "abstract-runner-test-assets")
public void assets() throws QueueException, TimeoutException {
assetTestCase.staticAndDynamicAssets("abstract-runner-test-assets");
}
}
}

View File

@@ -1,220 +0,0 @@
package io.kestra.core.runners;
import io.kestra.core.models.assets.*;
import io.kestra.core.models.executions.Execution;
import io.kestra.core.models.executions.TaskRun;
import io.kestra.core.queues.QueueException;
import io.kestra.core.services.AssetManagerFactory;
import io.kestra.core.services.AssetService;
import io.micronaut.context.annotation.Factory;
import io.micronaut.context.annotation.Replaces;
import jakarta.inject.Inject;
import jakarta.inject.Singleton;
import org.apache.commons.lang3.tuple.Pair;
import org.mockito.Mockito;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.TimeoutException;
import static org.assertj.core.api.Assertions.assertThat;
@Singleton
public class AssetTestCase {
@Inject
private AssetService mockedAssetService;
@Inject
private TestRunnerUtils testRunnerUtils;
private static final List<Asset> capturedAsyncCreate = new CopyOnWriteArrayList<>();
private static final List<Pair<AssetUser, Pair<List<AssetIdentifier>, List<AssetIdentifier>>>> capturedAssetLineage = new CopyOnWriteArrayList<>();
private static final List<Asset> capturedEnabledDynamicAssets = new CopyOnWriteArrayList<>();
private static final List<Asset> capturedDisabledDynamicAssets = new CopyOnWriteArrayList<>();
public void staticAndDynamicAssets(String tenantId) throws QueueException, TimeoutException {
Execution execution = testRunnerUtils.runOne(tenantId, "io.kestra.tests", "assets");
Mockito.verify(mockedAssetService, Mockito.times(1)).run();
// region assets-in-taskruns
List<TaskRun> taskRuns = execution.getTaskRunList().stream().toList();
assertThat(taskRuns).map(TaskRun::getAssets).map(AssetsInOut::getInputs).satisfiesExactlyInAnyOrder(
assets -> assertThat(assets).isEmpty(),
assets -> assertThat(assets).isEmpty(),
assets -> assertThat(assets).satisfiesExactlyInAnyOrder(
assetId -> assertThat(assetId.id()).isEqualTo("assets-flow-static-asset-non-existing-input-asset-uid")
),
assets -> assertThat(assets).satisfiesExactlyInAnyOrder(
assetId -> assertThat(assetId.id()).isEqualTo("assets-flow-static-asset-existing-input-uid")
)
);
assertThat(taskRuns).map(TaskRun::getAssets).map(AssetsInOut::getOutputs).satisfiesExactlyInAnyOrder(
assets -> assertThat(assets).satisfiesExactlyInAnyOrder(
AssetTestCase::assertEnabledDynamicAsset,
asset -> assertThat(asset.getId()).isEqualTo("assets-flow-static-emit-asset")
),
assets -> assertThat(assets).isEmpty(),
assets -> assertThat(assets).satisfiesExactlyInAnyOrder(
asset -> assertThat(asset.getId()).isEqualTo("assets-flow-static-asset-non-existing-output-uid")
),
assets -> assertThat(assets).satisfiesExactlyInAnyOrder(
asset -> assertThat(asset.getId()).isEqualTo("assets-flow-static-asset-existing-output-uid")
)
);
// endregion
// region dynamic-assets
assertThat(capturedEnabledDynamicAssets).anySatisfy(AssetTestCase::assertEnabledDynamicAsset);
assertThat(capturedEnabledDynamicAssets).noneMatch(asset -> asset.getId().equals("assets-flow-emit-asset-auto-false-uid"));
assertThat(capturedDisabledDynamicAssets).anySatisfy(AssetTestCase::assertDisabledDynamicAsset);
assertThat(capturedDisabledDynamicAssets).noneMatch(asset -> asset.getId().equals("assets-flow-emit-asset-uid"));
// endregion
// region asset-creation
assertThat(capturedAsyncCreate).satisfiesExactlyInAnyOrder(
AssetTestCase::assertEnabledDynamicAsset,
asset -> assertThat(asset.getId()).isEqualTo("assets-flow-static-emit-asset"),
asset -> assertThat(asset.getId()).isEqualTo("assets-flow-static-asset-non-existing-output-uid"),
asset -> assertThat(asset.getId()).isEqualTo("assets-flow-static-asset-existing-output-uid")
);
assertThat(capturedAsyncCreate).noneMatch(asset -> asset.getId().equals("assets-flow-emit-asset-auto-false-uid"));
// endregion
// region asset-lineage
assertThat(capturedAssetLineage).satisfiesExactlyInAnyOrder(
assetLineage -> {
AssetUser assetUser = assetLineage.getLeft();
assertAssetExecution(tenantId, assetUser, execution);
assertThat(assetUser.taskRunId()).isEqualTo(execution.getTaskRunList().stream().filter(taskRun -> taskRun.getTaskId().equals("emit-asset"))
.findFirst().map(TaskRun::getId).orElseThrow());
// No input assets
assertThat(assetLineage.getRight().getLeft()).isEmpty();
assertThat(assetLineage.getRight().getRight()).satisfiesExactlyInAnyOrder(
assetId -> assertThat(assetId.id()).isEqualTo("assets-flow-emit-asset-uid"),
assetId -> assertThat(assetId.id()).isEqualTo("assets-flow-static-emit-asset")
);
},
// No lineage for the second taskrun due to `enableAuto: false`, below is for the third one
assetLineage -> {
AssetUser assetUser = assetLineage.getLeft();
assertAssetExecution(tenantId, assetUser, execution);
assertThat(assetUser.taskRunId()).isEqualTo(execution.getTaskRunList().stream().filter(taskRun -> taskRun.getTaskId().equals("static-asset-non-existing-input"))
.findFirst().map(TaskRun::getId).orElseThrow());
assertThat(assetLineage.getRight().getLeft()).satisfiesExactlyInAnyOrder(
assetId -> assertThat(assetId.id()).isEqualTo("assets-flow-static-asset-non-existing-input-asset-uid")
);
assertThat(assetLineage.getRight().getRight()).satisfiesExactlyInAnyOrder(
assetId -> assertThat(assetId.id()).isEqualTo("assets-flow-static-asset-non-existing-output-uid")
);
},
assetLineage -> {
AssetUser assetUser = assetLineage.getLeft();
assertAssetExecution(tenantId, assetUser, execution);
assertThat(assetUser.taskRunId()).isEqualTo(execution.getTaskRunList().stream().filter(taskRun -> taskRun.getTaskId().equals("static-asset-existing-input"))
.findFirst().map(TaskRun::getId).orElseThrow());
assertThat(assetLineage.getRight().getLeft()).satisfiesExactlyInAnyOrder(
assetId -> assertThat(assetId.id()).isEqualTo("assets-flow-static-asset-existing-input-uid")
);
assertThat(assetLineage.getRight().getRight()).satisfiesExactlyInAnyOrder(
assetId -> assertThat(assetId.id()).isEqualTo("assets-flow-static-asset-existing-output-uid")
);
}
);
// endregion
}
private static void assertAssetExecution(String tenantId, AssetUser assetUser, Execution execution) {
assertThat(assetUser.tenantId()).isEqualTo(tenantId);
assertThat(assetUser.namespace()).isEqualTo("io.kestra.tests");
assertThat(assetUser.flowId()).isEqualTo("assets");
assertThat(assetUser.flowRevision()).isEqualTo(execution.getFlowRevision());
assertThat(assetUser.executionId()).isEqualTo(execution.getId());
}
private static void assertEnabledDynamicAsset(Asset asset) {
assertThat(asset).isInstanceOf(TableAsset.class);
TableAsset tableAsset = (TableAsset) asset;
assertThat(tableAsset.getId()).isEqualTo("assets-flow-emit-asset-uid");
assertThat(tableAsset.getType()).isEqualTo(TableAsset.ASSET_TYPE);
assertThat(tableAsset.getDisplayName()).isEqualTo("My Table Asset");
assertThat(tableAsset.getDescription()).isEqualTo("This is my table asset");
assertThat(tableAsset.getSystem()).isEqualTo("MY_DB_SYSTEM");
assertThat(tableAsset.getDatabase()).isEqualTo("my_database");
assertThat(tableAsset.getSchema()).isEqualTo("my_schema");
assertThat(tableAsset.getName()).isEqualTo("my_table");
assertThat(tableAsset.getMetadata().get("owner")).isEqualTo("data-team");
}
private static void assertDisabledDynamicAsset(Asset asset) {
assertThat(asset).isInstanceOf(TableAsset.class);
TableAsset tableAsset = (TableAsset) asset;
assertThat(tableAsset.getId()).isEqualTo("assets-flow-emit-asset-auto-false-uid");
assertThat(tableAsset.getType()).isEqualTo(TableAsset.ASSET_TYPE);
assertThat(tableAsset.getDisplayName()).isEqualTo("My Table Asset");
assertThat(tableAsset.getDescription()).isEqualTo("This is my table asset");
assertThat(tableAsset.getSystem()).isEqualTo("MY_DB_SYSTEM");
assertThat(tableAsset.getDatabase()).isEqualTo("my_database");
assertThat(tableAsset.getSchema()).isEqualTo("my_schema");
assertThat(tableAsset.getName()).isEqualTo("my_table");
assertThat(tableAsset.getMetadata().get("owner")).isEqualTo("data-team");
}
@Factory
static class MockFactory {
@Singleton
@Replaces(AssetService.class)
public AssetService mockedAssetService() {
return Mockito.spy(new AssetService() {
@Override
public void asyncUpsert(AssetUser assetUser, Asset asset) {
capturedAsyncCreate.add(asset);
}
@Override
public void assetLineage(AssetUser assetUser, List<AssetIdentifier> inputs, List<AssetIdentifier> outputs) {
capturedAssetLineage.add(Pair.of(assetUser, Pair.of(inputs, outputs)));
}
});
}
@Singleton
@Replaces(AssetManagerFactory.class)
public AssetManagerFactory mockedAssetManagerFactory() {
return Mockito.spy(new AssetManagerFactory() {
@Override
public AssetEmitter of(boolean enabled) {
if (!enabled) {
return new AssetEmitter() {
@Override
public void upsert(Asset asset) {
capturedDisabledDynamicAssets.add(asset);
}
@Override
public List<Asset> outputs() {
return new ArrayList<>();
}
};
}
return new AssetEmitter() {
private final List<Asset> localCapturedAssets = new CopyOnWriteArrayList<>();
@Override
public void upsert(Asset asset) {
localCapturedAssets.add(asset);
capturedEnabledDynamicAssets.add(asset);
}
@Override
public List<Asset> outputs() {
return localCapturedAssets;
}
};
}
});
}
}
}

View File

@@ -1,30 +0,0 @@
package io.kestra.core.runners.test;
import io.kestra.core.models.annotations.Plugin;
import io.kestra.core.models.annotations.PluginProperty;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.assets.TableAsset;
import io.kestra.core.models.tasks.*;
import io.kestra.core.runners.RunContext;
import jakarta.validation.constraints.NotNull;
import lombok.*;
import lombok.experimental.SuperBuilder;
@SuperBuilder
@ToString
@EqualsAndHashCode
@Getter
@NoArgsConstructor
@Plugin
public class AssetEmitter extends Task implements RunnableTask<VoidOutput> {
@NotNull
@PluginProperty
private Asset assetToEmit;
@Override
public VoidOutput run(RunContext runContext) throws Exception {
runContext.assets().upsert(assetToEmit);
return null;
}
}

View File

@@ -1,7 +1,5 @@
package io.kestra.core.validations;
import io.kestra.core.models.assets.AssetIdentifier;
import io.kestra.core.models.assets.AssetsDeclaration;
import io.kestra.core.models.flows.Flow;
import io.kestra.core.models.flows.GenericFlow;
import io.kestra.core.models.validations.ModelValidator;
@@ -9,9 +7,7 @@ import io.kestra.core.serializers.YamlParser;
import io.kestra.core.tenant.TenantService;
import io.kestra.core.utils.TestsUtils;
import io.kestra.core.junit.annotations.KestraTest;
import io.kestra.plugin.core.log.Log;
import jakarta.inject.Inject;
import jakarta.validation.ConstraintViolation;
import org.junit.jupiter.api.Test;
import io.kestra.core.models.validations.ValidateConstraintViolation;
import io.kestra.core.services.FlowService;
@@ -233,31 +229,6 @@ class FlowValidationTest {
assertThat(validate.get().getMessage()).contains("Duplicate preconditions with id [flows]");
}
@Test
void eeAllowsDefiningAssets() {
Flow flow = Flow.builder()
.id(TestsUtils.randomString())
.namespace(TestsUtils.randomNamespace())
.tasks(List.of(
Log.builder()
.id("log")
.type(Log.class.getName())
.message("any")
.assets(io.kestra.core.models.property.Property.ofValue(
new AssetsDeclaration(true, List.of(new AssetIdentifier(null, null, "anyId")), null))
)
.build()
))
.build();
Optional<ConstraintViolationException> violations = modelValidator.isValid(flow);
assertThat(violations.isPresent()).isEqualTo(true);
assertThat(violations.get().getConstraintViolations().stream().map(ConstraintViolation::getMessage)).satisfiesExactly(
message -> assertThat(message).contains("Task 'log' can't have any `assets` because assets are only available in Enterprise Edition.")
);
};
private Flow parse(String path) {
URL resource = TestsUtils.class.getClassLoader().getResource(path);
assert resource != null;
@@ -266,4 +237,4 @@ class FlowValidationTest {
return YamlParser.parse(file, Flow.class);
}
}
}

View File

@@ -1,83 +0,0 @@
id: assets
namespace: io.kestra.tests
tasks:
- id: emit-asset
type: io.kestra.core.runners.test.AssetEmitter
assetToEmit:
namespace: io.kestra.tests
id: assets-flow-emit-asset-uid
type: TABLE
displayName: My Table Asset
description: This is my table asset
system: MY_DB_SYSTEM
database: my_database
schema: my_schema
name: my_table
metadata:
owner: data-team
assets:
outputs:
- id: assets-flow-static-emit-asset
type: TABLE
displayName: My Static Table Asset
description: This is my static table asset
system: MY_DB_SYSTEM
database: my_database
schema: my_schema
name: my_static_table
metadata:
owner: data-team
- id: emit-asset-auto-false
type: io.kestra.core.runners.test.AssetEmitter
assets:
enableAuto: false
assetToEmit:
namespace: io.kestra.tests
id: assets-flow-emit-asset-auto-false-uid
type: TABLE
displayName: My Table Asset
description: This is my table asset
system: MY_DB_SYSTEM
database: my_database
schema: my_schema
name: my_table
metadata:
owner: data-team
# Expected to create an 'EXTERNAL' asset automatically as it doesn't exist
- id: static-asset-non-existing-input
type: io.kestra.plugin.core.debug.Return
format: "whatever"
assets:
inputs:
- id: assets-flow-static-asset-non-existing-input-asset-uid
outputs:
- namespace: io.kestra.tests
id: assets-flow-static-asset-non-existing-output-uid
type: TABLE
displayName: My Static Table Asset
description: This is my static table asset
system: MY_DB_SYSTEM
database: my_database
schema: my_schema
name: my_static_table
metadata:
owner: data-team
- id: static-asset-existing-input
type: io.kestra.plugin.core.debug.Return
format: "whatever"
assets:
inputs:
- id: assets-flow-static-asset-existing-input-uid
outputs:
- namespace: io.kestra.tests
id: assets-flow-static-asset-existing-output-uid
type: TABLE
displayName: My Static Table Asset
description: This is my static table asset
system: MY_DB_SYSTEM
database: my_database
schema: my_schema
name: my_static_table
metadata:
owner: data-team

View File

@@ -4,10 +4,6 @@ import io.kestra.core.debug.Breakpoint;
import io.kestra.core.exceptions.InternalException;
import io.kestra.core.metrics.MetricRegistry;
import io.kestra.core.models.Label;
import io.kestra.core.models.assets.AssetIdentifier;
import io.kestra.core.models.assets.AssetUser;
import io.kestra.core.models.assets.AssetsDeclaration;
import io.kestra.core.models.assets.AssetsInOut;
import io.kestra.core.models.executions.*;
import io.kestra.core.models.flows.FlowInterface;
import io.kestra.core.models.flows.FlowWithSource;
@@ -100,12 +96,6 @@ public class ExecutorService {
@Named(QueueFactoryInterface.WORKERTASKLOG_NAMED)
private QueueInterface<LogEntry> logQueue;
@Inject
private AssetService assetService;
@Inject
private RunContextInitializer runContextInitializer;
protected FlowMetaStoreInterface flowExecutorInterface() {
// bean is injected late, so we need to wait
if (this.flowExecutorInterface == null) {
@@ -906,35 +896,21 @@ public class ExecutorService {
boolean hasMockedWorkerTask = false;
record FixtureAndTaskRun(TaskFixture fixture, TaskRun taskRun) {}
if (executor.getExecution().getFixtures() != null) {
RunContext runContext = runContextInitializer.forExecutor((DefaultRunContext) runContextFactory.of(
executor.getFlow(),
executor.getExecution()
));
RunContext runContext = runContextFactory.of(executor.getFlow(), executor.getExecution());
List<WorkerTaskResult> workerTaskResults = executor.getExecution()
.getTaskRunList()
.stream()
.filter(taskRun -> taskRun.getState().getCurrent().isCreated())
.flatMap(taskRun -> executor.getExecution().getFixtureForTaskRun(taskRun).stream().map(fixture -> new FixtureAndTaskRun(fixture, taskRun)))
.map(throwFunction(fixtureAndTaskRun -> {
Optional<AssetsDeclaration> renderedAssetsDeclaration = runContext.render(executor.getFlow().findTaskByTaskId(fixtureAndTaskRun.taskRun.getTaskId()).getAssets()).as(AssetsDeclaration.class);
return WorkerTaskResult.builder()
.taskRun(fixtureAndTaskRun.taskRun()
.withState(Optional.ofNullable(fixtureAndTaskRun.fixture().getState()).orElse(State.Type.SUCCESS))
.withOutputs(
variablesService.of(StorageContext.forTask(fixtureAndTaskRun.taskRun),
fixtureAndTaskRun.fixture().getOutputs() == null ? null : runContext.render(fixtureAndTaskRun.fixture().getOutputs()))
)
.withAssets(new AssetsInOut(
renderedAssetsDeclaration.map(AssetsDeclaration::getInputs).orElse(Collections.emptyList()).stream()
.map(assetIdentifier -> assetIdentifier.withTenantId(executor.getFlow().getTenantId()))
.toList(),
fixtureAndTaskRun.fixture().getAssets() == null ? null : fixtureAndTaskRun.fixture().getAssets().stream()
.map(asset -> asset.withTenantId(executor.getFlow().getTenantId()))
.toList()
))
)
.build();
}
.map(throwFunction(fixtureAndTaskRun -> WorkerTaskResult.builder()
.taskRun(fixtureAndTaskRun.taskRun()
.withState(Optional.ofNullable(fixtureAndTaskRun.fixture().getState()).orElse(State.Type.SUCCESS))
.withOutputs(
variablesService.of(StorageContext.forTask(fixtureAndTaskRun.taskRun),
fixtureAndTaskRun.fixture().getOutputs() == null ? null : runContext.render(fixtureAndTaskRun.fixture().getOutputs()))
)
)
.build()
))
.toList();
@@ -1196,47 +1172,6 @@ public class ExecutorService {
metricRegistry.tags(workerTaskResult)
)
.record(taskRun.getState().getDurationOrComputeIt());
if (
!taskRun.getState().isFailed()
&& taskRun.getAssets() != null &&
(!taskRun.getAssets().getInputs().isEmpty() || !taskRun.getAssets().getOutputs().isEmpty())
) {
AssetUser assetUser = new AssetUser(
taskRun.getTenantId(),
taskRun.getNamespace(),
taskRun.getFlowId(),
newExecution.getFlowRevision(),
taskRun.getExecutionId(),
taskRun.getTaskId(),
taskRun.getId()
);
List<AssetIdentifier> outputIdentifiers = taskRun.getAssets().getOutputs().stream()
.map(asset -> asset.withTenantId(taskRun.getTenantId()))
.map(AssetIdentifier::of)
.toList();
List<AssetIdentifier> inputAssets = taskRun.getAssets().getInputs().stream()
.map(assetIdentifier -> assetIdentifier.withTenantId(taskRun.getTenantId()))
.toList();
try {
assetService.assetLineage(
assetUser,
inputAssets,
outputIdentifiers
);
} catch (QueueException e) {
log.warn("Unable to submit asset lineage event for {} -> {}", inputAssets, outputIdentifiers, e);
}
taskRun.getAssets().getOutputs().forEach(asset -> {
try {
assetService.asyncUpsert(assetUser, asset);
} catch (QueueException e) {
log.warn("Unable to submit asset upsert event for asset {}", asset.getId(), e);
}
});
}
}
}

View File

@@ -1,22 +0,0 @@
ALTER TABLE queues ALTER COLUMN "type" ENUM(
'io.kestra.core.models.executions.Execution',
'io.kestra.core.models.templates.Template',
'io.kestra.core.models.executions.ExecutionKilled',
'io.kestra.core.runners.WorkerJob',
'io.kestra.core.runners.WorkerTaskResult',
'io.kestra.core.runners.WorkerInstance',
'io.kestra.core.runners.WorkerTaskRunning',
'io.kestra.core.models.executions.LogEntry',
'io.kestra.core.models.triggers.Trigger',
'io.kestra.ee.models.audits.AuditLog',
'io.kestra.core.models.executions.MetricEntry',
'io.kestra.core.runners.WorkerTriggerResult',
'io.kestra.core.runners.SubflowExecutionResult',
'io.kestra.core.server.ClusterEvent',
'io.kestra.core.runners.SubflowExecutionEnd',
'io.kestra.core.models.flows.FlowInterface',
'io.kestra.core.runners.MultipleConditionEvent',
'io.kestra.ee.assets.AssetLineageEvent',
'io.kestra.ee.assets.AssetUpsertCommand',
'io.kestra.ee.assets.AssetStateEvent'
) NOT NULL

View File

@@ -1,22 +0,0 @@
ALTER TABLE queues MODIFY COLUMN `type` ENUM(
'io.kestra.core.models.executions.Execution',
'io.kestra.core.models.templates.Template',
'io.kestra.core.models.executions.ExecutionKilled',
'io.kestra.core.runners.WorkerJob',
'io.kestra.core.runners.WorkerTaskResult',
'io.kestra.core.runners.WorkerInstance',
'io.kestra.core.runners.WorkerTaskRunning',
'io.kestra.core.models.executions.LogEntry',
'io.kestra.core.models.triggers.Trigger',
'io.kestra.ee.models.audits.AuditLog',
'io.kestra.core.models.executions.MetricEntry',
'io.kestra.core.runners.WorkerTriggerResult',
'io.kestra.core.runners.SubflowExecutionResult',
'io.kestra.core.server.ClusterEvent',
'io.kestra.core.runners.SubflowExecutionEnd',
'io.kestra.core.models.flows.FlowInterface',
'io.kestra.core.runners.MultipleConditionEvent',
'io.kestra.ee.assets.AssetLineageEvent',
'io.kestra.ee.assets.AssetUpsertCommand',
'io.kestra.ee.assets.AssetStateEvent'
) NOT NULL;

View File

@@ -1,3 +0,0 @@
ALTER TYPE queue_type ADD VALUE IF NOT EXISTS 'io.kestra.ee.assets.AssetLineageEvent';
ALTER TYPE queue_type ADD VALUE IF NOT EXISTS 'io.kestra.ee.assets.AssetUpsertCommand';
ALTER TYPE queue_type ADD VALUE IF NOT EXISTS 'io.kestra.ee.assets.AssetStateEvent';

View File

@@ -328,10 +328,6 @@ public abstract class AbstractJdbcRepository {
return applyTriggerStateCondition(value, operation);
}
if (field.equals(QueryFilter.Field.METADATA)) {
return findMetadataCondition((Map<?, ?>) value, operation);
}
// Convert the field name to lowercase and quote it
Name columnName = getColumnName(field);
@@ -384,10 +380,6 @@ public abstract class AbstractJdbcRepository {
throw new InvalidQueryFiltersException("Unsupported operation: " + operation);
}
protected Condition findMetadataCondition(Map<?, ?> metadata, QueryFilter.Op operation) {
throw new InvalidQueryFiltersException("Unsupported operation: " + operation);
}
// Generate the condition for Field.STATE
@SuppressWarnings("unchecked")
private Condition generateStateCondition(Object value, QueryFilter.Op operation) {

View File

@@ -8,25 +8,31 @@ import io.micronaut.test.extensions.junit5.MicronautJunit5Extension;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.platform.commons.support.AnnotationSupport;
import java.util.Set;
public class KestraTestExtension extends MicronautJunit5Extension {
@Override
protected MicronautTestValue buildMicronautTestValue(Class<?> testClass) {
testProperties.put("kestra.jdbc.executor.thread-count", Runtime.getRuntime().availableProcessors() * 4);
return AnnotationSupport
.findAnnotation(testClass, KestraTest.class)
.map(kestraTestAnnotation -> new MicronautTestValue(
kestraTestAnnotation.application(),
kestraTestAnnotation.environments(),
kestraTestAnnotation.packages(),
kestraTestAnnotation.propertySources(),
kestraTestAnnotation.rollback(),
kestraTestAnnotation.transactional(),
kestraTestAnnotation.rebuildContext(),
kestraTestAnnotation.contextBuilder(),
kestraTestAnnotation.transactionMode(),
kestraTestAnnotation.startApplication(),
kestraTestAnnotation.resolveParameters()
))
.map(kestraTestAnnotation -> {
var envsSet = new java.util.HashSet<>(Set.of(kestraTestAnnotation.environments()));
envsSet.add("test");// add test env if not already present
return new MicronautTestValue(
kestraTestAnnotation.application(),
envsSet.toArray(new String[0]),
kestraTestAnnotation.packages(),
kestraTestAnnotation.propertySources(),
kestraTestAnnotation.rollback(),
kestraTestAnnotation.transactional(),
kestraTestAnnotation.rebuildContext(),
kestraTestAnnotation.contextBuilder(),
kestraTestAnnotation.transactionMode(),
kestraTestAnnotation.startApplication(),
kestraTestAnnotation.resolveParameters()
);
})
.orElse(null);
}

View File

@@ -74,7 +74,7 @@ abstract public class TestsUtils {
* @param prefix
* @return
*/
public static String randomString(String... prefix) {
private static String randomString(String... prefix) {
if (prefix.length == 0) {
prefix = new String[]{String.join("-", stackTraceToParts())};
}

3108
ui/package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -27,7 +27,7 @@
"@kestra-io/ui-libs": "^0.0.268",
"@vue-flow/background": "^1.3.2",
"@vue-flow/controls": "^1.1.2",
"@vue-flow/core": "^1.48.0",
"@vue-flow/core": "^1.48.1",
"@vueuse/core": "^14.1.0",
"ansi-to-html": "^0.7.2",
"axios": "^1.13.2",
@@ -39,7 +39,7 @@
"cytoscape": "^3.33.0",
"dagre": "^0.8.5",
"dotenv": "^17.2.3",
"element-plus": "2.12.0",
"element-plus": "2.13.0",
"humanize-duration": "^3.33.2",
"js-yaml": "^4.1.1",
"lodash": "^4.17.21",
@@ -59,15 +59,15 @@
"path-browserify": "^1.0.1",
"pdfjs-dist": "^5.4.449",
"pinia": "^3.0.4",
"posthog-js": "^1.308.0",
"posthog-js": "^1.310.1",
"rapidoc": "^9.3.8",
"semver": "^7.7.3",
"shiki": "^3.20.0",
"vue": "^3.5.25",
"vue": "^3.5.26",
"vue-axios": "^3.5.2",
"vue-chartjs": "^5.3.3",
"vue-gtag": "^3.6.3",
"vue-i18n": "^11.2.2",
"vue-i18n": "^11.2.7",
"vue-material-design-icons": "^5.3.1",
"vue-router": "^4.6.4",
"vue-sidebar-menu": "^5.9.1",
@@ -97,9 +97,9 @@
"@types/semver": "^7.7.1",
"@types/testing-library__jest-dom": "^6.0.0",
"@types/testing-library__user-event": "^4.2.0",
"@typescript-eslint/parser": "^8.50.0",
"@typescript-eslint/parser": "^8.50.1",
"@vitejs/plugin-vue": "^6.0.3",
"@vitejs/plugin-vue-jsx": "^5.1.2",
"@vitejs/plugin-vue-jsx": "^5.1.3",
"@vitest/browser": "^3.2.4",
"@vitest/coverage-v8": "^3.2.4",
"@vue/eslint-config-prettier": "^10.2.0",
@@ -120,29 +120,29 @@
"playwright": "^1.55.0",
"prettier": "^3.7.4",
"rimraf": "^6.1.2",
"rolldown-vite": "^7.2.11",
"rolldown-vite": "^7.3.0",
"rollup-plugin-copy": "^3.5.0",
"sass": "^1.97.0",
"storybook": "^9.1.16",
"sass": "^1.97.1",
"storybook": "^9.1.17",
"storybook-vue3-router": "^6.0.2",
"ts-node": "^10.9.2",
"typescript": "^5.9.3",
"typescript-eslint": "^8.50.0",
"typescript-eslint": "^8.50.1",
"uuid": "^13.0.0",
"vite": "npm:rolldown-vite@latest",
"vitest": "^3.2.4",
"vue-tsc": "^3.1.8"
"vue-tsc": "^3.2.1"
},
"optionalDependencies": {
"@esbuild/darwin-arm64": "^0.27.2",
"@esbuild/darwin-x64": "^0.27.2",
"@esbuild/linux-x64": "^0.27.2",
"@rollup/rollup-darwin-arm64": "^4.53.5",
"@rollup/rollup-darwin-x64": "^4.53.5",
"@rollup/rollup-linux-x64-gnu": "^4.53.5",
"@swc/core-darwin-arm64": "^1.15.5",
"@swc/core-darwin-x64": "^1.15.5",
"@swc/core-linux-x64-gnu": "^1.15.5"
"@rollup/rollup-darwin-arm64": "^4.54.0",
"@rollup/rollup-darwin-x64": "^4.54.0",
"@rollup/rollup-linux-x64-gnu": "^4.54.0",
"@swc/core-darwin-arm64": "^1.15.7",
"@swc/core-darwin-x64": "^1.15.7",
"@swc/core-linux-x64-gnu": "^1.15.7"
},
"overrides": {
"bootstrap": {

View File

@@ -43,7 +43,7 @@
REF_PATH_INJECTION_KEY,
ROOT_SCHEMA_INJECTION_KEY,
SCHEMA_DEFINITIONS_INJECTION_KEY,
UPDATE_TASK_FUNCTION_INJECTION_KEY
UPDATE_YAML_FUNCTION_INJECTION_KEY
} from "../../no-code/injectionKeys";
import {NoCodeProps} from "../../flows/noCodeTypes";
import {deepEqual} from "../../../utils/utils";
@@ -68,7 +68,7 @@
dashboardStore.sourceCode = YAML_UTILS.stringify(app);
}
provide(UPDATE_TASK_FUNCTION_INJECTION_KEY, (yaml) => {
provide(UPDATE_YAML_FUNCTION_INJECTION_KEY, (yaml) => {
editorUpdate(yaml)
})

View File

@@ -1,16 +1,13 @@
<template>
<el-tooltip placement="bottom" :content="t('playground.tooltip_persistence')">
<el-switch v-model="playgroundStore.enabled" :activeText="t('playground.toggle')" class="toggle" :class="{'is-active': playgroundStore.enabled}" />
<el-tooltip placement="bottom" :content="$t('playground.tooltip_persistence')">
<el-switch v-model="playgroundStore.enabled" :activeText="$t('playground.toggle')" class="toggle" :class="{'is-active': playgroundStore.enabled}" />
</el-tooltip>
</template>
<script setup lang="ts">
import {useI18n} from "vue-i18n";
import {usePlaygroundStore} from "../../stores/playground";
const {t} = useI18n();
const playgroundStore = usePlaygroundStore();
</script>
<style scoped lang="scss">

View File

@@ -6,7 +6,7 @@
<Keyboard />
</el-icon>
<span class="fs-6">
{{ t("editor_shortcuts.label") }}
{{ $t("editor_shortcuts.label") }}
</span>
</div>
</template>
@@ -27,7 +27,7 @@
</template>
</div>
<div class="text-break">
{{ t(command.description) }}
{{ $t(command.description) }}
</div>
</div>
</div>
@@ -35,11 +35,9 @@
</template>
<script setup lang="ts">
import {useI18n} from "vue-i18n";
import Keyboard from "vue-material-design-icons/Keyboard.vue";
import {useKeyShortcuts} from "../../utils/useKeyShortcuts";
const {t} = useI18n();
const {isKeyShortcutsDialogShown} = useKeyShortcuts();
const commands = [

View File

@@ -97,7 +97,6 @@
<script setup lang="ts">
import {
computed,
getCurrentInstance,
h,
inject,
onBeforeUnmount,
@@ -126,7 +125,7 @@
import uniqBy from "lodash/uniqBy";
import {useI18n} from "vue-i18n";
import {ElDatePicker} from "element-plus";
import {Moment} from "moment";
import moment, {Moment} from "moment";
import PlaceholderContentWidget from "../../composables/monaco/PlaceholderContentWidget";
import Utils from "../../utils/utils";
import {hashCode} from "../../utils/global";
@@ -137,7 +136,6 @@
import EditorType = editor.EditorType;
import {useRoute} from "vue-router";
const currentInstance = getCurrentInstance()!;
const {t} = useI18n();
const textAreaValue = computed({
@@ -371,8 +369,7 @@
}
}, {deep: true});
const nowMoment: Moment = currentInstance.appContext.config.globalProperties.$moment().startOf("day");
const nowMoment: Moment = moment().startOf("day");
function addedSuggestRows(mutations: MutationRecord[]) {
return mutations.flatMap(({addedNodes}) => {
const nodes = [...addedNodes];
@@ -461,7 +458,7 @@
endColumn: wordAtPosition?.endColumn ?? position?.column
},
// We don't use the selectedDate directly because if user modifies the input value directly it doesn't work otherwise
text: `${currentInstance.appContext.config.globalProperties.$moment(
text: `${moment(
datePicker.value!.$el.nextElementSibling.querySelector("input").value
).toISOString(true)} `,
forceMoveMarkers: true

View File

@@ -1,11 +1,11 @@
<template>
<nav class="d-flex align-items-center w-100 gap-3 top-bar">
<SidebarToggleButton
v-if="layoutStore.sideMenuCollapsed"
@toggle="layoutStore.setSideMenuCollapsed(false)"
/>
<div class="d-flex flex-column flex-grow-1 flex-shrink-1 overflow-hidden top-title">
<div class="d-flex align-items-end gap-2">
<SidebarToggleButton
v-if="layoutStore.sideMenuCollapsed"
@toggle="layoutStore.setSideMenuCollapsed(false)"
/>
<div class="d-flex flex-column gap-2">
<el-breadcrumb v-if="breadcrumb">
<el-breadcrumb-item v-for="(item, x) in breadcrumb" :key="x" :class="{'pe-none': item.disabled}">

View File

@@ -43,7 +43,9 @@
BLOCK_SCHEMA_PATH_INJECTION_KEY,
CLOSE_TASK_FUNCTION_INJECTION_KEY,
CREATE_TASK_FUNCTION_INJECTION_KEY,
CREATING_FLOW_INJECTION_KEY,
CREATING_TASK_INJECTION_KEY,
DEFAULT_NAMESPACE_INJECTION_KEY,
EDIT_TASK_FUNCTION_INJECTION_KEY,
EDITING_TASK_INJECTION_KEY,
FIELDNAME_INJECTION_KEY,
@@ -55,7 +57,7 @@
REF_PATH_INJECTION_KEY,
ROOT_SCHEMA_INJECTION_KEY,
SCHEMA_DEFINITIONS_INJECTION_KEY,
UPDATE_TASK_FUNCTION_INJECTION_KEY,
UPDATE_YAML_FUNCTION_INJECTION_KEY,
} from "./injectionKeys";
import {useFlowFields, SECTIONS_IDS} from "./utils/useFlowFields";
import debounce from "lodash/debounce";
@@ -65,6 +67,7 @@
import {useKeyboardSave} from "./utils/useKeyboardSave";
import {deepEqual} from "../../utils/utils";
import {useScrollMemory} from "../../composables/useScrollMemory";
import {defaultNamespace} from "../../composables/useNamespaces";
const props = defineProps<NoCodeProps>();
@@ -166,6 +169,8 @@
provide(REF_PATH_INJECTION_KEY, props.refPath);
provide(PANEL_INJECTION_KEY, panel)
provide(POSITION_INJECTION_KEY, props.position ?? "after");
provide(CREATING_FLOW_INJECTION_KEY, flowStore.isCreating ?? false);
provide(DEFAULT_NAMESPACE_INJECTION_KEY, computed(() => flowStore.flow?.namespace ?? defaultNamespace() ?? "company.team"));
provide(CREATING_TASK_INJECTION_KEY, props.creatingTask);
provide(EDITING_TASK_INJECTION_KEY, props.editingTask);
provide(FIELDNAME_INJECTION_KEY, props.fieldName);
@@ -184,7 +189,7 @@
emit("closeTask")
})
provide(UPDATE_TASK_FUNCTION_INJECTION_KEY, (yaml) => {
provide(UPDATE_YAML_FUNCTION_INJECTION_KEY, (yaml) => {
editorUpdate(yaml)
})

View File

@@ -44,7 +44,6 @@
<script setup lang="ts">
import {computed, inject, ref} from "vue";
import {BLOCK_SCHEMA_PATH_INJECTION_KEY} from "../../injectionKeys";
import {useFlowStore} from "../../../../stores/flow";
import Creation from "./taskList/buttons/Creation.vue";
import Element from "./taskList/Element.vue";
import * as YAML_UTILS from "@kestra-io/ui-libs/flow-yaml-utils";
@@ -53,7 +52,7 @@
import {
CREATING_TASK_INJECTION_KEY, FULL_SCHEMA_INJECTION_KEY, FULL_SOURCE_INJECTION_KEY,
PARENT_PATH_INJECTION_KEY, REF_PATH_INJECTION_KEY,
PARENT_PATH_INJECTION_KEY, REF_PATH_INJECTION_KEY, UPDATE_YAML_FUNCTION_INJECTION_KEY,
} from "../../injectionKeys";
import {SECTIONS_MAP} from "../../../../utils/constants";
import {getValueAtJsonPath} from "../../../../utils/utils";
@@ -83,8 +82,6 @@
inheritAttrs: false
});
const flowStore = useFlowStore();
interface Task {
id:string,
type:string
@@ -150,6 +147,8 @@
const movedIndex = ref(-1);
const updateYaml = inject(UPDATE_YAML_FUNCTION_INJECTION_KEY, () => {});
const moveElement = (
items: Record<string, any>[] | undefined,
elementID: string,
@@ -171,7 +170,7 @@
movedIndex.value = -1;
}, 200);
flowStore.flowYaml =
updateYaml(
YAML_UTILS.swapBlocks({
source:flow.value,
section: SECTIONS_MAP[section.value.toLowerCase() as keyof typeof SECTIONS_MAP],
@@ -179,6 +178,7 @@
key2:items[newIndex][keyName],
keyName,
})
);
};
const fullSchema = inject(FULL_SCHEMA_INJECTION_KEY, ref<Record<string, any>>({}));

View File

@@ -8,18 +8,17 @@
</template>
<script lang="ts" setup>
import {computed, onMounted} from "vue";
import {useFlowStore} from "../../../../stores/flow";
import {onMounted, inject, computed, provide} from "vue";
import NamespaceSelect from "../../../namespaces/components/NamespaceSelect.vue";
import {CREATING_FLOW_INJECTION_KEY, DEFAULT_NAMESPACE_INJECTION_KEY} from "../../injectionKeys";
const modelValue = defineModel<string>();
const flowStore = useFlowStore();
const isCreating = computed(() => flowStore.isCreating);
const isCreating = inject(CREATING_FLOW_INJECTION_KEY, false);
const defaultNamespace = inject(DEFAULT_NAMESPACE_INJECTION_KEY, computed(() => ""));
provide(DEFAULT_NAMESPACE_INJECTION_KEY, computed(() => modelValue.value || defaultNamespace.value));
onMounted(() => {
const flowNamespace = flowStore.flow?.namespace;
const flowNamespace = defaultNamespace.value;
if (!modelValue.value && flowNamespace) {
modelValue.value = flowNamespace;
}

View File

@@ -24,6 +24,7 @@ export const POSITION_INJECTION_KEY = Symbol("position-injection-key") as Inject
* NOTE: different from the `isCreating` flag coming from the store. `isCreating` refers to the Complete flow being in creation
*/
export const CREATING_TASK_INJECTION_KEY = Symbol("creating-injection-key") as InjectionKey<boolean>
export const CREATING_FLOW_INJECTION_KEY = Symbol("creating-flow-injection-key") as InjectionKey<boolean>
/**
* When creating anew task, allows to specify a field where the new task should be injected.
* @example
@@ -51,9 +52,9 @@ export const EDIT_TASK_FUNCTION_INJECTION_KEY = Symbol("edit-function-injection-
*/
export const CLOSE_TASK_FUNCTION_INJECTION_KEY = Symbol("close-function-injection-key") as InjectionKey<() => void>
/**
* We call this function when a task is changed, as soon as the first click or type is done
* Call this function to update the full Yaml content
*/
export const UPDATE_TASK_FUNCTION_INJECTION_KEY = Symbol("update-function-injection-key") as InjectionKey<(yaml: string) => void>
export const UPDATE_YAML_FUNCTION_INJECTION_KEY = Symbol("update-function-injection-key") as InjectionKey<(yaml: string) => void>
/**
* Set this to override the contents of the no-code editor with a component of your choice
* This is used to display the metadata edition inputs
@@ -92,4 +93,6 @@ export const SCHEMA_DEFINITIONS_INJECTION_KEY = Symbol("schema-definitions-injec
export const DATA_TYPES_MAP_INJECTION_KEY = Symbol("data-types-injection-key") as InjectionKey<ComputedRef<Record<string, string[] | undefined>>>
export const ON_TASK_EDITOR_CLICK_INJECTION_KEY = Symbol("on-task-editor-click-injection-key") as InjectionKey<(elt?: Partial<NoCodeElement>) => void>;
export const ON_TASK_EDITOR_CLICK_INJECTION_KEY = Symbol("on-task-editor-click-injection-key") as InjectionKey<(elt?: Partial<NoCodeElement>) => void>;
export const DEFAULT_NAMESPACE_INJECTION_KEY = Symbol("default-namespace-injection-key") as InjectionKey<ComputedRef<string>>;

View File

@@ -16,7 +16,7 @@
import {PLUGIN_DEFAULTS_SECTION, SECTIONS_MAP} from "../../../utils/constants";
import {
CLOSE_TASK_FUNCTION_INJECTION_KEY,
UPDATE_TASK_FUNCTION_INJECTION_KEY,
UPDATE_YAML_FUNCTION_INJECTION_KEY,
FULL_SOURCE_INJECTION_KEY, CREATING_TASK_INJECTION_KEY,
PARENT_PATH_INJECTION_KEY, POSITION_INJECTION_KEY,
REF_PATH_INJECTION_KEY, EDIT_TASK_FUNCTION_INJECTION_KEY,
@@ -37,7 +37,7 @@
const fieldName = inject(FIELDNAME_INJECTION_KEY, undefined);
const blockSchemaPath = inject(BLOCK_SCHEMA_PATH_INJECTION_KEY, ref(""));
const updateTask = inject(UPDATE_TASK_FUNCTION_INJECTION_KEY, () => {})
const updateTask = inject(UPDATE_YAML_FUNCTION_INJECTION_KEY, () => {})
const closeTaskAddition = inject(
CLOSE_TASK_FUNCTION_INJECTION_KEY,

View File

@@ -3,7 +3,7 @@
<template #additional-right>
<ul>
<li>
<el-button v-if="canCreate" tag="router-link" :to="{name: 'flows/create', query: {namespace: $route.query.namespace}}" :icon="Plus" type="primary">
<el-button v-if="canCreate" tag="router-link" :to="{name: 'flows/create', query: {namespace: $route.query.namespace}}" :icon="Plus" type="secondary">
{{ $t('create_flow') }}
</el-button>
</li>
@@ -30,7 +30,7 @@
<el-button
v-if="isOSS"
@click="startTour"
:icon="Plus"
:icon="Compass"
size="large"
type="primary"
class="px-3 p-4 section-1-link product-link"
@@ -39,7 +39,7 @@
</el-button>
<el-button
v-else
:icon="Plus"
:icon="Compass"
tag="router-link"
:to="{name: 'flows/create'}"
size="large"
@@ -74,6 +74,7 @@
import {useCoreStore} from "../../stores/core";
import {useI18n} from "vue-i18n";
import Plus from "vue-material-design-icons/Plus.vue";
import Compass from "vue-material-design-icons/Compass.vue";
import Play from "vue-material-design-icons/Play.vue";
import OnboardingBottom from "override/components/OnboardingBottom.vue";
import kestraWelcome from "../../assets/onboarding/kestra_welcome.svg";

View File

@@ -156,7 +156,7 @@
<el-form-item :label="$t('secret.key')" prop="key">
<el-input v-model="secret.key" :disabled="secret.update" required />
</el-form-item>
<el-form-item v-if="!secret.update" :label="$t('secret.name')" prop="value">
<el-form-item v-if="!secret.update" :label="$t('secret.name')" prop="value" required>
<MultilineSecret v-model="secret.value" :placeholder="secretModalTitle" />
</el-form-item>
<el-form-item v-if="secret.update" :label="$t('secret.name')" prop="value">

View File

@@ -1,3 +1,4 @@
import {computed, watch} from "vue";
import {useI18n} from "vue-i18n";
import {configureMonacoYaml} from "monaco-yaml";
import * as monaco from "monaco-editor/esm/vs/editor/editor.api";
@@ -21,6 +22,7 @@ import {
registerPebbleAutocompletion
} from "./pebbleLanguageConfigurator";
import {usePluginsStore} from "../../../stores/plugins";
import {useBlueprintsStore} from "../../../stores/blueprints";
import {languages} from "monaco-editor/esm/vs/editor/editor.api";
import CompletionItem = languages.CompletionItem;
@@ -34,11 +36,14 @@ export class YamlLanguageConfigurator extends AbstractLanguageConfigurator {
}
async configureLanguage(pluginsStore: ReturnType<typeof usePluginsStore>) {
const validateYAML = computed(() => useBlueprintsStore().validateYAML);
watch(validateYAML, (shouldValidate) => configureMonacoYaml(monaco, {validate: shouldValidate}));
configureMonacoYaml(monaco, {
enableSchemaRequest: true,
hover: localStorage.getItem("hoverTextEditor") === "true",
completion: true,
validate: true,
validate: validateYAML.value ?? true,
format: true,
schemas: yamlSchemas()
});

View File

@@ -1,5 +1,11 @@
<template>
<SideBar v-if="menu" :menu :showLink="showLink" @menu-collapse="onCollapse">
<SideBar
v-if="menu"
:menu
:showLink
@menu-collapse="onCollapse"
:class="{overlay: verticalLayout}"
>
<template #footer>
<Auth />
</template>
@@ -11,6 +17,9 @@
import SideBar from "../../components/layout/SideBar.vue";
import Auth from "../../override/components/auth/Auth.vue";
import {useBreakpoints, breakpointsElement} from "@vueuse/core";
const verticalLayout = useBreakpoints(breakpointsElement).smallerOrEqual("sm");
withDefaults(defineProps<{
showLink?: boolean
}>(), {

View File

@@ -56,7 +56,10 @@
<div v-if="!system && blueprint.tags?.length > 0" class="tags-section">
<span v-for="tag in processedTags(blueprint.tags)" :key="tag.original" class="tag-item">{{ tag.display }}</span>
</div>
<div class="text-section">
<div v-if="blueprint.template" class="tags-section">
<span class="tag-item">{{ $t('template') }}</span>
</div>
<div class="text-section">
<h3 class="title">
{{ blueprint.title ?? blueprint.id }}
</h3>
@@ -151,6 +154,7 @@
id: string;
tags: string[];
title?: string;
template?: Record<string, any>;
}[] | undefined>(undefined);
const error = ref(false);
const icon = {ContentCopy};

View File

@@ -25,6 +25,8 @@ interface Blueprint {
[key: string]: any;
}
export type TemplateArgument = Record<string, Input>;
export interface BlueprintTemplate {
source: string;
templateArguments: Record<string, Input>;
@@ -55,6 +57,8 @@ export const useBlueprintsStore = defineStore("blueprints", () => {
const source = ref<string | undefined>(undefined);
const graph = ref<any | undefined>(undefined);
const validateYAML = ref<boolean>(true); // Used to enable/disable YAML validation in Monaco editor, for the purpose of Templated Blueprints
const getBlueprints = async (options: Options) => {
const PARAMS = {params: options.params, ...VALIDATE};
@@ -166,6 +170,8 @@ export const useBlueprintsStore = defineStore("blueprints", () => {
source,
graph,
validateYAML,
getBlueprints,
getBlueprint,
getBlueprintSource,

View File

@@ -1,6 +1,10 @@
@import "@kestra-io/ui-libs/src/scss/variables.scss";
#app {
.v-sidebar-menu.vsm_expanded.overlay {
position: absolute;
}
.vsm--item {
padding: 0 30px;
transition: padding 0.2s ease;

View File

@@ -883,6 +883,12 @@
"description": "Nach Trigger-ID filtern",
"label": "Trigger-ID"
},
"triggerState": {
"description": "Nach Trigger-Zustand filtern",
"disabled": "Deaktiviert",
"enabled": "Aktiviert",
"label": "Trigger-Zustand"
},
"update": "Aktualisieren",
"value": "Wert",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "Weiter",
"no_flows": "Keine Flows unter der tutorial Namespace verfügbar.",
"previous": "Zurück",
"skip": "Tutorial überspringen",
"skip": "Produkt-Tour überspringen",
"steps": {
"0": {
"content": "Wir freuen uns, dass Sie hier sind.<br />Lassen Sie uns Ihren ersten Flow erstellen.",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 Willkommen bei Kestra!",
"welcome aboard content": "Nutzen Sie unsere geführte Tour, um Ihren ersten Flow zu erstellen, und schauen Sie sich Blueprints an, um weitere Beispiele zu finden.",
"welcome button create": "Meinen ersten Flow erstellen",
"welcome button create": "Produkt-Tour starten",
"welcome display require": "Führen Sie Ihren <strong>ersten Flow</strong> aus, um loszulegen",
"welcome_page": {
"guide": "Benötigen Sie Unterstützung, um Ihren ersten flow auszuführen?",

View File

@@ -540,7 +540,7 @@
"welcome aboard": "\uD83D\uDE80 Welcome to Kestra!",
"welcome aboard content": "Use our Guided Tour to create your first flow and check Blueprints to find more examples.",
"welcome display require": "Run your <strong>first flow</strong> to get started",
"welcome button create": "Create my first flow",
"welcome button create": "Start Product Tour",
"live help": "Live help",
"show task documentation": "Show task documentation",
"hide task documentation": "Hide task documentation",
@@ -947,7 +947,7 @@
"next": "Next",
"previous": "Previous",
"finish": "Finish",
"skip": "Skip Tutorial",
"skip": "Skip Product Tour",
"no_flows": "No flows available under tutorial namespace.",
"steps": {
"0": {

View File

@@ -883,6 +883,12 @@
"description": "Filtrar por trigger ID",
"label": "ID de Trigger"
},
"triggerState": {
"description": "Filtrar por estado del trigger",
"disabled": "Desactivado",
"enabled": "Habilitado",
"label": "Estado del Trigger"
},
"update": "Actualizar",
"value": "Valor",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "Siguiente",
"no_flows": "No hay flows disponibles bajo el namespace del tutorial.",
"previous": "Anterior",
"skip": "Saltar Tutorial",
"skip": "Omitir recorrido del producto",
"steps": {
"0": {
"content": "Estamos encantados de tenerte aquí.<br />Vamos a crear tu primer flow.",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 ¡Bienvenido a Kestra!",
"welcome aboard content": "Usa nuestro Tour Guiado para crear tu primer flow y revisa los Blueprints para encontrar más ejemplos.",
"welcome button create": "Crear mi primer flow",
"welcome button create": "Iniciar Tour del Producto",
"welcome display require": "Ejecuta tu <strong>primer flow</strong> para comenzar",
"welcome_page": {
"guide": "¿Necesitas orientación para ejecutar tu primer flow?",

View File

@@ -883,6 +883,12 @@
"description": "Filtrer par trigger ID",
"label": "ID du trigger"
},
"triggerState": {
"description": "Filtrer par état du trigger",
"disabled": "Désactivé",
"enabled": "Activé",
"label": "État du Trigger"
},
"update": "Mettre à jour",
"value": "Valeur",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "Suivant",
"no_flows": "Aucun flux disponible dans l'espace de noms du tutoriel.",
"previous": "Précédent",
"skip": "Passer le tutoriel",
"skip": "Passer la visite guidée du produit",
"steps": {
"0": {
"content": "Nous sommes ravis de vous avoir ici.<br />Créons votre premier flux.",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 Bienvenue à bord !",
"welcome aboard content": "Tout est prêt pour Kestra, commencez la création de votre flow et admirez la magie !",
"welcome button create": "Créer mon premier flow",
"welcome button create": "Démarrer la visite guidée du produit",
"welcome display require": "Prêt à commencer à utiliser Kestra ? Créons ensemble <strong>votre premier flow</strong> !",
"welcome_page": {
"guide": "Besoin d'aide pour exécuter votre premier flow ?",

View File

@@ -883,6 +883,12 @@
"description": "ट्रिगर ID द्वारा फ़िल्टर करें",
"label": "ट्रिगर ID"
},
"triggerState": {
"description": "ट्रिगर स्थिति द्वारा फ़िल्टर करें",
"disabled": "अक्षम",
"enabled": "सक्रिय",
"label": "ट्रिगर स्थिति"
},
"update": "अपडेट",
"value": "मान",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "अगला",
"no_flows": "ट्यूटोरियल namespace के अंतर्गत कोई flows उपलब्ध नहीं हैं।",
"previous": "पिछला",
"skip": "ट्यूटोरियल छोड़ें",
"skip": "उत्पाद टूर छोड़ें",
"steps": {
"0": {
"content": "हमें खुशी है कि आप यहाँ हैं।<br />आइए अपना पहला flow बनाएं।",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 केस्ट्रा में आपका स्वागत है!",
"welcome aboard content": "अपना पहला flow बनाने के लिए हमारे Guided Tour का उपयोग करें और अधिक उदाहरण खोजने के लिए Blueprints देखें।",
"welcome button create": "मेरा पहला flow बनाएं",
"welcome button create": "उत्पाद यात्रा शुरू करें",
"welcome display require": "शुरू करने के लिए अपना <strong>पहला flow</strong> चलाएँ",
"welcome_page": {
"guide": "क्या आपको अपना पहला flow निष्पादित करने के लिए मार्गदर्शन चाहिए?",

View File

@@ -883,6 +883,12 @@
"description": "Filtra per trigger ID",
"label": "ID del trigger"
},
"triggerState": {
"description": "Filtra per stato del trigger",
"disabled": "Disabilitato",
"enabled": "Abilitato",
"label": "Stato del Trigger"
},
"update": "Aggiorna",
"value": "Valore",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "Successivo",
"no_flows": "Nessun flow disponibile sotto il namespace del tutorial.",
"previous": "Precedente",
"skip": "Salta Tutorial",
"skip": "Salta il Tour del Prodotto",
"steps": {
"0": {
"content": "Siamo entusiasti di averti qui.<br />Creiamo il tuo primo flow.",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 Benvenuto su Kestra!",
"welcome aboard content": "Usa il nostro Tour Guidato per creare il tuo primo flow e controlla i Blueprints per trovare altri esempi.",
"welcome button create": "Crea il mio primo flow",
"welcome button create": "Inizia il Tour del Prodotto",
"welcome display require": "Esegui il tuo <strong>primo flow</strong> per iniziare",
"welcome_page": {
"guide": "Hai bisogno di assistenza per eseguire il tuo primo flow?",

View File

@@ -883,6 +883,12 @@
"description": "トリガーIDでフィルター",
"label": "トリガーID"
},
"triggerState": {
"description": "トリガー状態でフィルター",
"disabled": "無効",
"enabled": "有効",
"label": "トリガー状態"
},
"update": "更新",
"value": "値",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "次へ",
"no_flows": "チュートリアルnamespaceに利用可能なflowはありません。",
"previous": "前へ",
"skip": "チュートリアルをスキップ",
"skip": "プロダクトツアーをスキップ",
"steps": {
"0": {
"content": "ここに来てくれてとても嬉しいです。<br />最初のflowを作成しましょう。",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 Kestraへようこそ",
"welcome aboard content": "ガイド付きツアーを利用して最初のflowを作成し、Blueprintsでさらに多くの例を見つけてください。",
"welcome button create": "最初のflowを作成",
"welcome button create": "プロダクトツアーを開始",
"welcome display require": "<strong>最初のflow</strong>を実行して始めましょう",
"welcome_page": {
"guide": "最初のflowを実行するためのガイダンスが必要ですか",

View File

@@ -883,6 +883,12 @@
"description": "트리거 ID로 필터링",
"label": "트리거 ID"
},
"triggerState": {
"description": "트리거 상태별 필터링",
"disabled": "비활성화됨",
"enabled": "사용 가능",
"label": "트리거 상태"
},
"update": "업데이트",
"value": "값",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "다음",
"no_flows": "튜토리얼 namespace에 사용할 수 있는 flow가 없습니다.",
"previous": "이전",
"skip": "튜토리얼 건너뛰기",
"skip": "제품 투어 건너뛰기",
"steps": {
"0": {
"content": "여기 오신 것을 환영합니다.<br />첫 번째 flow를 만들어 봅시다.",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 Kestra에 오신 것을 환영합니다!",
"welcome aboard content": "가이드 투어를 사용하여 첫 번째 flow를 만들고 Blueprints에서 더 많은 예제를 확인하세요.",
"welcome button create": "첫 번째 flow 만들기",
"welcome button create": "제품 투어 시작",
"welcome display require": "<strong>첫 번째 flow</strong>를 실행하여 시작하세요",
"welcome_page": {
"guide": "첫 번째 flow를 실행하는 데 도움이 필요하신가요?",

View File

@@ -883,6 +883,12 @@
"description": "Filtruj według trigger ID",
"label": "Identyfikator Trigger"
},
"triggerState": {
"description": "Filtruj według stanu triggera",
"disabled": "Wyłączone",
"enabled": "Włączone",
"label": "Stan Trigger"
},
"update": "Aktualizuj",
"value": "Wartość",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "Następny",
"no_flows": "Brak flowów w namespace o nazwie tutorial.",
"previous": "Poprzedni",
"skip": "Pomiń samouczek",
"skip": "Pomiń Przewodnik po Produkcie",
"steps": {
"0": {
"content": "Jesteśmy zachwyceni, że jesteś tutaj.<br />Stwórzmy twój pierwszy flow.",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 Witamy w Kestra!",
"welcome aboard content": "Skorzystaj z naszego Przewodnika, aby stworzyć swój pierwszy flow i sprawdź Blueprints, aby znaleźć więcej przykładów.",
"welcome button create": "Stwórz mój pierwszy flow",
"welcome button create": "Rozpocznij Przewodnik po Produkcie",
"welcome display require": "Uruchom swój <strong>pierwszy flow</strong>, aby rozpocząć",
"welcome_page": {
"guide": "Potrzebujesz wskazówek, jak uruchomić swój pierwszy flow?",

View File

@@ -883,6 +883,12 @@
"description": "Filtrar por trigger ID",
"label": "ID do Trigger"
},
"triggerState": {
"description": "Filtrar por estado do trigger",
"disabled": "Desativado",
"enabled": "Habilitado",
"label": "Estado do Trigger"
},
"update": "Atualizar",
"value": "Valor",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "Próximo",
"no_flows": "Não há flows disponíveis no namespace do tutorial.",
"previous": "Anterior",
"skip": "Pular Tutorial",
"skip": "Pular Tour do Produto",
"steps": {
"0": {
"content": "Estamos entusiasmados em tê-lo aqui.<br />Vamos criar seu primeiro flow.",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 Bem-vindo ao Kestra!",
"welcome aboard content": "Use nosso Tour Guiado para criar seu primeiro flow e confira os Blueprints para encontrar mais exemplos.",
"welcome button create": "Criar meu primeiro flow",
"welcome button create": "Iniciar Tour do Produto",
"welcome display require": "Execute seu <strong>primeiro flow</strong> para começar",
"welcome_page": {
"guide": "Precisa de orientação para executar seu primeiro flow?",

View File

@@ -883,6 +883,12 @@
"description": "Filtrar por trigger ID",
"label": "ID do Trigger"
},
"triggerState": {
"description": "Filtrar por estado do trigger",
"disabled": "Desativado",
"enabled": "Habilitado",
"label": "Estado do Trigger"
},
"update": "Atualizar",
"value": "Valor",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "Próximo",
"no_flows": "Não há flows disponíveis no namespace do tutorial.",
"previous": "Anterior",
"skip": "Pular Tutorial",
"skip": "Pular Tour do Produto",
"steps": {
"0": {
"content": "Estamos entusiasmados em tê-lo aqui.<br />Vamos criar seu primeiro flow.",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 Bem-vindo ao Kestra!",
"welcome aboard content": "Use nosso Tour Guiado para criar seu primeiro flow e confira os Blueprints para encontrar mais exemplos.",
"welcome button create": "Criar meu primeiro flow",
"welcome button create": "Iniciar Tour do Produto",
"welcome display require": "Execute seu <strong>primeiro flow</strong> para começar",
"welcome_page": {
"guide": "Precisa de orientação para executar seu primeiro flow?",

View File

@@ -883,6 +883,12 @@
"description": "Фильтр по trigger ID",
"label": "ID триггера"
},
"triggerState": {
"description": "Фильтр по состоянию trigger",
"disabled": "Отключено",
"enabled": "Включено",
"label": "Состояние Trigger"
},
"update": "Обновить",
"value": "Значение",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "Далее",
"no_flows": "Нет доступных flows в namespace учебника.",
"previous": "Назад",
"skip": "Пропустить учебник",
"skip": "Пропустить ознакомительный тур с продуктом",
"steps": {
"0": {
"content": "Мы рады видеть вас здесь.<br />Давайте создадим ваш первый flow.",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 Добро пожаловать в Kestra!",
"welcome aboard content": "Используйте наше Руководство, чтобы создать ваш первый flow и ознакомьтесь с Blueprints для поиска дополнительных примеров.",
"welcome button create": "Создать мой первый flow",
"welcome button create": "Начать ознакомительный тур с продуктом",
"welcome display require": "Запустите ваш <strong>первый flow</strong>, чтобы начать",
"welcome_page": {
"guide": "Нужна помощь в выполнении вашего первого flow?",

View File

@@ -883,6 +883,12 @@
"description": "按 trigger ID 筛选",
"label": "触发器 ID"
},
"triggerState": {
"description": "按触发器状态筛选",
"disabled": "禁用",
"enabled": "启用",
"label": "触发状态"
},
"update": "更新",
"value": "值",
"workerId": {
@@ -1308,7 +1314,7 @@
"next": "下一步",
"no_flows": "教程命名空间下没有可用的流程。",
"previous": "上一步",
"skip": "跳过教程",
"skip": "跳过产品指南",
"steps": {
"0": {
"content": "我们很高兴你在这里。<br />让我们创建你的第一个流程。",
@@ -1857,7 +1863,7 @@
},
"welcome aboard": "🚀 欢迎使用 Kestra!",
"welcome aboard content": "使用我们的引导游览来创建你的第一个流程,并查看蓝图以找到更多示例。",
"welcome button create": "创建我的第一个流程",
"welcome button create": "开始产品导览",
"welcome display require": "运行你的 <strong>第一个流程</strong> 以开始",
"welcome_page": {
"guide": "需要指导来执行您的第一个flow吗",

View File

@@ -9,9 +9,6 @@ import io.kestra.core.exceptions.DeserializationException;
import io.kestra.core.exceptions.IllegalVariableEvaluationException;
import io.kestra.core.metrics.MetricRegistry;
import io.kestra.core.models.Label;
import io.kestra.core.models.assets.Asset;
import io.kestra.core.models.assets.AssetsDeclaration;
import io.kestra.core.models.assets.AssetsInOut;
import io.kestra.core.models.executions.*;
import io.kestra.core.models.flows.State;
import io.kestra.core.models.tasks.Output;
@@ -957,15 +954,7 @@ public class DefaultWorker implements Worker {
try {
Variables variables = variablesService.of(StorageContext.forTask(taskRun), workerTaskCallable.getTaskOutput());
if (workerTask.getTask().getAssets() != null) {
List<Asset> outputAssets = runContext.assets().outputs();
Optional<AssetsDeclaration> renderedAssetsDeclaration = runContext.render(workerTask.getTask().getAssets()).as(AssetsDeclaration.class);
renderedAssetsDeclaration.map(AssetsDeclaration::getOutputs).ifPresent(outputAssets::addAll);
taskRun = taskRun.withOutputs(variables).withAssets(new AssetsInOut(
renderedAssetsDeclaration.map(AssetsDeclaration::getInputs).orElse(null),
outputAssets
));
}
taskRun = taskRun.withOutputs(variables);
} catch (Exception e) {
logger.warn("Unable to save output on taskRun '{}'", taskRun, e);
}