Commit Graph

198 Commits

Author SHA1 Message Date
Loïc Mathieu
318f2b7d5a chore(cli,jdbc-postgres): fix some compilation warnings 2025-03-31 13:17:35 +02:00
Loïc Mathieu
7696d41d5f fix(jdbc): resubmit worker job to the good worker group
Fixes #7730
2025-03-07 16:44:28 +01:00
aeSouid
2bb5ac61c2 Labels multiselect: (#7544)
combine the labels filter by operation to handle multiselect when building the query
2025-02-25 16:50:38 +01:00
Loïc Mathieu
cf10269f25 fix(jdbc): be resilient to DataException
We usually fail fast, but when a DataException is thrown it means the JDBC driver throws an exception with error code 22: data exception.
As the exception is from the data not the database or the network, there is no point of failfast, we throw a QueueException that may or may not be handled gracefully by the call site.
2025-02-24 16:56:10 +01:00
Loïc Mathieu
13ac335c93 feat(jdbc): clean more eagerly some queues based on configuration 2025-02-12 16:35:46 +01:00
Loïc Mathieu
9d717caf95 feat(jdbc): queues.updated should be set when the record is updated 2025-02-12 16:35:46 +01:00
aeSouid
296e1a67a5 Feat Search: (#6910)
New search with operation for all /search endpoints :
- define list  of operation and fields.
- Add custom parser for filter[field][operation]=value format
- create new Annotation @QueryFilterFormat to bind to specfic pojo
- Implement new abstract repo search method
2025-02-06 17:08:15 +01:00
Loïc Mathieu
671eb2b57e feat(jdbc-*): delete subflow_executions table
Fixes #6831
2025-02-04 10:34:39 +01:00
YannC
f1e2eea13d fix(): Handle correctly group by date field type (#7033)
close #6977
2025-01-29 13:07:12 +01:00
Loïc Mathieu
7cf4955814 feat(core, jdbc): change the state of a subflow restart parent execution 2025-01-27 11:29:17 +01:00
Florian Hussonnois
7b0f2e9160 fix(jdbc): update test config for flaky test on liveness (#6656) 2025-01-15 18:29:53 +01:00
Loïc Mathieu
9901470b74 feat(*): maintenance mode
Closes #2992
2025-01-06 17:45:10 +01:00
YannC
54b7a29a55 feat(): jdbc custom dashboard implementation (#6607)
* WIP: commit to be edited

* feat(*): finished implementation for custom dashboard in JDBC

* chore: disable metricRepository all() test because of leap year

* chore(): PR review
2025-01-06 14:02:53 +01:00
Ludovic DEHON
5064606350 fix(jdbc-postgres): escape special chars on full text search
close kestra-io/kestra-ee#2430
close kestra-io/kestra-ee#2454
2024-12-17 08:53:52 +01:00
YannC
2d4fb7684a fix(jdbc): missing SKIPPED state in DB migrations (#6487)
close #6461
2024-12-16 16:04:34 +01:00
brian-mulier-p
4943f9ab6a feat(*): introduce custom dashboards (#6144)
closes kestra-io/kestra-ee#1711

Co-authored-by: MilosPaunovic <paun992@hotmail.com>
2024-11-29 11:35:01 +01:00
Loïc Mathieu
13425326dd Revert "feat(core): remove the execution state from the scheduler (#1588)"
This reverts commit f7d3d0bcd4.
2024-11-28 18:24:52 +01:00
Loïc Mathieu
4e0b6308bd feat(core): flow SLA
Fixes #5857
2024-11-25 16:05:45 +01:00
Loïc Mathieu
34fa6ce910 feat(jdbc): Improve execution queued performance
Add date inside the index to speed up order by in case there are a lot of execution queued.
Skip locked records when selecting them as if there is a locked records it means you need to pop the next one.
2024-11-22 16:13:41 +01:00
Loïc Mathieu
5f6e04ea0b feat(jdbc): add a where clause when fetching max offset
Also some refactoring to reduce code duplication
2024-11-07 15:34:08 +01:00
Loïc Mathieu
730b2cac39 feat(core): MultipleConditon deadline
Fixes #5706
2024-10-30 12:23:58 +01:00
Loïc Mathieu
5228ccf0f8 feat(core, jdbc): introduce a JDBC indexer 2024-10-14 09:28:41 +02:00
Loïc Mathieu
9f686a28cc feat(core,jdbc,webserver): plugin default from Flow source
This would avoid having default values set in the Flow object (JSON) representation so allows to set default that override the object default value (like the one from Lombok `@Builder.Default`).

Fixes #2260
Fixes #2797
2024-10-08 09:04:57 +02:00
YannC
82d2170fff fix(jdbc): compatible with Postgres 14 (#5019)
close #4825
2024-09-23 13:51:04 +02:00
Loïc Mathieu
d40c5b35b8 fix(jdbc): forEachItem tests that are flaky with SQLServer 2024-09-11 20:06:36 +02:00
Loïc Mathieu
7b73eed068 feat: Implements message protection for the JDBC Queue
Allow to set a limit in size for messages, when exceeded, the message will be refused by the Queue.
The JDBC executor and worker has been updated to handle a too big message and fail the execution.
2024-09-11 20:06:36 +02:00
Loïc Mathieu
1d7982406c feat(jdbc-postgres): improve JSONB performance 2024-09-11 17:50:46 +02:00
Loïc Mathieu
90c1c71e42 fix(jdbc-postgres): remove not-needed serialization in Postgres queue 2024-08-19 09:15:08 +02:00
Loïc Mathieu
3881d778a4 fix(core,jdbc): PurgeLog with levels in postgres
Fixes #4604
2024-08-08 12:57:50 +02:00
Loïc Mathieu
2416760fe9 feat(core,jdbc,webserver): find trigger by worker id
Part-of: https://github.com/kestra-io/kestra-ee/issues/1299
2024-07-15 17:41:18 +02:00
Ludovic DEHON
bc98de3519 refactor(core): remove lots of compilation warning 2024-06-24 00:02:59 +02:00
Ludovic DEHON
e9f8a8f3c2 feat(core): use a standard logger for unit test 2024-06-21 15:05:57 +02:00
Ludovic DEHON
bd5d5c5758 feat(core): move test utils to a proper package 2024-06-20 22:58:40 +02:00
Loïc Mathieu
76f7038b52 feat: Kestra Platform BOM 2024-06-20 15:15:22 +02:00
brian-mulier-p
b15d2cf409 feat(memory): remove memory runner and introduce @KestraTest (#3948) 2024-06-12 19:17:57 +02:00
Florian Hussonnois
bc34aeb16c chore(jdbc): cleanup QueueInterface 2024-06-10 19:01:21 +02:00
Florian Hussonnois
c06938f155 chore(core): remove useless method pause for WorkerJobQueueInterface 2024-06-10 10:15:58 +02:00
Ludovic DEHON
5fd7a2b3a3 feat(cicd): publish allure report 2024-06-08 22:39:14 +02:00
brian.mulier
e6b89ab77c fix(queues): add forUpdate boolean to receive method to allow non-locking receiving 2024-06-05 16:49:25 +02:00
brian.mulier
cc64e526cc fix(core): hold cancellation runnable references to run them upon shutdown 2024-06-05 16:49:25 +02:00
Loïc Mathieu
3d82f1435a feat(jdbc-*): drop worker_instance table
It is not used anymore since multiple releases.

Fixes #3498
2024-04-30 17:26:36 +02:00
Florian Hussonnois
4feb91c6fb build(gradle): use gradle configurations for micronaut deps 2024-04-24 16:15:08 +02:00
Loïc Mathieu
da63207cdb chore(*): upgrade dependencies 2024-04-09 15:29:13 +02:00
YannC
e58ff958a0 feat(core): retry flowable (#3473)
* feat(core): retry flowable

* feat(core): migrate retry memory runner test to jdbc/kafka runner test

* feat(core): memory executor delayType use
2024-04-08 16:43:21 +02:00
YannC
81f281e92b feat(): retry-flow (#3392)
* feat(): retry-flow

* fix: rename behavior

* feat: created a metadata props for executions

* fix(ui): translate + new metadata prop
2024-04-03 15:42:39 +02:00
Florian Hussonnois
5e48602477 chore(core): fix bean lifecycle to properly handle context close (#3318)
Fix: #3318
2024-03-27 14:17:16 +01:00
YannC
96f1eef502 feat(): retry revamp (#3322)
* feat(): init revamp

* feat(): first working iteration

* feat(): first working iteration

* feat(): cleanup code

* feat(): correct implem of new retrying state

* feat(): memoryExecutor + handle errors

* fix(core): handle allow failure

* fix(): better coverage + change random

* fix: review changes

* fix: prevent flow from continuing while retrying
2024-03-25 10:35:45 +01:00
Florian Hussonnois
4e810abef9 feat(core): enchance liveness core api to support metrics
This commit also removes the admin/service view
2024-03-22 09:51:32 +01:00
Florian Hussonnois
015659ae1b feat(core): enhance Kestra services liveness mechanism
part-of: #3055
2024-03-18 17:11:00 +01:00
Loïc Mathieu
46e3fa015d feat(jdbc*): add an index on logs.timestamp
Fixes #2376

An issue exists in Postgres when we request ERROR logs on a wide timestamp range, it's cardinality calculation leads to ignore the existing indexes. Adding an index only on the timestamp column is a last resort for queries not finding existing indexes useful.
The same index is added to the other databases even if we didn't have clues on the same issue for consistency.
2024-03-05 14:05:21 +01:00