57 Commits

Author SHA1 Message Date
Loïc Mathieu
9a4b569d85 feat(storage): limit object name to 255 inside the local storage
Part-of: #12273
2025-11-10 12:24:49 +01:00
Nicolas K.
cc5f73ae06 wip(storages): add non tenant dependant method to storage interface (#10637)
* wip(storages): add non tenant dependant method to storage interface

* feat(storages): #10636 add instance method to retrieve resources without the tenant id

* fix(stores): #4353 failing unit tests after now that tenant id can't be null

---------

Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-08-13 11:00:25 +02:00
Nicolas K.
4f8ea9afb1 feat(storage): refactor getPath (#9104)
Co-authored-by: nKwiatkowski <nkwiatkowski@kestra.io>
2025-05-28 14:12:58 +02:00
Loïc Mathieu
fc8732f96e chore: use @Nullable from Jakarata annotations 2025-04-07 17:01:52 +02:00
Loïc Mathieu
b7f9a32015 fix(storage-local): path traversal guard should include File.separator
Today, we check that a file didn't contains '..' which is too aggressive, we should check that it didn't contains '../' or '..\' only.
2025-01-03 10:06:52 +01:00
Hyoseop Song
ba0e770a05 Add subpath guard function to avoid non-relative subpath parsed URI 2024-12-04 10:40:32 +01:00
Loïc Mathieu
78f5ff2b7b feat(*): add namespace as a parameter of the internal storage 2024-11-21 12:09:07 +01:00
YannC
b399907016 fix(core): encode filename (#5593)
close #5589
2024-10-22 16:05:45 +02:00
brian.mulier
59f10d452a fix(core): windows backslashes in paths were leading to wrong URI being created leading to error upon execution deletion 2024-10-07 11:15:05 +02:00
YannC
4d46ebd364 fix(local-storage): Set windows path to linux path (#5248)
close #5247
2024-09-30 20:45:07 +02:00
YannC
e5cd7d2c2f fix(): use string instead of path to be Windows compatible for namespace files (#4471) 2024-07-30 11:39:53 +02:00
brian.mulier
4f385ddbab feat(core): Backend + Tasks + Pebble function for KV-Store 2024-07-05 18:32:39 +02:00
Ludovic DEHON
e9f8a8f3c2 feat(core): use a standard logger for unit test 2024-06-21 15:05:57 +02:00
Ludovic DEHON
bd5d5c5758 feat(core): move test utils to a proper package 2024-06-20 22:58:40 +02:00
YannC
1bb4e723c5 fix(): namespaces files working in script tasks on Windows 2024-06-17 11:11:30 +02:00
YannC
78ec3c1bc4 fix(): replace Windows \ for / in LocalStorage 2024-06-12 14:09:09 +02:00
Ludovic DEHON
5fd7a2b3a3 feat(cicd): publish allure report 2024-06-08 22:39:14 +02:00
brian.mulier
21d6408439 fix(storage-local): add log in case of failing file visit 2024-05-24 15:21:12 +02:00
brian.mulier
326ec8f052 fix(storage-local): regression on LocalStorage allByPrefix 2024-05-24 15:12:19 +02:00
brian.mulier
0ab909446b fix(webserver): prevent zombie folders when deleting namespace files and there is no more file in it
closes kestra-io/terraform-provider-kestra#98
2024-05-24 13:22:38 +02:00
Florian Hussonnois
ce11b937f5 chore(core): remove Introspected from StorageInterface 2024-04-29 16:46:42 +02:00
Florian Hussonnois
b381ee53fc chore(core): migrate StorageInterface to service-loader mechanism
part-of: kestra-io/storage-minio#80
part-of: kestra-io/storage-s3#80
part-of: kestra-io/storage-gcs#119
part-of: kestra-io/storage-azure#53
2024-04-29 15:47:32 +02:00
Florian Hussonnois
4feb91c6fb build(gradle): use gradle configurations for micronaut deps 2024-04-24 16:15:08 +02:00
Florian Hussonnois
56884d5a72 refactor(core): migrate to standard Java ServiceLoader for plugins
Changes:
- add new top level interface `Plugin`
- add new `JsonDeserializer` aware of the `PluginRegistry`
- add new annotation processor for plugin
- migrate to hibernate-validator for the JSR-303 (no bean introspection dependencies)
- update all Validation annotations to be complitant with the JSR-303
- remove all KestraApplicationContext stuffs
- fix some tests

part-of: kestra-io/plugin-template-test#25
part-of: kestra-io/plugin-serdes#103
2024-04-24 16:15:08 +02:00
Loïc Mathieu
5cc33c90f3 fix(core): rename test application.yml to application-test.yml (#2225) 2024-01-15 11:07:11 +01:00
Ludovic DEHON
1e1785db7e chore(refactor): remove a bunch of compilation warning and remove deprecated StorageInterface method 2024-01-11 12:56:42 +01:00
brian.mulier
58467f6618 feat(core): all by prefix to also get directories 2023-12-15 09:18:05 +01:00
brian.mulier
2ff0862ad3 fix(storage): handle non-existing entry for filesByPrefix 2023-12-06 12:29:27 +01:00
brian.mulier
e27a99bc9a fix(storage): returning full uris for file search 2023-12-06 12:29:27 +01:00
brian.mulier
587c7d91fe feat(vscode): file search by name
closes #2368
2023-12-06 12:29:27 +01:00
brian.mulier
76d8619d06 fix(localstorage): nullable uris + added recursive directory creation in test suite 2023-11-08 18:03:47 +01:00
Martin
bfdc7d394a chore(core,storage): add a test suite for storage (#2352)
Storages have some behavior discrepencies. This PR add a test suite that
will attemp to resolve it by having the same test base on all storagei
type.
2023-10-20 15:48:39 +02:00
Martin
af42856a11 feat(core,webserver): multi file editor (#2234)
Add endpoint and method to storage interface to allow the new file
editor to create/move/delete and edit files and directories in a new
storage prefix.
2023-10-17 15:16:06 +02:00
Loïc Mathieu
0dc270d6c2 feat(core): multi tenant (#2217) 2023-10-12 14:40:38 +02:00
Loïc Mathieu
bde5ff692e feat: allow to cache contents inside the working directory 2023-07-28 11:45:40 +02:00
Ludovic DEHON
2e32fce598 fix(core): PurgeExecution tasks failed if there is no storage usage
close #518
2022-07-03 22:43:56 +02:00
Ludovic DEHON
8673accad6 feat(core): add a retry on storage interface 2022-01-26 21:43:38 +01:00
tchiotludo
a19df970bf chore(deps): update all java deps 2021-12-17 10:30:59 +01:00
tchiotludo
f2946f906b fix(core): fix flakky test 2021-05-31 12:15:28 +02:00
tchiotludo
d110782f50 fix(core): handle not found on Storage.size 2021-05-28 14:52:24 +02:00
tchiotludo
aea3e928e7 feat(tasks): add PurgeExecution storage tasks 2021-05-18 14:52:45 +02:00
Ludovic DEHON
b55ac08ba9 chore(core): move from org.kestra to io.kestra (#306) 2021-03-24 16:06:29 +01:00
eric regnier
0cb75dcf7f feat(ui): show file size in ui (#274)
close #270

Co-authored-by: tchiotludo <tchiot.ludo@gmail.com>
2021-02-24 22:32:04 +01:00
Ludovic DEHON
eb419e1cac feat(deps): update to java deps (#222)
* update to micronaut 2.2 
* update to kafka 2.7 
* update to elastic 7.10
2021-01-20 13:01:38 +01:00
tchiotludo
5a3acef783 feat(core): move all documentation to JsonSchema 2020-10-27 11:23:57 +01:00
tchiotludo
716df081f9 test(storage-local): add a proper test on delete operation 2020-09-17 10:51:54 +02:00
tchiotludo
55c2cfe2da feat(tasks): add a storage.delete tasks to allow deletion of internal storage file 2020-07-03 11:43:27 +02:00
Kwang-in (Dennis) Jung
8fe7457679 fix(storage-local): check if local dir is created
relate to #102
2020-03-28 14:23:41 +01:00
tchiotludo
3acca1c20c fix(core): use buffered input when possible 2020-03-06 14:52:57 +01:00
Ludovic DEHON
b2912f50dd feat(core): standardize input type file
remove exception for `inputs.file.uri` to `inputs.file` to be consistent 

relate to #78
2020-03-02 18:29:22 +01:00