1
0
mirror of synced 2026-01-24 07:01:51 -05:00
Commit Graph

53 Commits

Author SHA1 Message Date
Octavia Squidington III
b9de100058 Bump Airbyte version from 0.40.28 to 0.40.29 (#21767)
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
2023-01-23 23:22:45 -08:00
Pedro S. Lopez
220f193c3b fix loading remote definitions via cron (#21410)
* make loading definitions part of the public DefinitionsProvider API

* fix failing test

* use micronaut caching

* revert

* cache the fetch method instead

* remove final

* configure cache to 15s on cron
2023-01-14 00:49:20 +00:00
Octavia Squidington III
02e0258948 Bump Airbyte version from 0.40.27 to 0.40.28 (#21359)
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
2023-01-12 16:47:07 -08:00
Jonathan Pearlin
7676af5f5f Migrate airbyte-bootloader to Micronaut (#21073)
* Migrate airbyte-bootloader to Micronaut

* PR feedback

* More PR feedback

* Rename variable for clarity

* Add properties to support cloud

* Formatting

* Use default values for env vars

* Re-organization to support cloud overrides

* Disable conditional logging

* More singleton cleanup

* test: try CI without fluentbit

* Revert "test: try CI without fluentbit"

This reverts commit 8fa0f74106.

* test: enable SSH on EC2 runner

* Revert "test: enable SSH on EC2 runner"

This reverts commit e4867aae09.

* Avoid early database connection on startup

* Fix compile issues from refactor

* Formatting

Co-authored-by: perangel <perangel@gmail.com>
2023-01-12 09:47:25 -05:00
Octavia Squidington III
00fb4bba09 Bump Airbyte version from 0.40.26 to 0.40.27 (#21135)
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
2023-01-06 14:27:01 -08:00
Jonathan Pearlin
31c28f5956 Use explicit configuration for Micronaut endpoints (#20759) 2023-01-06 14:16:36 -05:00
Jimmy Ma
04e6566527 Only add definitions updater cron when enabled (#20838)
* Only add definitions updater cron when enabled

* Remove unused property
2023-01-03 20:19:26 +00:00
Octavia Squidington III
27bd807fd1 Bump Airbyte version from 0.40.25 to 0.40.26 (#20630)
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
2022-12-18 15:54:55 -08:00
Marcos Marx
e29d5ff0e2 Airbyte Cron: renable schedule to file cleaner (#20317)
* readd schedule to cleaner

* import scheduled
2022-12-11 17:37:34 -03:00
Octavia Squidington III
f5e15dee77 Bump Airbyte version from 0.40.24 to 0.40.25 (#20279)
Co-authored-by: girarda <girarda@users.noreply.github.com>
2022-12-08 17:30:50 -08:00
Octavia Squidington III
1581821930 Bump Airbyte version from 0.40.23 to 0.40.24 (#20206)
Co-authored-by: girarda <girarda@users.noreply.github.com>
2022-12-07 15:44:56 -08:00
Octavia Squidington III
77112b02bf Bump Airbyte version from 0.40.22 to 0.40.23 (#19941)
Co-authored-by: davinchia <davinchia@users.noreply.github.com>
2022-11-30 20:02:24 +00:00
Octavia Squidington III
4c5d6ead94 Bump Airbyte version from 0.40.21 to 0.40.22 (#19687)
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
Co-authored-by: lmossman <lake@airbyte.io>
2022-11-21 17:43:32 -08:00
Octavia Squidington III
8341076cb3 Bump Airbyte version from 0.40.20 to 0.40.21 (#19634)
Co-authored-by: krishnaglick <krishnaglick@users.noreply.github.com>
2022-11-18 20:39:06 -05:00
Octavia Squidington III
ed4ec64712 Bump Airbyte version from 0.40.19 to 0.40.20 (#19614)
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
2022-11-18 11:53:57 -08:00
Octavia Squidington III
061bce197a Bump Airbyte version from 0.40.18 to 0.40.19 (#19579)
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
2022-11-18 00:26:01 -08:00
Octavia Squidington III
b299688f3b Bump Airbyte version from 0.40.17 to 0.40.18 (#18827)
Co-authored-by: grishick <grishick@users.noreply.github.com>
2022-11-01 19:35:23 -07:00
Charles
87a0a2a27c hide ConfigPersistence inside ConfigRepository to discourage use (#18803) 2022-11-01 16:51:09 -07:00
Charles
20ab047ae0 Remove the bulk actions from ConfigPersistence (#18800) 2022-11-01 16:50:03 -07:00
Jonathan Pearlin
657f997aae Update to latest Temporal SDK release (#18492) 2022-10-26 12:54:00 -04:00
Octavia Squidington III
73336ff4de Bump Airbyte version from 0.40.16 to 0.40.17 (#18316)
Co-authored-by: edgao <edgao@users.noreply.github.com>
2022-10-21 14:00:39 -07:00
Octavia Squidington III
ce0bbf3f75 Bump Airbyte version from 0.40.15 to 0.40.16 (#18268)
Co-authored-by: evantahler <evantahler@users.noreply.github.com>
2022-10-20 13:29:54 -07:00
Jonathan Pearlin
eed3257c9b Use constructor injection for singleton (#18163) 2022-10-19 11:59:09 -04:00
Jonathan Pearlin
b4c16bfdbb Custom APM Tracing (#17947)
* Add APM tracing to airbyte-workers

* Add custom traces to airbyte-cron

* Formatting

* Refactor tag constants

* Handle potential null object

* Handle null tag values
2022-10-18 13:14:29 -04:00
Octavia Squidington III
a5a0409ad5 Bump Airbyte version from 0.40.14 to 0.40.15 (#17970)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-10-17 13:33:20 -07:00
Benoit Moriceau
7a71c55c1f Bmoric/remove dep server worker (#17894)
* test [ci skip]

* Autogenerated files

* Add missing annotation

* Remove unused json2Schema block from worker

* Move tess

* Missing deps and format

* Fix test build

* TMP

* Add missing dependencies

* PR comments

* Tmp

* [ci skip] Tmp

* Fix acceptance test and add the seed dependency

* Fix build

* For diff

* tmp

* Build pass

* make the worker to be  on the platform only

* fix setting.yaml

* Fix pmd

* Fix Cron

* Add chart

* Fix cron

* Fix server build.gradle

* Fix jar conflict

* PR comments

* Add cron micronaut environemnt
2022-10-17 12:35:04 -07:00
Jonathan Pearlin
246172bd6c Use the Airbyte Base Java Image for all services (#17843)
* Use the Amazon Corretto based base image for all services

* Add comment

* Use airbyte-base-java-image
2022-10-12 08:59:02 -04:00
Jonathan Pearlin
e95a983563 Various Micronaut-related cleanup items (#17611) 2022-10-11 10:14:23 -04:00
Octavia Squidington III
9cb32437e4 Bump Airbyte version from 0.40.13 to 0.40.14 (#17762)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-10-07 14:18:21 -07:00
Benoit Moriceau
922527b4cf Revert "Use java-datadog-tracer-base image (#17625)" (#17759)
This reverts commit c170173334.
2022-10-07 13:38:45 -07:00
Octavia Squidington III
e382a6e6db Bump Airbyte version from 0.40.12 to 0.40.13 (#17682)
Co-authored-by: xiaohansong <xiaohansong@users.noreply.github.com>
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
2022-10-06 15:12:18 -07:00
Jonathan Pearlin
c170173334 Use java-datadog-tracer-base image (#17625)
* Use java-datadog-tracer-base image

* Restore build args

* Explicity install tar executable
2022-10-06 14:35:37 -04:00
Octavia Squidington III
71c8e3c062 Bump Airbyte version from 0.40.11 to 0.40.12 (#17653)
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
2022-10-05 18:25:09 -07:00
Pedro S. Lopez
69f53eb56a airbyte-cron: update connector definitions from remote (#16438)
* add db connection injection to airbyte-cron

* load definitions

* fix patch version check for non-semver tags

* apply helper tests

* logging updates

* fix remote definitions provider: add tombstones

* docker compose env updates

* add test for tombstone presence

* rename helper class

* config updates

* dont use optionals as fields

* use apply helper instead of directly calling load data in bootloader

* avoid pmd warn

* add docstring

* sort docker compose env vars

* updates for javax -> jakarta

* clean up docker compose, update kube deployment env
2022-10-05 17:21:37 -04:00
Octavia Squidington III
e333aa9365 Bump Airbyte version from 0.40.10 to 0.40.11 (#17610)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-10-05 08:35:39 -07:00
Benoit Moriceau
65a35b98bd Bmoric/clean applciation yml (#17353)
* Cleanup application.yml

* Clean up cron application.yml and kube overlay

* Avoid flooding the logs with waiting for temporal namespace message

* Rm common app initializer

* Try to re-introduce an initializer
2022-09-29 16:21:38 -07:00
Pedro S. Lopez
12b46694dc 🐛 cron/workspace-cleaner: only run task for docker environments (#17168)
* dont run cleanup on kubernetes

* use the micronaut way™

* add env var to kube resource

* use micronaut environment instead

* no need for the env var
2022-09-29 15:53:16 -04:00
Octavia Squidington III
1c40fe6644 Bump Airbyte version from 0.40.9 to 0.40.10 (#17359)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-09-28 17:09:25 -07:00
Benoit Moriceau
3c457e051d Bmoric/restore temporal cleaning cron (#17258)
* Revert "Revert "Bmoric/temporal cleaning cron (#16414)" (#17192)"

This reverts commit 751667949e.

* Generate an artifact

* Change missing property from value to property
2022-09-27 10:59:39 -07:00
Jonathan Pearlin
896c2c9751 Switch to jakarta annotations (#17251) 2022-09-27 11:18:22 -04:00
Benoit Moriceau
751667949e Revert "Bmoric/temporal cleaning cron (#16414)" (#17192)
This reverts commit bbb59d8675.
2022-09-26 16:58:23 -07:00
Benoit Moriceau
bbb59d8675 Bmoric/temporal cleaning cron (#16414)
* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Add the injected temporal client to airbyte-cron

* Update cron to make it work

* Remove useless Bean factory

* PR comments

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Improve conditional bean check

* Match existing Optional functionality

* Add notEquals check

* Add missing env var to Helm chart

* Fix typo

* Mark LogConfigs as Singleton

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Improve conditional bean check

* Match existing Optional functionality

* Add notEquals check

* Add missing env var to Helm chart

* Fix typo

* Mark LogConfigs as Singleton

* Env vars for log/state storage type

* Remove use of Optional in bean declarations

* Fix typo in config property name

* Support Temporal Cloud namespace

* Change to @Value

* Use correct value for conditional check

* Upgrade Micronaut

* Fix merge conflict

* Formatting

* Add missing env var

* Use sync task queue environment variable

* Handle sync task queue as set

* format and force http

* Handle case where sync task queue is empty

* Add correct path to config property

* Remove unused import

* Remove conflict

* Remove unused parameter

* Formatting

* Use pattern for condition process factory beans

* Cleanup

* PR feedback

* Revert hack for testing

* Fix temporal restart by status (#16447)

* Update application.yml

* Re add worker dep

* Add missing env var

* PR comments

* Bmoric/move temporal client (#16778)

* tmp

* tmp

* View diff

* Move part of the temporal client

* tmp

* copy Temporal Utils test

* Uniq Temporal Utils

* Uniq Temporal Workflow Utils

* Move CancellationHandler

* Move commons-temporal to being shared between platform, connector and CLI

* Rm worker dependency from cron

* Fix build

* Update with right value for cron.

* Fix dep conflict

Co-authored-by: jdpgrailsdev <jpearlin1@gmail.com>
Co-authored-by: pmossman <parker@airbyte.io>
Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io>
2022-09-26 15:14:35 -07:00
Octavia Squidington III
8dfb8b0c92 Bump Airbyte version from 0.40.8 to 0.40.9 (#17093)
Co-authored-by: terencecho <terencecho@users.noreply.github.com>
2022-09-23 09:46:09 -05:00
Octavia Squidington III
387b96a924 Bump Airbyte version from 0.40.7 to 0.40.8 (#17009)
Co-authored-by: gosusnp <gosusnp@users.noreply.github.com>
2022-09-21 15:57:29 -07:00
Volodymyr Pochtar
a72cb7d1f8 feat: replace openjdk with amazoncorretto:17.0.4 on platform for security compliance (#16919) 2022-09-20 19:01:48 +03:00
Octavia Squidington III
607a7d68ff Bump Airbyte version from 0.40.6 to 0.40.7 (#16785)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-09-16 11:36:44 -07:00
Benoit Moriceau
930cb309b5 remove eager (#16756)
* remove eager

* Rm dep worker on cron
2022-09-14 15:55:20 -07:00
Octavia Squidington III
83799372cf Bump Airbyte version from 0.40.5 to 0.40.6 (#16656)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-09-13 11:56:33 -07:00
Octavia Squidington III
d8285cb24f Bump Airbyte version from 0.40.4 to 0.40.5 (#16610)
Co-authored-by: alovew <alovew@users.noreply.github.com>
2022-09-12 14:29:38 -07:00
Anne
3039ebf9bf update version (#16605) 2022-09-12 13:09:05 -07:00