1
0
mirror of synced 2026-01-07 09:05:45 -05:00
Commit Graph

37 Commits

Author SHA1 Message Date
Octavia Squidington III
b299688f3b Bump Airbyte version from 0.40.17 to 0.40.18 (#18827)
Co-authored-by: grishick <grishick@users.noreply.github.com>
2022-11-01 19:35:23 -07:00
Charles
87a0a2a27c hide ConfigPersistence inside ConfigRepository to discourage use (#18803) 2022-11-01 16:51:09 -07:00
Charles
20ab047ae0 Remove the bulk actions from ConfigPersistence (#18800) 2022-11-01 16:50:03 -07:00
Jonathan Pearlin
657f997aae Update to latest Temporal SDK release (#18492) 2022-10-26 12:54:00 -04:00
Octavia Squidington III
73336ff4de Bump Airbyte version from 0.40.16 to 0.40.17 (#18316)
Co-authored-by: edgao <edgao@users.noreply.github.com>
2022-10-21 14:00:39 -07:00
Octavia Squidington III
ce0bbf3f75 Bump Airbyte version from 0.40.15 to 0.40.16 (#18268)
Co-authored-by: evantahler <evantahler@users.noreply.github.com>
2022-10-20 13:29:54 -07:00
Jonathan Pearlin
eed3257c9b Use constructor injection for singleton (#18163) 2022-10-19 11:59:09 -04:00
Jonathan Pearlin
b4c16bfdbb Custom APM Tracing (#17947)
* Add APM tracing to airbyte-workers

* Add custom traces to airbyte-cron

* Formatting

* Refactor tag constants

* Handle potential null object

* Handle null tag values
2022-10-18 13:14:29 -04:00
Octavia Squidington III
a5a0409ad5 Bump Airbyte version from 0.40.14 to 0.40.15 (#17970)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-10-17 13:33:20 -07:00
Benoit Moriceau
7a71c55c1f Bmoric/remove dep server worker (#17894)
* test [ci skip]

* Autogenerated files

* Add missing annotation

* Remove unused json2Schema block from worker

* Move tess

* Missing deps and format

* Fix test build

* TMP

* Add missing dependencies

* PR comments

* Tmp

* [ci skip] Tmp

* Fix acceptance test and add the seed dependency

* Fix build

* For diff

* tmp

* Build pass

* make the worker to be  on the platform only

* fix setting.yaml

* Fix pmd

* Fix Cron

* Add chart

* Fix cron

* Fix server build.gradle

* Fix jar conflict

* PR comments

* Add cron micronaut environemnt
2022-10-17 12:35:04 -07:00
Jonathan Pearlin
246172bd6c Use the Airbyte Base Java Image for all services (#17843)
* Use the Amazon Corretto based base image for all services

* Add comment

* Use airbyte-base-java-image
2022-10-12 08:59:02 -04:00
Jonathan Pearlin
e95a983563 Various Micronaut-related cleanup items (#17611) 2022-10-11 10:14:23 -04:00
Octavia Squidington III
9cb32437e4 Bump Airbyte version from 0.40.13 to 0.40.14 (#17762)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-10-07 14:18:21 -07:00
Benoit Moriceau
922527b4cf Revert "Use java-datadog-tracer-base image (#17625)" (#17759)
This reverts commit c170173334.
2022-10-07 13:38:45 -07:00
Octavia Squidington III
e382a6e6db Bump Airbyte version from 0.40.12 to 0.40.13 (#17682)
Co-authored-by: xiaohansong <xiaohansong@users.noreply.github.com>
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
2022-10-06 15:12:18 -07:00
Jonathan Pearlin
c170173334 Use java-datadog-tracer-base image (#17625)
* Use java-datadog-tracer-base image

* Restore build args

* Explicity install tar executable
2022-10-06 14:35:37 -04:00
Octavia Squidington III
71c8e3c062 Bump Airbyte version from 0.40.11 to 0.40.12 (#17653)
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
2022-10-05 18:25:09 -07:00
Pedro S. Lopez
69f53eb56a airbyte-cron: update connector definitions from remote (#16438)
* add db connection injection to airbyte-cron

* load definitions

* fix patch version check for non-semver tags

* apply helper tests

* logging updates

* fix remote definitions provider: add tombstones

* docker compose env updates

* add test for tombstone presence

* rename helper class

* config updates

* dont use optionals as fields

* use apply helper instead of directly calling load data in bootloader

* avoid pmd warn

* add docstring

* sort docker compose env vars

* updates for javax -> jakarta

* clean up docker compose, update kube deployment env
2022-10-05 17:21:37 -04:00
Octavia Squidington III
e333aa9365 Bump Airbyte version from 0.40.10 to 0.40.11 (#17610)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-10-05 08:35:39 -07:00
Benoit Moriceau
65a35b98bd Bmoric/clean applciation yml (#17353)
* Cleanup application.yml

* Clean up cron application.yml and kube overlay

* Avoid flooding the logs with waiting for temporal namespace message

* Rm common app initializer

* Try to re-introduce an initializer
2022-09-29 16:21:38 -07:00
Pedro S. Lopez
12b46694dc 🐛 cron/workspace-cleaner: only run task for docker environments (#17168)
* dont run cleanup on kubernetes

* use the micronaut way™

* add env var to kube resource

* use micronaut environment instead

* no need for the env var
2022-09-29 15:53:16 -04:00
Octavia Squidington III
1c40fe6644 Bump Airbyte version from 0.40.9 to 0.40.10 (#17359)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-09-28 17:09:25 -07:00
Benoit Moriceau
3c457e051d Bmoric/restore temporal cleaning cron (#17258)
* Revert "Revert "Bmoric/temporal cleaning cron (#16414)" (#17192)"

This reverts commit 751667949e.

* Generate an artifact

* Change missing property from value to property
2022-09-27 10:59:39 -07:00
Jonathan Pearlin
896c2c9751 Switch to jakarta annotations (#17251) 2022-09-27 11:18:22 -04:00
Benoit Moriceau
751667949e Revert "Bmoric/temporal cleaning cron (#16414)" (#17192)
This reverts commit bbb59d8675.
2022-09-26 16:58:23 -07:00
Benoit Moriceau
bbb59d8675 Bmoric/temporal cleaning cron (#16414)
* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Add the injected temporal client to airbyte-cron

* Update cron to make it work

* Remove useless Bean factory

* PR comments

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Improve conditional bean check

* Match existing Optional functionality

* Add notEquals check

* Add missing env var to Helm chart

* Fix typo

* Mark LogConfigs as Singleton

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Improve conditional bean check

* Match existing Optional functionality

* Add notEquals check

* Add missing env var to Helm chart

* Fix typo

* Mark LogConfigs as Singleton

* Env vars for log/state storage type

* Remove use of Optional in bean declarations

* Fix typo in config property name

* Support Temporal Cloud namespace

* Change to @Value

* Use correct value for conditional check

* Upgrade Micronaut

* Fix merge conflict

* Formatting

* Add missing env var

* Use sync task queue environment variable

* Handle sync task queue as set

* format and force http

* Handle case where sync task queue is empty

* Add correct path to config property

* Remove unused import

* Remove conflict

* Remove unused parameter

* Formatting

* Use pattern for condition process factory beans

* Cleanup

* PR feedback

* Revert hack for testing

* Fix temporal restart by status (#16447)

* Update application.yml

* Re add worker dep

* Add missing env var

* PR comments

* Bmoric/move temporal client (#16778)

* tmp

* tmp

* View diff

* Move part of the temporal client

* tmp

* copy Temporal Utils test

* Uniq Temporal Utils

* Uniq Temporal Workflow Utils

* Move CancellationHandler

* Move commons-temporal to being shared between platform, connector and CLI

* Rm worker dependency from cron

* Fix build

* Update with right value for cron.

* Fix dep conflict

Co-authored-by: jdpgrailsdev <jpearlin1@gmail.com>
Co-authored-by: pmossman <parker@airbyte.io>
Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io>
2022-09-26 15:14:35 -07:00
Octavia Squidington III
8dfb8b0c92 Bump Airbyte version from 0.40.8 to 0.40.9 (#17093)
Co-authored-by: terencecho <terencecho@users.noreply.github.com>
2022-09-23 09:46:09 -05:00
Octavia Squidington III
387b96a924 Bump Airbyte version from 0.40.7 to 0.40.8 (#17009)
Co-authored-by: gosusnp <gosusnp@users.noreply.github.com>
2022-09-21 15:57:29 -07:00
Volodymyr Pochtar
a72cb7d1f8 feat: replace openjdk with amazoncorretto:17.0.4 on platform for security compliance (#16919) 2022-09-20 19:01:48 +03:00
Octavia Squidington III
607a7d68ff Bump Airbyte version from 0.40.6 to 0.40.7 (#16785)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-09-16 11:36:44 -07:00
Benoit Moriceau
930cb309b5 remove eager (#16756)
* remove eager

* Rm dep worker on cron
2022-09-14 15:55:20 -07:00
Octavia Squidington III
83799372cf Bump Airbyte version from 0.40.5 to 0.40.6 (#16656)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2022-09-13 11:56:33 -07:00
Octavia Squidington III
d8285cb24f Bump Airbyte version from 0.40.4 to 0.40.5 (#16610)
Co-authored-by: alovew <alovew@users.noreply.github.com>
2022-09-12 14:29:38 -07:00
Anne
3039ebf9bf update version (#16605) 2022-09-12 13:09:05 -07:00
Evan Tahler
834ac1a3bb Remove shared getDockerBuildTask to enable parallel docker builds (#16384)
Define logic in root build.gradle to create a task in each subproject if the subproject contains dockerImageName in the gradle.properties file.

Some caveats:
- We also define a copyGeneratedTar task that is applied to all subprojects with the dockerImageName property.
- This does not need to be used in each subproject. We leave it up to each subproject to define what task is depends on since some projects have more custom copy logic.
2022-09-09 18:40:35 -07:00
Jimmy Ma
bf487913d2 Add scheduled task to clean up old files from workspace (#16247)
* Add airbyte-cron to bumpversion

* Update airbyte-cron version to current

* Add workspace clean up job

* Add missing env var to docker-compose

* Update file deletion logging
2022-09-02 13:21:17 -07:00
Benoit Moriceau
d800e3dadc Add scheduled service (#15761)
* Add logger scheduled service

* Use eager bean initialization
2022-08-19 23:29:47 +02:00