1
0
mirror of synced 2026-01-07 09:05:45 -05:00
Commit Graph

17 Commits

Author SHA1 Message Date
Anne
fdb96d0a63 Notifications Workflow (#18735)
* notification workflow
2022-11-15 13:58:30 -08:00
Benoit Moriceau
b3db914265 Change where a connection is deleted (#19096)
* Tmp

* Move when the deletion is performed

* Re-enable disable test

* PR comments

* Use cancel

* rename

* Fix test and version check position

* Log exception
2022-11-09 13:05:27 -08:00
Benoit Moriceau
85e6792e36 More comprehensive temporal error message (#18608)
* To be remove

* Remove the signal for waiting after a failed activity and ensure we are waiting the expected time

* Revert "To be remove"

This reverts commit 3a5f7b4f72.

* Remove unused and move failure reason to the helper

* Avoid repetitive new Set()
2022-10-28 15:18:23 -07:00
Jimmy Ma
e32a641cbe Use Protocol Versions in Syncs (#18324)
* Pass protocol version into IntegrationLauncherConfig

* Use VersionedStreamStreamFactory in AirbyteSource/Destination

* Add AirbyteMessageBufferedWriter

* Use VersionedBufferedWriter
2022-10-27 12:57:33 -07:00
Jonathan Pearlin
657f997aae Update to latest Temporal SDK release (#18492) 2022-10-26 12:54:00 -04:00
Jimmy Ma
cf28f7a01f Add message migration to discover schema (#18205) 2022-10-21 09:32:42 -07:00
Parker Mossman
4e236b5b42 Add Geography support to RouterService (#17902)
* router service uses geography in database instead of env var

move geography map to a helper that can be overridden with a separate implementation in cloud

format

pmd

fix import

move geography mapper interface to airbyte-commons-temporal

* add DefaultGeographyMapper back in

* remove all args constructor and extranneous import

* rename GeographyMapper to TaskQueueMapper
2022-10-20 14:48:27 -07:00
Benoit Moriceau
7a71c55c1f Bmoric/remove dep server worker (#17894)
* test [ci skip]

* Autogenerated files

* Add missing annotation

* Remove unused json2Schema block from worker

* Move tess

* Missing deps and format

* Fix test build

* TMP

* Add missing dependencies

* PR comments

* Tmp

* [ci skip] Tmp

* Fix acceptance test and add the seed dependency

* Fix build

* For diff

* tmp

* Build pass

* make the worker to be  on the platform only

* fix setting.yaml

* Fix pmd

* Fix Cron

* Add chart

* Fix cron

* Fix server build.gradle

* Fix jar conflict

* PR comments

* Add cron micronaut environemnt
2022-10-17 12:35:04 -07:00
Benoit Moriceau
f650005f2a Remove the dependency between the orchestrator container and the worker app (#17570)
* test [ci skip]

* Autogenerated files

* Add missing annotation

* Remove unused json2Schema block from worker

* Move tess

* Missing deps and format

* Fix test build

* Add missing dependencies

* PR comments

* Fix acceptance test and add the seed dependency

* Fix build

* Fix build again
2022-10-11 17:12:00 -07:00
Benoit Moriceau
e038ea846f Revert "Revert "Do not wait the end of a reset to return an update (#17591)" (#17640)" (#17669)
This reverts commit 77084beecd.
2022-10-07 08:02:22 -07:00
Benoit Moriceau
77084beecd Revert "Do not wait the end of a reset to return an update (#17591)" (#17640)
This reverts commit b4fd1ea3bd.
2022-10-05 16:08:43 -07:00
Benoit Moriceau
b4fd1ea3bd Do not wait the end of a reset to return an update (#17591)
* Directly save after update

* Fix existing tests

* Rm unused

* Format

* Add test t

* Format
2022-10-05 13:22:48 -07:00
Jonathan Pearlin
759a27ec45 Use Micronaut auto-detected environment for docker vs kubernetes choices (#17388)
* Use Micronaut auto-detected environment for docker vs kubernetes choices

* Handle set, but blank env var

* Formatting

* Revert change

* Add default value

* Explicitly set WORKER_ENVIRONMENT for container orchestrator

* Make public

* Add sleep to allow Temporal cache to populate

Co-authored-by: Jimmy Ma <gosusnp@users.noreply.github.com>
2022-10-04 15:14:13 -04:00
Benoit Moriceau
65a35b98bd Bmoric/clean applciation yml (#17353)
* Cleanup application.yml

* Clean up cron application.yml and kube overlay

* Avoid flooding the logs with waiting for temporal namespace message

* Rm common app initializer

* Try to re-introduce an initializer
2022-09-29 16:21:38 -07:00
Benoit Moriceau
3c457e051d Bmoric/restore temporal cleaning cron (#17258)
* Revert "Revert "Bmoric/temporal cleaning cron (#16414)" (#17192)"

This reverts commit 751667949e.

* Generate an artifact

* Change missing property from value to property
2022-09-27 10:59:39 -07:00
Benoit Moriceau
751667949e Revert "Bmoric/temporal cleaning cron (#16414)" (#17192)
This reverts commit bbb59d8675.
2022-09-26 16:58:23 -07:00
Benoit Moriceau
bbb59d8675 Bmoric/temporal cleaning cron (#16414)
* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Add the injected temporal client to airbyte-cron

* Update cron to make it work

* Remove useless Bean factory

* PR comments

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Improve conditional bean check

* Match existing Optional functionality

* Add notEquals check

* Add missing env var to Helm chart

* Fix typo

* Mark LogConfigs as Singleton

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Improve conditional bean check

* Match existing Optional functionality

* Add notEquals check

* Add missing env var to Helm chart

* Fix typo

* Mark LogConfigs as Singleton

* Env vars for log/state storage type

* Remove use of Optional in bean declarations

* Fix typo in config property name

* Support Temporal Cloud namespace

* Change to @Value

* Use correct value for conditional check

* Upgrade Micronaut

* Fix merge conflict

* Formatting

* Add missing env var

* Use sync task queue environment variable

* Handle sync task queue as set

* format and force http

* Handle case where sync task queue is empty

* Add correct path to config property

* Remove unused import

* Remove conflict

* Remove unused parameter

* Formatting

* Use pattern for condition process factory beans

* Cleanup

* PR feedback

* Revert hack for testing

* Fix temporal restart by status (#16447)

* Update application.yml

* Re add worker dep

* Add missing env var

* PR comments

* Bmoric/move temporal client (#16778)

* tmp

* tmp

* View diff

* Move part of the temporal client

* tmp

* copy Temporal Utils test

* Uniq Temporal Utils

* Uniq Temporal Workflow Utils

* Move CancellationHandler

* Move commons-temporal to being shared between platform, connector and CLI

* Rm worker dependency from cron

* Fix build

* Update with right value for cron.

* Fix dep conflict

Co-authored-by: jdpgrailsdev <jpearlin1@gmail.com>
Co-authored-by: pmossman <parker@airbyte.io>
Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io>
2022-09-26 15:14:35 -07:00