1
0
mirror of synced 2026-01-30 16:01:55 -05:00
Commit Graph

27 Commits

Author SHA1 Message Date
Maxime Carbonneau-Leclerc
48af92ad78 Concurrent CDK: if exception is AirbyteTracedException, raise this an… (#37443) 2024-04-19 20:32:46 +00:00
Maxime Carbonneau-Leclerc
b49c5006ea Break Python application with status 1 on exception (#37390) 2024-04-18 16:54:55 -04:00
Maxime Carbonneau-Leclerc
d2c8e63ea5 Fix partitioned state saving issue (#37389) 2024-04-17 22:57:45 -04:00
Maxime Carbonneau-Leclerc
a900c7841f Add StreamDescriptor as params of AirbyteTracedException.__init__ (#37108) 2024-04-12 17:11:12 -04:00
Ella Rohm-Ensing
b7819d9f6c python: assert actual == expected ordering (#36980) 2024-04-11 15:16:33 +00:00
Maxime Carbonneau-Leclerc
bbf69aea7f Concurrent CDK: support partitioned states (#36811) 2024-04-09 16:48:12 -04:00
Ella Rohm-Ensing
2ac5248387 Emit record counts in state messages for concurrent streams (#35907)
Co-authored-by: brianjlai <brian.lai@airbyte.io>
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
2024-03-08 19:08:59 -05:00
Ella Rohm-Ensing
acbdc2d6e1 Introduce FinalStateCursor to emit state messages at the end of full refresh syncs (#35905)
Co-authored-by: brianjlai <brian.lai@airbyte.io>
2024-03-08 16:58:26 -05:00
Brian Lai
ef98194673 Emit final state message for full refresh syncs and consolidate read flows (#35622) 2024-03-05 01:05:06 -05:00
Artem Inzhyyants
0954ad3d3a Airbyte CDK: add interpolation for request options (#35485)
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
2024-02-22 19:40:44 +01:00
Catherine Noll
e8910e427a File-based CDK: make incremental syncs concurrent (#34540) 2024-02-07 20:41:04 -05:00
Maxime Carbonneau-Leclerc
ca8590e2b4 Have StateBuilder return our actual state object and not simply a dict (#34625) 2024-01-30 08:46:03 -05:00
Maxime Carbonneau-Leclerc
2c8b47b100 Emit state when no partitions are generated for ccdk (#34605) 2024-01-30 08:45:49 -05:00
Catherine Noll
eb31e4d2ba File-based CDK: make full refresh concurrent (#34411) 2024-01-29 19:33:50 -05:00
Maxime Carbonneau-Leclerc
b9c1897cfc Fix concurrent deadlock (#34454) 2024-01-24 12:35:21 -05:00
Catherine Noll
e3e58cc063 Concurrent CDK: fix state message ordering (#34131) 2024-01-18 11:35:40 -05:00
Alexandre Girard
0faa69d899 concurrent cdk: improve resource usage and stop waiting on the main thread (#33669)
Co-authored-by: Augustin <augustin@airbyte.io>
2024-01-17 23:54:02 -08:00
Catherine Noll
86321b7945 Concurrent CDK: add state converter for ISO timestamps with milliseco… (#33531) 2023-12-20 11:53:59 -05:00
Maxime Carbonneau-Leclerc
ba83309bb1 [ISSUE #32870] Adding entrypoint wrapper and migrating file based and… (#33103) 2023-12-06 08:46:38 -05:00
Alexandre Girard
a84902e8be concurrent cdk: Read multiple streams concurrently (#32411) 2023-11-28 15:00:00 -08:00
Maxime Carbonneau-Leclerc
b3f0e059d5 Fixing futures pruning (#32399) 2023-11-09 17:52:58 -05:00
Maxime Carbonneau-Leclerc
71d50635cc [ISSUE #32070] concurrent cdk improve futures handling (#32277) 2023-11-08 09:16:39 -05:00
Catherine Noll
4f44e33f5c Concurrent CDK: handle legacy state messages (#31964) 2023-11-02 08:21:08 -04:00
Maxime Carbonneau-Leclerc
32fdd7fd72 [ISSUE #29573] Concurrent CDK: incremental syncs (#31466)
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: girarda <girarda@users.noreply.github.com>
Co-authored-by: Catherine Noll <clnoll@users.noreply.github.com>
2023-11-01 12:00:25 -04:00
Alexandre Girard
7da2822488 Concurrent CDK: catch exceptions from worker thread and add integration test scenarios (#31245)
Co-authored-by: girarda <girarda@users.noreply.github.com>
2023-10-23 08:39:58 -07:00
Alexandre Girard
04c4fea5cc 🐛 Concurrent CDK bug fixes (#31402) 2023-10-16 12:06:35 -07:00
Alexandre Girard
25fc396cdf CDK: ThreadBasedConcurrentStream skeleton and top-level AbstractStream (#30111)
Co-authored-by: girarda <girarda@users.noreply.github.com>
Co-authored-by: Maxime Carbonneau-Leclerc <maxi297@users.noreply.github.com>
Co-authored-by: Catherine Noll <clnoll@users.noreply.github.com>
2023-10-11 16:46:02 -07:00