jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-21 02:51:29 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,867 Commits 5,109 Branches 518 Tags
master
Commit Graph

375 Commits

Author SHA1 Message Date
Xiaohan Song
acb9ce1f86 [source-postgres] Fix duplicate streams in postgres (#40719) 2024-07-05 12:05:52 -07:00
Akash Kulkarni
3a3e058ae8 [DB-sources] : Improve heartbeat logic (#40516) 2024-07-01 12:51:33 -07:00
Evan Tahler
76878000d6 [docs] source-postgres - add link explaining modest increase cost of REPLICA IDENTITY FULL (#40611) 
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
 2024-06-28 21:58:01 +00:00
Evan Tahler
0cfaf79ab2 source-postgres: Remove strict check for stream states of unknown types (#40559) 2024-06-26 16:14:29 -07:00
Marius Posta
d5e447bf49 source-postgres: remove adaptiveFetch=true (#40223) 2024-06-23 12:46:07 -07:00
Xiaohan Song
b80a72888c [source-postgres/mssql/mysql] Send state and count for full refresh (#39349) 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: Anjay Goel <anjay.g@dashtoon.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Baz <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Natalie Kwong <38087517+nataliekwong@users.noreply.github.com>
Co-authored-by: Audrey Maldonado <audrey.maldonado@gmail.com>
Co-authored-by: gosusnp <gosusnp@users.noreply.github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Abdul Rahman Zantout <abed-zantout@live.com>
Co-authored-by: Anatolii Yatsuk <35109939+tolik0@users.noreply.github.com>
Co-authored-by: Dhroov Makwana <pabloescoder@gmail.com>
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Ben Church <ben@airbyte.io>
Co-authored-by: Justin Flannery <juftin@juftin.com>
Co-authored-by: Mal Hancock <mallory@archangelic.space>
Co-authored-by: Enrique Alcázar Garzás <enrique.alcazar@omniloy.com>
Co-authored-by: Daniela García Nistor <58893405+gndaniela@users.noreply.github.com>
Co-authored-by: Carlo Nuccio <carlonuccio91@gmail.com>
Co-authored-by: Natik Gadzhi <natik@respawn.io>
Co-authored-by: Charlie Duong <charlie.duong@hey.com>
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
Co-authored-by: Cristina Mariscal <166420606+cmm-airbyte@users.noreply.github.com>
Co-authored-by: cristina.mariscal <cristina.mariscal@cristina.mariscal--MacBook-Pro---DFJ27FJFXX>
Co-authored-by: Gonzalo Villafañe Tapia <gvillafanetapia@gmail.com>
Co-authored-by: Jérémy Denquin <jdenquin@users.noreply.github.com>
Co-authored-by: Maxime Carbonneau-Leclerc <3360483+maxi297@users.noreply.github.com>
Co-authored-by: williammcguinness <wmcguinness@findoctave.com>
Co-authored-by: Marius Posta <marius@airbyte.io>
Co-authored-by: Stephane Geneix <147216312+stephane-airbyte@users.noreply.github.com>
Co-authored-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Akash Kulkarni <113392464+akashkulk@users.noreply.github.com>
Co-authored-by: Danylo Jablonski <150933663+DanyloGL@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
 2024-06-17 13:30:56 -04:00
Evan Tahler
47ae1b8e60 Source Postgres - fix PR ID for v.3.3.27 (#39413) 2024-06-14 12:42:16 -07:00
Evan Tahler
b88a41d339 postgres source/destination - bump postgres driver version (#39460) 
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
 2024-06-14 15:35:17 -04:00
Akash Kulkarni
6db30d6a51 [Source-postgres] : Bump cdk to latest (#39474) 2024-06-14 09:37:58 -07:00
Marius Posta
32f78307c8 source-postgres: connect with adaptiveFetch=true, take 2 (#38773) 2024-06-13 11:31:09 -04:00
Yue Li
3b3392317f source-postgres: upgrade Debezium to 2.6.2 (#39353) 
fixes airbytehq/airbyte-internal-issues#7739
 2024-06-08 20:45:29 -07:00
Yue Li
859b596383 source-postgres: read() throws config exception upon xmin wraparound (#38875) 
fixes airbytehq/airbyte-internal-issues#8006
 2024-06-05 12:17:14 -07:00
Yue Li
2791dd6565 source-postgres: check() throws config error upon detecting transaction ID wraparound (#38836) 
fixes airbytehq/airbyte-internal-issues#8008

This PR adds one more checkOperation for check(), which throws a config exception if Postgres has experienced one or more xmin wraparounds.
 2024-06-04 17:46:08 -07:00
Yue Li
b2846c03ec source-postgres: improve UI message and doc on xmin (#38848) 
The current recommendation for using xmin on the UI of the Postgres source connector is misleading; we want to say that the database is not experiencing a high transaction rate such that it is less likely to trigger xmin wraparound soon.

Added sections on the doc and troubleshooting page regarding Xmin as well.
 2024-06-04 16:06:23 -07:00
Audrey Maldonado
b9690ba76e Wrapping changelog in details for a collapsible view (#38801) 2024-06-04 13:40:53 -04:00
Xiaohan Song
2d194fa359 Set isResumeable flag in catalog (#38584) 
Co-authored-by: Dhroov Makwana <pabloescoder@gmail.com>
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Natik Gadzhi <natik@respawn.io>
Co-authored-by: Danylo Jablonski <150933663+DanyloGL@users.noreply.github.com>
Co-authored-by: Maxime Carbonneau-Leclerc <3360483+maxi297@users.noreply.github.com>
Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Ben Church <ben@airbyte.io>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Christo Grabowski <108154848+ChristoGrab@users.noreply.github.com>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
Co-authored-by: Catherine Noll <clnoll@users.noreply.github.com>
Co-authored-by: Audrey Maldonado <audrey.maldonado@gmail.com>
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
 2024-05-30 10:23:09 -07:00
Akash Kulkarni
2bf0bf8d07 Bump versions for SQL server, Postgres (#38775) 2024-05-29 18:33:50 -04:00
Xiaohan Song
c63d40998a Publish cdk for postgres (#38771) 2024-05-29 13:47:23 -07:00
Xiaohan Song
b104667a0c Stream status on postgres (#38716) 
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
 2024-05-29 13:05:20 -07:00
Evan Tahler
a914027d74 [docs] Vendor-Specific limitations added to troubleshooting sections (#38351) 
Co-authored-by: Stephane Geneix <147216312+stephane-airbyte@users.noreply.github.com>
 2024-05-21 14:42:22 +00:00
Xiaohan Song
0db33b8494 Revert "source-postgres: connect with adaptiveFetch=true" (#38365) 2024-05-20 19:23:09 -04:00
Marius Posta
c97bbfc4ed source-postgres: connect with adaptiveFetch=true (#38289) 
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
 2024-05-20 10:49:19 -07:00
Evan Tahler
3a971c3791 Mention transaction id wrap around in xmin warning (#38235) 2024-05-17 12:44:46 -05:00
Evan Tahler
49fc60dea2 source-postgres - Streams not in the CDC publication still have a cursor and PK (#38303) 2024-05-16 16:18:26 -07:00
Subodh Kant Chaturvedi
073b94062c db-sources: disable counts for state messages for FULL_REFRESH streams (#38208) 
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
 2024-05-15 11:15:26 -07:00
Akash Kulkarni
fcbfaa376f [DB source errors] : Handle common transient errors (#38104) 2024-05-13 20:27:07 -04:00
Xiaohan Song
59cdc36db6 [source-postgres] Fix final state message on state (#38171) 2024-05-13 19:17:03 -04:00
Xiaohan Song
4b92f751cd [source-postgres] bugfix: Exclude views in incremental iterator (#38130) 2024-05-10 18:55:03 -04:00
Xiaohan Song
80920d1394 Postgres on Resumable full refresh (#37112) 2024-05-10 12:13:37 -04:00
Tyler B
9d95dc0cfe Update postgres.md (#38096) 2024-05-09 12:48:05 -07:00
Akash Kulkarni
d26bd10c87 [DB sources] : Add plumbing for adding transient errors (#38030) 2024-05-09 10:21:58 -07:00
Alex
c4a2a26bd7 Postgres Docs: Fix Changelog Formatting (#38047) 2024-05-07 15:24:23 -05:00
Natik Gadzhi
cb80e6922a [tools] prettier rules for .md + formatting cleanup 2024-05-07 08:19:33 -07:00
Rodi Reich Zilberman
f7f7f7ef2e Set debezium num retries property to zero (#37758) 2024-05-02 03:04:03 -04:00
Rodi Reich Zilberman
deabdbb16a Add debezium engine logs (#37754) 2024-05-01 20:25:57 -04:00
Akash Kulkarni
201e7e3630 [Source-postgres] : Remove debezium retries (#37726) 2024-05-01 08:26:29 -07:00
Xiaohan Song
587e874eb5 Remove excessive logs in counter (#37541) 2024-04-24 13:31:46 -04:00
Akash Kulkarni
d790b7d5df [Source-postgres] : Better error messages on switching between sync m… (#37509) 2024-04-24 12:39:07 -04:00
Akash Kulkarni
9b9ec1cecd [source-postgres] : Remove legacy bad values handling code (#37445) 2024-04-22 13:47:57 -04:00
Akash Kulkarni
fda082966f [Source-postgres] : Populate airyte_meta.changes for xmin path (#36982) 2024-04-10 16:25:23 -07:00
Akash Kulkarni
0296c43169 [Db analytics] : add message for data type serialization error (#36981) 2024-04-10 15:46:15 -07:00
Akash Kulkarni
1815e38dd8 [Source-Postgres] : Bump cdk, reduce timeout (#36865) 2024-04-10 15:36:17 -04:00
Marius Posta
2495575795 java-cdk: re-export airbyte-api dependency (#36759) 2024-04-03 10:43:05 -07:00
Akash Kulkarni
2825239050 [Source-Postgres] : Toggle useLocalCdk = false (#36739) 2024-04-01 19:35:38 -04:00
Akash Kulkarni
e3f3379120 [Source-postgres] : Move to new Kotlin CDK (#36584) 2024-04-01 16:51:04 -04:00
Akash Kulkarni
9dee83729e [Source-postgres] : Source operations suport for meta column (#36432) 2024-03-26 10:27:20 -07:00
Xiaohan Song
1b99f1ab89 DBZ iterator migration to use SourceStateIterator (#36333) 2024-03-25 21:50:33 -07:00
Xiaohan Song
3eb18c9737 Move cursor state decorating iterator to sourceStateIterator (#35599) 2024-03-21 13:47:02 -07:00
Akash Kulkarni
609d602cec [Source-postgres] : Compare each record’s lsn_commit value instead of lsn_proc. (#35939) 2024-03-13 12:49:09 -07:00
Akash Kulkarni
ee6b1a0ee6 [source-postgres] : Add retries back to the DBZ connector (#35904) 2024-03-11 15:31:43 -07:00