🎉 Source MSSQL: implementation for CDC (#4689)
* first few classes for mssql cdc
* wip
* mssql cdc working against unit tests
* increment version
* add cdc acceptance test
* tweaks
* add file
* working on comprehensive tests
* change isolation from snapshot to read_committed_snapshot
* finalised type tests
* Revert "change isolation from snapshot to read_committed_snapshot"
This reverts commit 20c6768071.
* small docstring fix
* remove unused imports
* stress test fixes
* minor formatting improvements
* mssql cdc docs
* finish off cdc docs
* format fix
* update connector version
* add to changelog
* fix for sql server agent offline failing cdc enable on tables
* final structure
* few more updates
* undo unwanted changes
* add abstract test + more refinement
* remove CDC metadata to debezium
* use new cdc abstraction for mysql
* undo wanted change
* use cdc abstraction for postgres
* add files
* pull in latest changes
* ready
* rename class + add missing property
* use renamed class + move constants to MySqlSource
* use renamed class + move constants to PostgresSource
* move debezium to bases + upgrade debezium version + review comments
* downgrade version + minor fixes
* bring in latest changes from cdc abstraction
* reset to minutes
* bring in the latest changes
* format
* fix build
* address review comments
* bring in latest changes
* bring in latest changes
* use common abstraction for CDC via debezium for sql server
* remove debezium from build
* finalise PR
* should return Optional
* pull in latest changes
* pull in latest changes
* address review comments
* use common abstraction for CDC via debezium for mysql (#4604)
* use new cdc abstraction for mysql
* undo wanted change
* pull in latest changes
* use renamed class + move constants to MySqlSource
* bring in latest changes from cdc abstraction
* format
* bring in latest changes
* pull in latest changes
* use common abstraction for CDC via debezium for postgres (#4607)
* use cdc abstraction for postgres
* add files
* ready
* use renamed class + move constants to PostgresSource
* bring in the latest changes
* bring in latest changes
* pull in latest changes
* lower version for tests to run on CI
* format
* Update docs/integrations/sources/mssql.md
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* addressing review comments
* fix for testGetTargetPosition
* format changes
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
This commit is contained in:
committed by
GitHub
parent
b18cedd18f
commit
cc7b376044
@@ -14,7 +14,7 @@ The Airbyte Protocol outputs records from sources. Records from `UPDATE` stateme
|
||||
|
||||
We add some metadata columns for CDC sources:
|
||||
|
||||
* `ab_cdc_lsn` (specific to postgres source) is the point in the log where the record was retrieved
|
||||
* `ab_cdc_lsn` (postgres and sql server sources) is the point in the log where the record was retrieved
|
||||
* `ab_cdc_log_file` & `ab_cdc_log_pos` (specific to mysql source) is the file name and position in the file where the record was retrieved
|
||||
* `ab_cdc_updated_at` is the timestamp for the database transaction that resulted in this record change and is present for records from `DELETE`/`INSERT`/`UPDATE` statements
|
||||
* `ab_cdc_deleted_at` is the timestamp for the database transaction that resulted in this record change and is only present for records from `DELETE` statements
|
||||
@@ -32,10 +32,10 @@ We add some metadata columns for CDC sources:
|
||||
|
||||
* [Postgres](../integrations/sources/postgres.md) (For a quick video overview of CDC on Postgres, click [here](https://www.youtube.com/watch?v=NMODvLgZvuE&ab_channel=Airbyte))
|
||||
* [MySQL](../integrations/sources/mysql.md)
|
||||
* [Microsoft SQL Server / MSSQL](../integrations/sources/mssql.md)
|
||||
|
||||
## Coming Soon
|
||||
|
||||
* [SQL Server / MSSQL](../integrations/sources/mssql.md)
|
||||
* Oracle DB
|
||||
* Please [create a ticket](https://github.com/airbytehq/airbyte/issues/new/choose) if you need CDC support on another database!
|
||||
|
||||
|
||||
Reference in New Issue
Block a user