* mysql-cdc:implementation to validate that saved offset still exists on server
* Automated Commit - Format and Process Resources Changes
* add the check to the main code flow + test
* address review comments
---------
Co-authored-by: subodh1810 <subodh1810@users.noreply.github.com>
* Proof of concept parallel source stream reading implementation for MySQL
* Automated Change
* Add read method that supports concurrent execution to Source interface
* Remove parallel iterator
* Ensure that executor service is stopped
* Automated Commit - Format and Process Resources Changes
* Expose method to fix compilation issue
* Use concurrent map to avoid access issues
* Automated Commit - Format and Process Resources Changes
* Ensure concurrent streams finish before closing source
* Fix compile issue
* Formatting
* Exclude concurrent stream threads from orphan thread watcher
* Automated Commit - Format and Process Resources Changes
* Refactor orphaned thread logic to account for concurrent execution
* PR feedback
* Implement readStreams in wrapper source
* Automated Commit - Format and Process Resources Changes
* Add readStream override
* Automated Commit - Format and Process Resources Changes
* 🤖 Auto format source-mysql code [skip ci]
* 🤖 Auto format source-mysql code [skip ci]
* 🤖 Auto format source-mysql code [skip ci]
* 🤖 Auto format source-mysql code [skip ci]
* 🤖 Auto format source-mysql code [skip ci]
* Debug logging
* Reduce logging level
* Replace synchronized calls to System.out.println when concurrent
* Close consumer
* Flush before close
* Automated Commit - Format and Process Resources Changes
* Remove charset
* Use ASCII and flush periodically for parallel streams
* Test performance harness patch
* Automated Commit - Format and Process Resources Changes
* Cleanup
* Logging to identify concurrent read enabled
* Mark parameter as final
---------
Co-authored-by: jdpgrailsdev <jdpgrailsdev@users.noreply.github.com>
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
Co-authored-by: rodireich <rodireich@users.noreply.github.com>
* DNC
* Add test models
* Add model test
* Remove underscore from metadata files
* Regenerate models
* Add test to check for key transformation
* Allow additional fields on metadata
* Delete transform
* Add airbyte internal
* Add tests
* First pass
* Set destinations to same levels as sources
* Best guess at missing statuses
* Best guess at _ql
* Add separate enum class
* Fix support level name
* Update templates
* Add one more test
* use LSN as default cursor for postgres CDC
* Fixed static constant
* Set lsn default cursor value for postgres sync
* Bumped metadata and dockerfile versions
* Disable acceptance backwards compatibility discovery test as this is a breaking change
---------
Co-authored-by: Conor <cpdeethree@users.noreply.github.com>
* wip
* wip 2
* undo unwatned change
* some refactoring
* fix few tests
* more fixing
* more fixes
* Automated Commit - Format and Process Resources Changes
* more improvements
* 1 more test
* another test
* add flag for ctid enabling
* fix conflicts
* else block is not required
* use emittedAt
* skip WAL processing if streams under vacuuming
---------
Co-authored-by: subodh1810 <subodh1810@users.noreply.github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
* Fix issue in streaming JDBC database
* Bump + publish
* 🤖 Auto format source-db2 code [skip ci]
* 🤖 Auto format source-mssql code [skip ci]
* 🤖 Auto format source-postgres-strict-encrypt code [skip ci]
* 🤖 Auto format source-postgres code [skip ci]
* 🤖 Auto format source-mssql-strict-encrypt code [skip ci]
* 🤖 Auto format source-oracle code [skip ci]
* 🤖 Auto format source-tidb code [skip ci]
* 🤖 Auto format source-snowflake code [skip ci]
* 🤖 Auto format source-redshift code [skip ci]
* 🤖 Auto format source-mysql code [skip ci]
* 🤖 Auto format source-mysql-strict-encrypt code [skip ci]
---------
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
* Move icons to connector folder
* Delete old icons
* Update upload logic
* Add icon url to definitions
* Update registry model
* Populate cdn url
* DNC butcher the pipeline
* Low hanging fruit fixes
* Fix bucket name
* Merge old and new approaches
* Fix metadata upload step
* Format
* Fix test
* Adds support for incremental syncs with GLOBAL state message emission
* Adds args to Source to not break, future work is to add same features to Source perf-harness and ./gradlew :spotlessJavaApply
* initial commit
* cleanup
* cleanup
* Throw a configuration exception in case of bad jdbc url param
* End-to-End test session variable jdbc param
* reafctoring sanity
* sanity
* bump version and update note
* cherry pick fix to unrelated build error
* fix failing test
---------
Co-authored-by: Ryan Fu <ryan.fu@airbyte.io>
Split out the smallest set of reasonable changes from #26086 .
My goal was to split out the interface, as well as show how the interface it's meant to be used.
Follow up PRs:
- Split out classes from BufferManager and add more tests there.
- Add in the AsyncConsumer with tests.
- Add in the StagingConsumer factory.
* Convert source mysql decimal number with 0 decimal point to an int
* Put int representation in json for decimal with 0 decimal digits
* bump docker image ver and release note
* auto-bump connector version
---------
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
