* Bump total allocation of the source's available memory for the JDBC buffer from 60% to 70%
* Profiling setup
* Update Dockerfile
* Fixes
* Update FetchSizeConstants.java
* Update dockerfiles
* Update JdbcStreamingQueryConfig.java
* auto-bump connector version
* Manually bump source-mysql
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Source mysql: mark unknown column exception as config error
* Source mysql: format code
* Source mysql: bump version
* Source mysql: bump version
* Source mysql: bump spec and definition version
* Source mysql: bump spec and definition version
* Source mysql: revert spec and definition version with unfamiliar changes
* Source mysql: bump spec and definition version
* Source mysql: revert manually bumped spec and definition version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Test
* Check for REPLICATION CLIENT PRIVILEGE
* format
* undo format
* Fix test + comments
* Simplify checked function
* Bump up version + log
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* mysql-source:tinyint-unsigned should not be identified as boolean
* upgrade version
* fix encrypted spec
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Postgres/MySQL Source Strict Encrypt: stop enforce SSL if ssl mode disabled
* fixed checkstyle
* updated changelog
* add tests
* replaced MySQL test to mysql-strict-encrypt module
* fixed Connection Refused for mysql test
* replaced Postgres Source strict-encrypt tests into new class
* bump version
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Source postgres: encode database name
* Source postgres, mysql: move encoding in util class, apply for mysql
* Source postgres, mysql: make var final
* Source postgres, mysql: bump version
* Source postgres, mysql: format code
* auto-bump connector version
* Source mysql: bump version
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Changes method from returning null to throwing Exception to match method structure
* Added javadoc explaining that KeyPair should not be returning null
* Moved ConnectionErrorException to commons package
* Adds error handling for SshWrappedDestinations and markdown to include changes
* Bumps version numbers for Postgres, MySQL, MSSQL
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Publishes Postgres, MySQL, MSSQL source with changes from #18041
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Add cursor_record_count to db stream state
* Add cursor record count to cursor info
* Emit max cursor record count
* Add original cursor record count
* Unify logging format
* Add backward compatible methods
* Update unit tests for state decorating iterator
* Update test (not done yet)
* Fix one more unit test
* Change where clause operator according to record count
* Add branch for null cursor
* Skip saving record count when it is 0
* Fix log wording
* Set mock record count in test
* Check cursor value instead of cursor info
* Fix source jdbc test
* Read record count from state
* Fix tests
* Add an acceptance test case
* Fix npe
* Change record count from int to long to avoid type conversion
* Fix references
* Fix oracle container
* Use uppercase for snowflake
* Use uppercase for db2
* Fix and use uppercase
* Update test case to include the edge case
* Format code
* Remove extra assertion in clickhouse
* Merge ms sql incremental query method
* Log query for debugging
* Clean up name_and_timestamp table
* Fix db2 tests
* Fix mssql tests
* Fix oracle tests
* Fix oracle tests
* Fix cockroachdb tests
* Fix snowflake tests
* Add changelog
* Fix mssql tests
* Fix db2-strict-encrypt tests
* Fix oracle-strict-encrypt tests
* Bump postgres version
* Fix oracle-strict-encrypt tests
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* 8890 Fix large table issue by fetch size
* 8890 Bump version
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* upgrade debezium version to 1.9.6
* test override not required
* bump version
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Fix URL of MySqlSourceDatatypeTest.java
* Fix naming of JDBC URL parameters field to match Postgres
* Fix a typo in JDBC params description
* Bump version to 1.0.0 for beta
* replace e.g. with example
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Make error more descriptive in case of no encryption state
* Fix failing test
* Change error message on strict-encrypt postgres destination as well. Fix failing test
* update versions and changelogs
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* mysql source : implement support for snapshot of new tables in cdc mode
* undo unwanted changes
* add more assertions
* format
* fix build
* fix build
* revert acceptance test changes
* bump version
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Use LOG4J2 to wrap connectors logs to JSON format
* log connector's stack trace directly as a message
* add stack_trace field to json template
* bump versions
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* revert versions for destinations: postgres, mssql, mysql
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Cleaned error messages factory PR
* Bumped MySQL and Postgres version
* Fixed messages and typos in test
* Fixes the changelog conflict with per-stream state
* Added note for flaky test
* Bumps mysql version to match changelog
* Added exception objects to all LOGGER.error for more visibility
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* implement support for per-stream state in mysql source
* enable flag in tests
* activate per stream in tests
* enable per stream state in test
* bump version
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Fixed bucket naming for S3
* removed redundant configs
* MySQL Source : Standardize spec.json for DB connectors that support log-based CDC replication
* fixed strict encrypt tests
* fixed mysql tests
* bump version
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Emit intermediate state messages from mysql source
* Add pr id
* Try ec2 runner with larger disk
* Revert "Try ec2 runner with larger disk"
This reverts commit fb4d99489d.
* enforce ssl in strict mysql and postgres source
* Add test to verify tunnel aware strict SSL behavior
* bump version numbers and changelogs
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* new ssl flow for connectors
* Bump versions and update changelogs
* Address review comments
* sanity
* Revert Dockerfile versions
* Fix error in test
* The test certificate we use do not contain a valid hostname so we can only verify ca trust but not hostname match
* verify-ca and verify-full SSL modes are not related to whether a client certificate is used. Make client certificate optional on both modes
* Bump Dockerfile versions and correct change notes.
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* fix handling for temporal data type columns with default values
* add tests
* minor NIT comment
* bump version
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* add logic to skip incremental mode for tables which do not have a field types that could not be used as cursor
* added tests for cursor fields incremental updates
* remove not used methods
* fix tests
* fix formatting
* revert RedshiftInsertDestinationAcceptanceTest
* fix ssl clickhouse test
* fix codereview comments
* fix clickhouse strict encrypt test
* fix postgres source after merge from master
* optimize ssl clickhouse test logic
* fix db2 test
* added check for cursor field should be not nullable, fix formatting
* fixed tests, formatting, issue after merge from master
* fix merge issues
* fix test for source-jdbc and remove some code used for local testing
* added description for cockroach db tests
* test changes
* refactoring for cursor field tests after code review comments
* refactoring for cursor field tests after code review comments
* fix code review comments
* remove unused imports
* fix formatting
* fix compilation error
* fix postgres and postgress ssl test
* Fixed bucket naming for S3
* remove CHAR, NCHAR from possible cursor types
* remove SYMBOL from possible cursor types
* removed redundant configs
* fixed mysql-strict-encrypt tests
* fixed mongodb-v2 tests
* minor format changes
* bump version
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: vmaltsev <ihbectop@gmail.com>
Co-authored-by: subodh <subodh1810@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* updated mysql source specification and added field for root and clients SSL certificates
* added SSL mode for mysql source
* fixed code style
* updated run process timeout
* updated method for create keystore and updated tests
* updated normalization version for postgres destination
* updated normalization version for postgres destination
* added tests for connection with certificates
* updated tests for connection with full certificates and added tests for CA certificate
* updated tests
* updated source-mysql-strict-encrypt and updated versions
* updated code style
* updated doc
* updated specs
* fixed minor remarks
* fixed minor remarks
* updated tests
* fixed remarks and updated specification
* fixed mysql sources connectors version
* added CDC + SSL Certificates tests
* added property for CDC and added tests for test SSL with CDC together
* fixed MySqlStrictEncryptJdbcSourceAcceptanceTest for work with datetime format
* added property for CDC and added tests for test SSL with CDC together
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Allow additional properties in db stream state
* Bump postgres version
* Bump mysql version
* Bump mssql version
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* generate airbyte_type:integer
* normalization accepts `airbyte_type: integer`
* handles ints+longs
* update avro for consistency
* delete long type for now, treat all ints as longs
* update avro type mappings
{type:number, airbyte_type:integer} -> long
{type:number, airbyte_type:big_integer} -> string (i.e. "unbounded integer")
* fix test
* remove long handling
* Revert "remove long handling"
This reverts commit 33ade8d2831e675c3545ac6019d200ec312e54d9.
* Revert "update avro type mappings"
This reverts commit 5b0349badad7545efe8e1191291a628445fe1c84.
* Revert "delete long type for now, treat all ints as longs"
This reverts commit 018efd4a5d0c59f392fd8e3b0d0967c666b72947.
* Revert "update avro for consistency"
This reverts commit bcf47c6799b5906deb4f219d7f6e64ea73b41b74.
* newline@eof
* update test
* slightly better local tests
* fix test
* missed a few cases
* postgres tests use correct hostnames
* fix normalization
* fix int macro
* add test case
* normalization test output
* handle int/long correctly
* fix types for other DBs
* uint32 -> bigint; tests
* add type value assertions
* more test updates
* regenerate output
* reconcile big_integer to match docs
* update comment
* fix type
* fix mysql constructor call
* bigint only has 38 digits
* fix s3 ints, fix DAT test case
* big_integer should be string
* reduce to 28 digit big_ints
* fix test setup, mysql
* kill big_integer tests
* regenerate output
* version bumps
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
