* source-snowflake: use a safer method for parsing a BigInteger cursor value (#22358)
* use a safer method for parsing a BigInteger cursor value
* Add testing
* fix format change
* Fix failing integration tests
* Try removing the failing incremental test
* Try removing the failing incremental test
* Fix failing test
* Add metadata to connector logs (log level, class name, method name and line number) (#23105)
* Issue #17861 Add labels, class, method name and line numbers to connector logs
* Refactored unit test
* fix for warning about UTF8 charset in test class
---------
Co-authored-by: prateekmukhedkar <prateek@airbyte.io>
* Update docker image and release notes
* auto-bump connector version
* manually bump version on spec
---------
Co-authored-by: Prateek Mukhedkar <123108018+prateekmukhedkar@users.noreply.github.com>
Co-authored-by: prateekmukhedkar <prateek@airbyte.io>
Co-authored-by: Sergio Ropero <sergio@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Add initial commit.
Missing testing
* Add unit test revoking replication privilege.
* Small fix in the documentation
* Change from `RuntimeException` to `ConfigErrorException`
* Improve test validation
* Add commentodoc
* bump version!
* Bump strict script
* auto-bump connector version
---------
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* This changes allows to filter out system views created out of system namespaces
* Add extra view
* Fix issue
* Bump Postgres source version
* bump version
* Bump alloydb
* Bump versioning
* auto-bump connector version
* Set default implementation and remove from unneeded places
---------
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* fix-postgres-source: get rid of short lived objects
* cache column info as well
* get rid of constructor
* upgrade version
* auto-bump connector version
* update definition
---------
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Bump total allocation of the source's available memory for the JDBC buffer from 60% to 70%
* Profiling setup
* Update Dockerfile
* Fixes
* Update FetchSizeConstants.java
* Update dockerfiles
* Update JdbcStreamingQueryConfig.java
* auto-bump connector version
* Manually bump source-mysql
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* revert back to erroring out on a null cursor value
* Update dockerfiles and release notes
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Update documentation for Postgres:
- Add details if you need CDC sync for some tables and also sync views.
- Reformat tables.
* Update documentation for Postgres:
- Add details if you need CDC sync for some tables and also sync views.
- Reformat tables.
* Make documentation user friendly
* Fix table
* postgres-source-cdc:handle null values for array data types
* fix test
* upgrade version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* WIP
* fix issues
* fix test
* format
* more tests
* one more test
* make lsn_commit_behaviour not required
* update expected spec
* update expected spec
* update expected spec
* PR comments
* PR comments
* upgrade version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* source-postgres:just throw a warning if a cursor column contains null value
* ignore test + update review comment
* use Disabled annotation
* upgrade version
* upgrade version for alloy db strict encrypt as well
* auto-bump connector version
* update definitions
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Stop sync on a null value in a cursor column
* Fix quoting in query to solve a failing test
* Fix another escaping issue in query
* Fix failing test
* Fix failing tests
* Test view with null value cursor
* Improve error message
* bump dockerfile version and update note
* bump version to 1.0.31
* Fix failing test
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Postgres Source Strict Encrypt: Allow connections with sslmodes 'allow' and 'prefer' if SSH tunnel established
* updated changelog
* fixed tests
* add test_strictness_level to acceptance-test-config.yml
* remove test_strictness_level to due to failed SAT test
* bump version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Initial working commit
* Code sanity. Provide no-on implementation to mysql, MSSql to allow compilation.
* Update test
* sanity
* sanity
* sanity
* sanity
* sanity
* changes per review comments
* Make heartbeat change waittime configurable.
* Trying to bypass test strictness test
* Trying to bypass test strictness test
* Trying to bypass test strictness test
* fix acceptance test config format
* add missing SAT test in config
* revert back changes in acceptance-test-config.yml
* Version and notes
* auto-bump connector version
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Postgres Source: use native Postgres timeout if it's not set by the user
* refactoring
* updated connection timeout logic and added tests for different datasources creation
* fixed pmd
* refactoring
* bump version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Postgres/MySQL Source Strict Encrypt: stop enforce SSL if ssl mode disabled
* fixed checkstyle
* updated changelog
* add tests
* replaced MySQL test to mysql-strict-encrypt module
* fixed Connection Refused for mysql test
* replaced Postgres Source strict-encrypt tests into new class
* bump version
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Source postgres: encode database name
* Source postgres, mysql: move encoding in util class, apply for mysql
* Source postgres, mysql: make var final
* Source postgres, mysql: bump version
* Source postgres, mysql: format code
* auto-bump connector version
* Source mysql: bump version
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Changes method from returning null to throwing Exception to match method structure
* Added javadoc explaining that KeyPair should not be returning null
* Moved ConnectionErrorException to commons package
* Adds error handling for SshWrappedDestinations and markdown to include changes
* Bumps version numbers for Postgres, MySQL, MSSQL
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Publishes Postgres, MySQL, MSSQL source with changes from #18041
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Add cursor_record_count to db stream state
* Add cursor record count to cursor info
* Emit max cursor record count
* Add original cursor record count
* Unify logging format
* Add backward compatible methods
* Update unit tests for state decorating iterator
* Update test (not done yet)
* Fix one more unit test
* Change where clause operator according to record count
* Add branch for null cursor
* Skip saving record count when it is 0
* Fix log wording
* Set mock record count in test
* Check cursor value instead of cursor info
* Fix source jdbc test
* Read record count from state
* Fix tests
* Add an acceptance test case
* Fix npe
* Change record count from int to long to avoid type conversion
* Fix references
* Fix oracle container
* Use uppercase for snowflake
* Use uppercase for db2
* Fix and use uppercase
* Update test case to include the edge case
* Format code
* Remove extra assertion in clickhouse
* Merge ms sql incremental query method
* Log query for debugging
* Clean up name_and_timestamp table
* Fix db2 tests
* Fix mssql tests
* Fix oracle tests
* Fix oracle tests
* Fix cockroachdb tests
* Fix snowflake tests
* Add changelog
* Fix mssql tests
* Fix db2-strict-encrypt tests
* Fix oracle-strict-encrypt tests
* Bump postgres version
* Fix oracle-strict-encrypt tests
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* postgres-source: handle 24:00:00 value for time column
* address review comment
* fix test + use LocalTime.MAX
* bump version
* update alloy db as well
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Match entry in client cert keystore to hardcoded postgres driver value
* Match entry in client cert keystore to hardcoded postgres driver value
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* upgrade debezium version to 1.9.6
* test override not required
* bump version
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Make error more descriptive in case of no encryption state
* Fix failing test
* Change error message on strict-encrypt postgres destination as well. Fix failing test
* update versions and changelogs
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
