* Release source connectors
* Fix issue with database connection in test
* Fix failing tests due to authentication
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* WIP Per-stream state support for Postgres source
* Fix failing test
* Improve code coverage
* Make global the default state manager
* Add legacy adapter state manager
* Formatting
* Include legacy state for backwards compatibility
* Add global state manager
* Implement Global/CDC state handling
* Fix test issues
* Fix issue with updated method signature
* Handle empty state case in global state manager
* Adjust to protocol changes
* Fix failing acceptance tests
* Fix failing test
* Fix unmodifiable list issue
* Fix unmodifiable exception
* PR feedback
* Abstract global state manager selection
* Handle conversion between different state types
* Handle invalid conversion
* Rename parameter
* Refactor state manager creation
* Fix failing tests
* Fix failing integration tests
* Add CDC test
* Fix failing integration test
* Revert change
* Fix failing integration test
* Use per-stream for postgres tests
* Formatting
* Correct stream descriptor validation
* Correct permalink
* PR feedback
* Add deployment mode to env shared with jobs
* Add adaptive runners
* Migrate postgres source to use adaptive runner
* Add an array of specs in docker image spec definition
* Add copyright
* Parse docker image spec with specs list
* Update spec yaml files
* Pass in DEPLOYMENT_MODE to docker compose file
* Revert "Parse docker image spec with specs list"
This reverts commit 8fe41dd3b7.
* Revert changes in docker image spec
* Read cloud specific spec files based on deployment mode
* Revert "Update spec yaml files"
This reverts commit 059f326432.
* Publish cloud spec file if necessary
* Fix upload script
* Move test files
* Update docker compose file
* Format code
* Add comment about spec filename
* Add unit tests
* Remove redundant jdbc acceptance test
When running `PostgresStrictEncryptJdbcSourceAcceptanceTest`, the `discover` method tests always fail because there are unexpected columns in the catalog:
- `wakeup_at`
- `last_visited_at`
- `last_comment_at`
These columns only exist in `PostgresJdbcSourceAcceptanceTest`. And this failure cannot be reproduced locally.
The hypothesis is that when the JDBC unit tests are run on CI, they are run in parallel, and the same testcontainer is used for both tests. That's why the strict encrypt test can discover columns from the oridinary unit test.
Given that the JDBC strict encrypt test is basically redundant, it is removed.
* Postgres Source: fixed truncated precision if the value of the millisecond or second is 0
* check CI with 1.15.3 testcontainer
* check CI with 1.15.3 testcontainer
* returned latest version of testcontainer
* fixed checkstyle
* fixed checkstyle
* returned latest testcontainer version
* updated CHANGELOG
* bump version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Fall back to parsing w/ or w/o TZ if parsing a date or a time string fails
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Switch to measure max row byte size
* Reduce fetch size change logs
* Update unit tests
* Determine jdbc buffer size based on max memory
* Bump postgres version
* Bump postgres version
* Bump mysql version
* Bump mssql version
* Format java code
* Increase hikari connection timeout
* Update data source default parameters
* auto-bump connector version
* Mark postgres 0.4.21 as not published
* Revert "Bump mysql version"
This reverts commit ad9135258c.
* Fix unit test
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Part 1 of #13122.
Rename airbyte-db:lib to airbyte-db:db-lib.
Rename airbyte-metrics:lib to airbyte-metrics:metrics-lib
Rename airbyte-protocol:models to airbyte-protocol:protocol-models.
Explanation for what is happening:
Identically named subprojects have the following issues:
- publishing as is leads to classpath confusion when the jars with the same names are placed in the Java distribution. This leads to NoClassDefFound errors on runtime.
- deconflicting the jar names without changing directory names leads to dependency errors as the OSS jar pom files are generated using project dependencies (suggesting a dependency a sibling subproject in the same repo) that use subprojects group and name as a reference. This means the generated jars look for Jars that do not exists (as their names have been changed) and cannot compile.
- the workaround to changing a subproject's name involves resetting the subproject's name in the settings.gradle and depending on the new name in each build.gradle. This increases configuration burden and decreases the ease of reading, since one will have to check the settings.gradle to know what the right subproject name is. See Projects with same name lead to unintended conflict resolution gradle/gradle#847 for more info.
- given that Gradle itself doesn't have support for identically named subprojects (see the linked issue), the simplest solution is to not allow duplicated directories. I've only renamed conflicting directories here to keep things simple. I will create a follow up issues to enforce non-identical subproject names in our builds.
* cdc - add support of tsquery type
* rebase on master
* Add changes made by the build
* Update version in Dockerfile
* auto-bump connector version
Co-authored-by: grishick <greg@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Better database connection handling for connectors
* Log connection error
* Properly close connection
* Remove unused method
* Close data source
* Use utility to close data source
* Use utility to close data source
* PR feedback
* Add Databricks driver
* Use driver class enum
* Use correct config
* Ensure config created before use
* Fix failing integration test
* Create DSLContext before use
* Address integration test failures
* Ensure DSLContext is closed
* Fix compile error
* Use correct datasource
* Use correct connection properties
* Close DSLContext
* Close DSLContext
* Fix integration test failures
* Properly close datasource
* Fix compilation issues
* Use existing database object
* Wrap close in try/finally
* Update test
* Wrap close in try/finally
* Ensure DSLContext is created
* Revert change to test
* Use correct data source
* Remove unused import
* More cleanup
* Add missing annotation
* Only initialize data source once
* Remove unused import
* Force testcontainers version
* Fix testcontainer issue
* Fix failing test
* Properly close all data sources
* Clear data sources after closing
* Fix compile error
* Fix compilation error
* Add missing method
* Remove host in azure username
* Bump version
* Fix test
* Replace host with jdbc_url
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Use role-based permissions for source-postgres
Modify permissions query to add recursive lookup of role-based
permissions. This solves problems users are having with not being able
to list tables for postgres souces
* Add unit test of discovery logic
* Default scaffold to use adaptive streaming config
* Switch more connectors to use adaptive streaming config
* Bump version for cockroach db
* Bump version for db2
* Bump mssql version
* Bump mysql version
* Bump oracle version
* Bump postgres version
* Bump redshift version
* Bump snowflake version
* Bump tidb version
* auto-bump connector version
* Fix db2 findbug issue
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* Fix more findbug issues
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* Fix findbug issue for mysql-strict-encrypt
* Fix findbugs issue for oracle source
* auto-bump connector version
* Remove suppress warnings annotation
* Fix oracle encrypt tests
* Fix oracle encrypt acceptance test
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Merge all streaming configs to one
* Implement new streaming query config
* Format code
* Fix comparison
* Use double for mean byte size
* Update fetch size only when changed
* Calculate mean size by sampling n rows
* Add javadoc
* Change min fetch size to 1
* Add comment by buffer size
* Update java connector template
* Perform division first
* Add unit test for fetching large rows
* Format code
* Fix connector compilation error
* combine all
working locally
missing fields in click view and keyqord_view
add date to campaigns
add fields to click_view
test
* configured_catalog campaigns
* rem keywordview
* conflicts
* string type
* remove network type
* change ad group id to integer
* fix tests
* bump connector version
* bump connector config doc vesion
* run format
Co-authored-by: Konrad <konrad@PB-Mac8.local>
Co-authored-by: schlattk <konrad@pensionbee.com>
* Use time zone that does not switch to DST
* Fix timezone tests
- Use timezone MST, which is explicitly defined as 'standard time' and will not be affected by DST
* JDBC sources: update for sources that not have permissions issues
* Postgres, CockroachDB and DB2 sources: updated case when user has none permitted table
* Postgres, CockroachDB and DB2 sources: updated case when user has none permitted table
* Postgres, CockroachDB and DB2 sources: bump versions
* Postgres, CockroachDB and DB2 sources: bump source specs
* Postgres, CockroachDB and DB2 sources: revert changes for postgres-strict-encrypt
