* Bump version for redshift, bigquery, and snowflake
* auto-bump connector version
* auto-bump connector version
* Log failed refresh token response
* Revert snowflake version bump
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Add cursor_record_count to db stream state
* Add cursor record count to cursor info
* Emit max cursor record count
* Add original cursor record count
* Unify logging format
* Add backward compatible methods
* Update unit tests for state decorating iterator
* Update test (not done yet)
* Fix one more unit test
* Change where clause operator according to record count
* Add branch for null cursor
* Skip saving record count when it is 0
* Fix log wording
* Set mock record count in test
* Check cursor value instead of cursor info
* Fix source jdbc test
* Read record count from state
* Fix tests
* Add an acceptance test case
* Fix npe
* Change record count from int to long to avoid type conversion
* Fix references
* Fix oracle container
* Use uppercase for snowflake
* Use uppercase for db2
* Fix and use uppercase
* Update test case to include the edge case
* Format code
* Remove extra assertion in clickhouse
* Merge ms sql incremental query method
* Log query for debugging
* Clean up name_and_timestamp table
* Fix db2 tests
* Fix mssql tests
* Fix oracle tests
* Fix oracle tests
* Fix cockroachdb tests
* Fix snowflake tests
* Add changelog
* Fix mssql tests
* Fix db2-strict-encrypt tests
* Fix oracle-strict-encrypt tests
* Bump postgres version
* Fix oracle-strict-encrypt tests
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Use LOG4J2 to wrap connectors logs to JSON format
* log connector's stack trace directly as a message
* add stack_trace field to json template
* bump versions
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* revert versions for destinations: postgres, mssql, mysql
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* generate airbyte_type:integer
* normalization accepts `airbyte_type: integer`
* handles ints+longs
* update avro for consistency
* delete long type for now, treat all ints as longs
* update avro type mappings
{type:number, airbyte_type:integer} -> long
{type:number, airbyte_type:big_integer} -> string (i.e. "unbounded integer")
* fix test
* remove long handling
* Revert "remove long handling"
This reverts commit 33ade8d2831e675c3545ac6019d200ec312e54d9.
* Revert "update avro type mappings"
This reverts commit 5b0349badad7545efe8e1191291a628445fe1c84.
* Revert "delete long type for now, treat all ints as longs"
This reverts commit 018efd4a5d0c59f392fd8e3b0d0967c666b72947.
* Revert "update avro for consistency"
This reverts commit bcf47c6799b5906deb4f219d7f6e64ea73b41b74.
* newline@eof
* update test
* slightly better local tests
* fix test
* missed a few cases
* postgres tests use correct hostnames
* fix normalization
* fix int macro
* add test case
* normalization test output
* handle int/long correctly
* fix types for other DBs
* uint32 -> bigint; tests
* add type value assertions
* more test updates
* regenerate output
* reconcile big_integer to match docs
* update comment
* fix type
* fix mysql constructor call
* bigint only has 38 digits
* fix s3 ints, fix DAT test case
* big_integer should be string
* reduce to 28 digit big_ints
* fix test setup, mysql
* kill big_integer tests
* regenerate output
* version bumps
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
* auto-bump connector version [ci skip]
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* unfinished jdbcsource separation
* creation AbstactRelation
* Migrate StateManager to new abstract level (JdbcSource -> RelationalSource)
* fix imports
* move configs to Database level + fix MySql source
* make in line jdbc source with a new impl
* Fix ScaffoldJavaJdbcSource template
* rename `AbstractField` to `CommonField`. Now it
s not an abstract class.
+ add default implementation for `AbstractRelationalDbSource.getFullyQualifiedTableName`
* format
* rename generated files in line with their location
* bonus renaming
* move utility methods specific for jdbc source to a proper module
* internal review update
* BigQueryDatabase impl without row transformation
* add Static method for BigQueryDatabase instancing
* remove data type parameter limitation + rename class parameters
* Move DataTypeUtils from jdbs to common + impl basic types BigQueryUtils
* make DB2 in line with new relational abstract classes
* add missing import
* cover all biqquery classes + add type transformation method from StandardSQLTypeName to JsonSchemaPrimitive
* close unused connections
* add table list extract method
* bigquery source connector
* return all tables for a whole project instead of a dataset
* impl incremental fetch
* bigquery source connector
* bigquery source connector
* remove unnecessary databaseid
* add primitive type filtering
* add temporary workaround for test database.
* add dataset location
* fix table info retrieving
* handle dataset config
* Add working comprehensive test without data cases
* minor changes in the source processing
* acceptance tests; discover method fix
* discover method fix
* first comprehensinve test
* Comprehensive tests for the BigQuery source + database timeout config
* bigquery acceptance tests fix; formatting
* fix incremental sync using date, datetime, time and timestamp types
* Implement source checks: basic and dataset
* format
* revert: airbyte_protocol.by
* internal review update
* Add possibility to get list of comprehensive tests in a Markdown table format.
* Update airbyte-integrations/connectors/source-bigquery/src/main/resources/spec.json
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* review update
* Implement processing for arrays and structures
* format
* added bigquery secrets
* added bigquery secrets
* spec fix
* test configs fix
* extend mapping for Arrays and Structs
* Process nested arrays
* handle arrays of records properly.
* format
* BigQuery source docs
* docs readme update
* hide evidences
* fix changlog order
* Add bigquery to source_defintions yaml
Co-authored-by: heade <danildubinin2@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
