1
0
mirror of synced 2025-12-25 02:09:19 -05:00
Files
airbyte/docs/integrations/sources
Andrii Leonets 9151d83bcf 🐛 BigQuery source: Fix nested arrays (#4981)
* unfinished jdbcsource separation

* creation AbstactRelation

* Migrate StateManager to new abstract level (JdbcSource -> RelationalSource)

* fix imports

* move configs to Database level + fix MySql source

* make in line jdbc source with a new impl

* Fix ScaffoldJavaJdbcSource template

* rename `AbstractField` to `CommonField`. Now it
s not an abstract class.
+ add default implementation for `AbstractRelationalDbSource.getFullyQualifiedTableName`

* format

* rename generated files in line with their location

* bonus renaming

* move utility methods specific for jdbc source to a proper module

* internal review update

* BigQueryDatabase impl without row transformation

* add Static method for BigQueryDatabase instancing

* remove data type parameter limitation + rename class parameters

* Move DataTypeUtils from jdbs to common + impl basic types BigQueryUtils

* make DB2 in line with new relational abstract classes

* add missing import

* cover all biqquery classes + add type transformation method from StandardSQLTypeName to JsonSchemaPrimitive

* close unused connections

* add table list extract method

* bigquery source connector

* return all tables for a whole project instead of a dataset

* impl incremental fetch

* bigquery source connector

* bigquery source connector

* remove unnecessary databaseid

* add primitive type filtering

* add temporary workaround for test database.

* add dataset location

* fix table info retrieving

* handle dataset config

* Add working comprehensive test without data cases

* minor changes in the source processing

* acceptance tests; discover method fix

* discover method fix

* first comprehensinve test

* Comprehensive tests for the BigQuery source + database timeout config

* bigquery acceptance tests fix; formatting

* fix incremental sync using date, datetime, time and timestamp types

* Implement source checks: basic and dataset

* format

* revert: airbyte_protocol.by

* internal review update

* Add possibility to get list of comprehensive tests in a Markdown table format.

* Update airbyte-integrations/connectors/source-bigquery/src/main/resources/spec.json

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* review update

* Implement processing for arrays and structures

* format

* added bigquery secrets

* added bigquery secrets

* spec fix

* test configs fix

* extend mapping for Arrays and Structs

* Process nested arrays

* handle arrays of records properly.

* format

* BigQuery source docs

* docs readme update

* hide evidences

* fix changlog order

* Add bigquery to source_defintions yaml

Co-authored-by: heade <danildubinin2@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-07-27 20:27:12 +03:00
..
2021-06-08 10:36:31 +03:00
2021-05-27 16:38:23 +11:00
2021-07-23 12:37:27 +03:00
2021-06-23 18:06:31 +03:00
2021-07-09 09:06:04 -07:00
2021-06-18 17:45:09 -03:00
2021-07-21 10:51:48 -07:00
2021-06-22 20:22:02 +03:00
2021-05-19 18:36:09 -04:00
2021-07-08 18:15:54 +03:00
2021-07-26 21:55:07 -04:00
2021-05-20 14:45:52 +03:00
2021-06-10 12:52:47 -07:00
2021-07-13 13:06:50 +03:00
2021-07-20 00:17:53 -07:00

description
description
We're adding new source connectors every day: APIs, applications, databases...

Sources