9151d83bcf
* unfinished jdbcsource separation * creation AbstactRelation * Migrate StateManager to new abstract level (JdbcSource -> RelationalSource) * fix imports * move configs to Database level + fix MySql source * make in line jdbc source with a new impl * Fix ScaffoldJavaJdbcSource template * rename `AbstractField` to `CommonField`. Now it s not an abstract class. + add default implementation for `AbstractRelationalDbSource.getFullyQualifiedTableName` * format * rename generated files in line with their location * bonus renaming * move utility methods specific for jdbc source to a proper module * internal review update * BigQueryDatabase impl without row transformation * add Static method for BigQueryDatabase instancing * remove data type parameter limitation + rename class parameters * Move DataTypeUtils from jdbs to common + impl basic types BigQueryUtils * make DB2 in line with new relational abstract classes * add missing import * cover all biqquery classes + add type transformation method from StandardSQLTypeName to JsonSchemaPrimitive * close unused connections * add table list extract method * bigquery source connector * return all tables for a whole project instead of a dataset * impl incremental fetch * bigquery source connector * bigquery source connector * remove unnecessary databaseid * add primitive type filtering * add temporary workaround for test database. * add dataset location * fix table info retrieving * handle dataset config * Add working comprehensive test without data cases * minor changes in the source processing * acceptance tests; discover method fix * discover method fix * first comprehensinve test * Comprehensive tests for the BigQuery source + database timeout config * bigquery acceptance tests fix; formatting * fix incremental sync using date, datetime, time and timestamp types * Implement source checks: basic and dataset * format * revert: airbyte_protocol.by * internal review update * Add possibility to get list of comprehensive tests in a Markdown table format. * Update airbyte-integrations/connectors/source-bigquery/src/main/resources/spec.json Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * review update * Implement processing for arrays and structures * format * added bigquery secrets * added bigquery secrets * spec fix * test configs fix * extend mapping for Arrays and Structs * Process nested arrays * handle arrays of records properly. * format * BigQuery source docs * docs readme update * hide evidences * fix changlog order * Add bigquery to source_defintions yaml Co-authored-by: heade <danildubinin2@gmail.com> Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
22 lines
878 B
Markdown
22 lines
878 B
Markdown
# BigQuery Test Configuration
|
|
|
|
In order to test the BigQuery source, you need a service account key file.
|
|
|
|
## Community Contributor
|
|
|
|
As a community contributor, you will need access to a GCP project and BigQuery to run tests.
|
|
|
|
1. Go to the `Service Accounts` page on the GCP console
|
|
1. Click on `+ Create Service Account" button
|
|
1. Fill out a descriptive name/id/description
|
|
1. Click the edit icon next to the service account you created on the `IAM` page
|
|
1. Add the `BigQuery Data Editor` and `BigQuery User` role
|
|
1. Go back to the `Service Accounts` page and use the actions modal to `Create Key`
|
|
1. Download this key as a JSON file
|
|
1. Move and rename this file to `secrets/credentials.json`
|
|
|
|
## Airbyte Employee
|
|
|
|
1. Access the `BigQuery Integration Test User` secret on Rippling under the `Engineering` folder
|
|
1. Create a file with the contents at `secrets/credentials.json`
|