* copy files from edgao branch
* start writing create table statement
* add basic unit test setup
* create a table, probably
* remove outdated todo
* derp, one more column
* ugh
* add partitioning+clustering
* use StringSubstitutor
* substitutions in updateTable
* wip generate update/insert statement
* split up into smaller methods
* handle json types correctly
* rename stuff
* more json_query vs _value stuff
* minor tweak
* super basic test setup
* laying foundation for type parsing
* more stuff
* tweaks
* more progress on type parsing
* fix json_value stuff?
* misc fixes in insert
* fix dedupFinalTable
* add testDedupRaw
* full e2e test
* type parsing: gave up and mirrored the dbt code structure to avoid bugs
* type parsing - more cleanup
* handle column name collisions
* handle tablename collisions...?
* comments
* remove original ns/name from quotedstream
* also javadoc
* remove redundant method
* fix table rename
* add incremental append test
* add full refresh append test
* comment
* call T+D sql in a reasonable location for standard inserts
* add config option
* use config option here
* type parsing - fix fromJsonSchema
* gate everything
* log query + runtime
* add spec option temporarily
* Raw Table Updates
* fix more stuff
* first big pass at toDialectType
* no quotes
* wrap everything in quotes
* resolve some TODOs
* log sql statement in tests
* overwriteFinalTable returns optional
* minor clean up
* add raw dataset override
* try to preserve the original namespace for t+d?
* write to the raw table correctly
* update todos
* write directly to raw table
this is kind of dumb because we're still trying to do tmp table operations,
and we still don't ack state until the end of the entire sync.
* standard inserts write to raw table correctly
* imports + log statements
* move logs + add comment
* explicitly create raw table
* move comment to better place
* Typing issues
* bash attempt
* formatting updates
* formatting updates
* write to the airbyte schema by default unless overriden by config options
* standard inserts truncate raw table at start of sync
* full refresh overwrite will overwrite correctly!
* fix avro record schema parsing
* better raw table recreate
* rename raw table to match standard inserts
* full refresh overwrite does tmp table things
* small clean up
* small clean up
* remove errors entry if no errors
* pull out destination config into singleton
* clean up singleton stuff
* make sure dest config exists when trying to do lookups
* avoid stringifying null
* quick thoughts on alter table
* add basic cdc testcase
* tweak cdc test setup
* rename raw table to match standard inserts
* minor tweak
* delete exact sql string assertions
* switch to JSON type
* minor cleanup
* sql whitespace changes
* explain cdc deletions
* GCS Staging Full Refresh create temp table
* assert schema
* first out of order cdc test
* add another cdc test case (currently failing)
* better test structure
* make this work
* oops, fix test
* stop trying to delete deletion records
* minor improvements to code+test
* enable concurrent test runs on integration test
* move stuff to static initializer
* extract utility method
* formatting
* Move conditional to the base java package, replace conditionals which did not use the typing and deduping flag but should have been.
* 🤖 Auto format destination-bigquery code [skip ci]
* 🤖 Auto format destination-gcs code [skip ci]
* switch back to empty list; write big assert
* minor wording tweaks
* 🤖 Auto format destination-bigquery code [skip ci]
* 🤖 Auto format destination-gcs code [skip ci]
* DestinationConfigTest
* 🤖 Auto format destination-bigquery code [skip ci]
* 🤖 Auto format destination-gcs code [skip ci]
* formatting
* remove ParsedType
* 🤖 Auto format destination-gcs code [skip ci]
* 🤖 Auto format destination-bigquery code [skip ci]
* tests verify every data type
* 🤖 Auto format destination-bigquery code [skip ci]
* 🤖 Auto format destination-gcs code [skip ci]
* full update with all data types
* 🤖 Auto format destination-bigquery code [skip ci]
* 🤖 Auto format destination-gcs code [skip ci]
* move stuff to new base lib
* 🤖 Auto format destination-gcs code [skip ci]
* Automated Commit - Formatting Changes
* 🤖 Auto format destination-bigquery code [skip ci]
* fix test
* 🤖 Auto format destination-bigquery code [skip ci]
* 🤖 Auto format destination-bigquery code [skip ci]
* 🤖 Auto format destination-gcs code [skip ci]
* asserts in dedupFinalTable
* better asserts in dedupRawTable
* [wip] test case for all data types
* 🤖 Auto format destination-gcs code [skip ci]
* 🤖 Auto format destination-bigquery code [skip ci]
* AirbyteTypeTest
* Automated Commit - Formatting Changes
* remove comments
* test chooseOneOf
* slightly better test output
* Automated Commit - Formatting Changes
* add some awful pretty print code
* more comment
* minor tweaks
* verify array/object type
* fix test
* handle deletions more correctly
* test toDialectType
* Destinations v2: better namespace handling (#27682)
* [wip] better namespace handling
* 🤖 Auto format destination-bigquery code [skip ci]
* wip also implement in gcs
* get gcs working (?)
* 🤖 Auto format destination-bigquery code [skip ci]
* remove duplicate method
* 🤖 Auto format destination-bigquery code [skip ci]
* fixed my code style settings
* make ci happy?
* 🤖 Auto format destination-bigquery code [skip ci]
* make ci happy?
* remove incorrect test
* blank line change
* initialize singleton
---------
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
* reset args correctly
* Automated Commit - Formatting Changes
* more bash stuff
* parse implicit structs
* initialize singleton in more tests
* Automated Commit - Formatting Changes
* I missed this namespace handling thing
* test more schemas
* fix singular types specified in arrays
* Automated Commit - Formatting Changes
* disable test for unimplemented feature
* initialize singleton
* remove spec options; changelogs+metadata
* randomize namespace
* also bump dockerfile
* unremove namespace sanitizing in legacy mode
* ... disable the correct test
* even more unit test fixes!
* move integration test to integration tests
---------
Co-authored-by: Cynthia Yin <cynthia@airbyte.io>
Co-authored-by: Joe Bell <joseph.bell@airbyte.io>
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: edgao <edgao@users.noreply.github.com>
Co-authored-by: cynthiaxyin <cynthiaxyin@users.noreply.github.com>
* Update quickbooks following state management cleanup
* Update changelogs
* Update tests
* Ensure CDK version
* Update update_state to close_slice
* Update test to use partition end datetime
* Update considering fix
* renaming
* Source Typeform: Add OAuth support
* Update spec to support OAuth
* Update spec structure and check implementation
* fix flake warnings
* Update spec.json
* Update unit tests
* Update spec and specify access_token path in check
* Keep structure for previous API token configs
* Remove backwards compatibility entries
* Get auth headers from authenticator, add forms_ids back to spec
* 🤖 Auto format source-typeform code [skip ci]
* Add major version bump
* Update unit tests
* Make "credentials" a required property
* Update version, update CAT config
* Skip previous version in spec and check tests
* Add OAuth information to docs, update setup description
* Update documentaion
* Make start_date optional
* Set start_date as optional in docs
* Apply suggestions from code review
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
---------
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Incorrect way to do this
* Working
* Make tests pretty
* Revert "Incorrect way to do this"
This reverts commit f8e29594c1b5fa07bad805806f2571af883d27fd.
* Add backwards compatibility docs
* bump version
* format
---------
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
* Initial commit
* Update docs
* Update zendesk-support.md
* fix: add missing comma to unit test tuple
* Update unit_test.py
* 🤖 Auto format source-zendesk-support code [skip ci]
* increment expected_n_streams in unit tests
* increment one more thing for expected_n_streams in unit tests
* increment connector version
* fix unit test
---------
Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: sh4sh <6833405+sh4sh@users.noreply.github.com>
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
* migrated to 2021-11 api version
* 🤖 Auto format source-recharge code [skip ci]
* updated changelog
* updated request params
* increased timeout for sat
* increased timeout for sat
* removed stream slices
* added stream_slices
---------
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
* `yum clean all` after every yum install to save space
* docs and versions
* update env clean
* fix python install confusion
---------
Co-authored-by: Augustin <augustin@airbyte.io>
* add info blurb to Cloud Bucket Storage sources and destinations
* remove egress alert from destination docs
* remove egress alert from destinations, add it to sources
* revert sources to match master
* Add sponsored_product_campaign_negative_keywords stream
* revert changes
* 🤖 Auto format source-amazon-ads code [skip ci]
* Add sponsored_display_budget_rules stream
* 🤖 Auto format source-amazon-ads code [skip ci]
* ignore dailyBudget field
it can be updated, also it is sometimes integer, sometimes float"
* Fix sponsored_product_campaigns
* Fix sponsored_product_campaigns
* merged with the latest version
* updated source test
---------
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
