* Clean fieldPath before use in projection
* Update release notes
* Automated Commit - Format and Process Resources Changes
* Remove prefix $ from field name
---------
Co-authored-by: jdpgrailsdev <jdpgrailsdev@users.noreply.github.com>
* enable ctid feature flag for testing.
bump versions
* Removed PostgresSourceTest pre-ctid and turn CtidEnabledPostgresSourceTest back into PostgresJdbcSourceAcceptanceTest (#28341)
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
* Update PostgresSourceTest to no longer use Legacy AirbyteStateType
* Fix test.
Uncovered a bug in handling of money[] and float[]
* revert version change
* Orderly removal of ctid flag for cursor
* Handle correctly full refresh streams. (#28369)
* Handle correctly full refresh streams.
Handle xmin full refresh (currently blocked in ui).
remove unnecessary debug logs
* Test catalog with a mix of incremental and full refresh streams
* Filter out full refresh streams since they are not expected to be retuned by getIncrementalIterators()
* Filter out full refresh existing streams as well
* revert name to reflect incremental only streams
* 28505 xmin + ctid reclassify connection when ctid is not possible (#28548)
* reclassify to xmin stream we cannot read with ctid
* fix reclassify
* improve test
* Add initial load using xmin flow
* typo
* Added test for xmin initial load flow
* Improve testing for initial load with xmin flow
* sanity
* sanity
* bump versions and release notes
---------
Co-authored-by: Duy Nguyen <duy@airbyte.io>
* add new streams
* update acceptancte test config
* format files
* fix format and update docs
* remove for
* update expected records
* update expected records
* add new fields to ad reports streams
* fix unit test
* fix test and check connection
* fix unit test
* add conversions too
* add additional fields
* fix: format python files with flake
* fix: add cta_conversion, vta_conversion to streams.py
* fix: python format
* fix: updated expected_records for ad_reports stream
---------
Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Augustin <augustin@airbyte.io>
* Record statistics about source database/collection
* Formatting
* Remove unnecessary log message
* Only record statistics during sync
* Emit structured Google log data
* Revert use of System.out.println
* Try using Log4j ObjectMessage
* Revert "Try using Log4j ObjectMessage"
This reverts commit d518c87aa8.
* Prepare release
* Update changelog
* Correct release version
* rename for clarity
* fix cleanup method
* giant commit because I'm irresponsible
* rename constant
* better raw table creation
* fix build?
* move code around
* tweaks
* more code shuffling
* Automated Commit - Format and Process Resources Changes
* add tests
* minor tweak
* remove unimportant methods
* cleanup
* Automated Commit - Format and Process Resources Changes
* derp
* clean up tests
* some more fixes post-merge
* botched merge
* create NoopTyperDeduper
* try and update everything to work?
* tweak comment
* move suffix args to end of list
* fix exception message
* Automated Commit - Format and Process Resources Changes
* add sqlgenerator test for softReset
* only prepare once
* update log message
* do what intellij says
* implement one more test
* less indirection
* Automated Commit - Format and Process Resources Changes
* rename test
* use noop in test
* version bump + changelog
* use stringutils
* fix typo
* flip if-statement
* typo
* simplify logic
* fix schema change logic
* typo
* use spy for clarity
* Automated Commit - Format and Process Resources Changes
* better test teardown
* slightly better logs
* fix exception message
* softReset returns single string
* Automated Commit - Format and Process Resources Changes
* simplify if chain
---------
Co-authored-by: edgao <edgao@users.noreply.github.com>
* Add ability to detect differences in expected Schemas and perform soft resets
* Remove alter table for overwrite syncs since its unneccessary
* Updates after testing
* pr reorganize
* comments
* add collection util test
* Add Tests
* bump version
* Automated Commit - Format and Process Resources Changes
* Destination BigQuery - Reduce amount of typing and deduping for GCS staging (#28489)
* undo comment out
* centralize t&d logic for staging and standard, add valve to staging
* Share more logic for typing and deduping
* Remove record checking logic and use only time for staging inserts
* Add Javadoc
* Automated Commit - Format and Process Resources Changes
---------
Co-authored-by: jbfbell <jbfbell@users.noreply.github.com>
* Change TableNotMigratedException to extend runtime exception, remove SqlGenerator interface method
* Make Lambda slightly more readable
* add test for validating v2 schemas
* change soft reset to single string
* convert back to list, update dockerfile
* remove needless default
---------
Co-authored-by: jbfbell <jbfbell@users.noreply.github.com>
* added unexpected fields
* Update schema in forms stream
* Update schema for email_events, email_subscriptions, engagements, campaigns streams
* Update changelog
* Update schema for contact_lists and forms streams, update expected records
* Update version, remove fix step from dockerfile
* Add fields to goals and forms schema, add some fields to ignored_fields
* Add missing type declarations
---------
Co-authored-by: Arsen Losenko <20901439+arsenlosenko@users.noreply.github.com>
* Remove outdated note on part of the EU workflow running in the US
* Add note about storage of Airbyte users information
* Remove us ip from the list of ip to whitelist of eu data residency
* cherry-pick me: add more logs to migration guide check
* cherry-pick me: migration guide should be named according to name in metadata
* cherry-pick me: change logging for expected headers
* cherry-pick me: include migration guide file path in qa checks step
* add breaking changes section to docs, update link to breaking changes section
* update link to semantic versioning, add point about migration guide
* drop other checklist link
* add to docs checklist
* update tests to make all 'test_qa_checks' tests pass locally. refactor to make them easier to understand (for me at least)
* add test files
* remove accidental file
* Source Google Analytics (Universal Analytics): specify date format in spec
* Update changelog
* Specify pattern in spec to pass CAT
* Update spec to support previous date format
* Leave validation only for YYYY-MM-DD dates, as the only valid option
