* enable ctid feature flag for testing.
bump versions
* Removed PostgresSourceTest pre-ctid and turn CtidEnabledPostgresSourceTest back into PostgresJdbcSourceAcceptanceTest (#28341)
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
* Update PostgresSourceTest to no longer use Legacy AirbyteStateType
* Fix test.
Uncovered a bug in handling of money[] and float[]
* revert version change
* Orderly removal of ctid flag for cursor
* Handle correctly full refresh streams. (#28369)
* Handle correctly full refresh streams.
Handle xmin full refresh (currently blocked in ui).
remove unnecessary debug logs
* Test catalog with a mix of incremental and full refresh streams
* Filter out full refresh streams since they are not expected to be retuned by getIncrementalIterators()
* Filter out full refresh existing streams as well
* revert name to reflect incremental only streams
* 28505 xmin + ctid reclassify connection when ctid is not possible (#28548)
* reclassify to xmin stream we cannot read with ctid
* fix reclassify
* improve test
* Add initial load using xmin flow
* typo
* Added test for xmin initial load flow
* Improve testing for initial load with xmin flow
* sanity
* sanity
* bump versions and release notes
---------
Co-authored-by: Duy Nguyen <duy@airbyte.io>
* add new streams
* update acceptancte test config
* format files
* fix format and update docs
* remove for
* update expected records
* update expected records
* add new fields to ad reports streams
* fix unit test
* fix test and check connection
* fix unit test
* add conversions too
* add additional fields
* fix: format python files with flake
* fix: add cta_conversion, vta_conversion to streams.py
* fix: python format
* fix: updated expected_records for ad_reports stream
---------
Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Augustin <augustin@airbyte.io>
* Record statistics about source database/collection
* Formatting
* Remove unnecessary log message
* Only record statistics during sync
* Emit structured Google log data
* Revert use of System.out.println
* Try using Log4j ObjectMessage
* Revert "Try using Log4j ObjectMessage"
This reverts commit d518c87aa8.
* Prepare release
* Update changelog
* Correct release version
* rename for clarity
* fix cleanup method
* giant commit because I'm irresponsible
* rename constant
* better raw table creation
* fix build?
* move code around
* tweaks
* more code shuffling
* Automated Commit - Format and Process Resources Changes
* add tests
* minor tweak
* remove unimportant methods
* cleanup
* Automated Commit - Format and Process Resources Changes
* derp
* clean up tests
* some more fixes post-merge
* botched merge
* create NoopTyperDeduper
* try and update everything to work?
* tweak comment
* move suffix args to end of list
* fix exception message
* Automated Commit - Format and Process Resources Changes
* add sqlgenerator test for softReset
* only prepare once
* update log message
* do what intellij says
* implement one more test
* less indirection
* Automated Commit - Format and Process Resources Changes
* rename test
* use noop in test
* version bump + changelog
* use stringutils
* fix typo
* flip if-statement
* typo
* simplify logic
* fix schema change logic
* typo
* use spy for clarity
* Automated Commit - Format and Process Resources Changes
* better test teardown
* slightly better logs
* fix exception message
* softReset returns single string
* Automated Commit - Format and Process Resources Changes
* simplify if chain
---------
Co-authored-by: edgao <edgao@users.noreply.github.com>
* Add ability to detect differences in expected Schemas and perform soft resets
* Remove alter table for overwrite syncs since its unneccessary
* Updates after testing
* pr reorganize
* comments
* add collection util test
* Add Tests
* bump version
* Automated Commit - Format and Process Resources Changes
* Destination BigQuery - Reduce amount of typing and deduping for GCS staging (#28489)
* undo comment out
* centralize t&d logic for staging and standard, add valve to staging
* Share more logic for typing and deduping
* Remove record checking logic and use only time for staging inserts
* Add Javadoc
* Automated Commit - Format and Process Resources Changes
---------
Co-authored-by: jbfbell <jbfbell@users.noreply.github.com>
* Change TableNotMigratedException to extend runtime exception, remove SqlGenerator interface method
* Make Lambda slightly more readable
* add test for validating v2 schemas
* change soft reset to single string
* convert back to list, update dockerfile
* remove needless default
---------
Co-authored-by: jbfbell <jbfbell@users.noreply.github.com>
* added unexpected fields
* Update schema in forms stream
* Update schema for email_events, email_subscriptions, engagements, campaigns streams
* Update changelog
* Update schema for contact_lists and forms streams, update expected records
* Update version, remove fix step from dockerfile
* Add fields to goals and forms schema, add some fields to ignored_fields
* Add missing type declarations
---------
Co-authored-by: Arsen Losenko <20901439+arsenlosenko@users.noreply.github.com>
* Source Google Analytics (Universal Analytics): specify date format in spec
* Update changelog
* Specify pattern in spec to pass CAT
* Update spec to support previous date format
* Leave validation only for YYYY-MM-DD dates, as the only valid option
* update setup steps for notion connector
* expand setup instructions for integration
* edits to field explanations
* complete setup steps on main docs page
* complete inapp setup instructions
* fix deprecated link
* add extra space at end of inapp setup
* fix link to full docs page
* add information on integration capabilities
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds
* #2393 Source Google Ads: include today in the date range
* update changelog
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds
* Add stream ShippingRates
* #2363 source mixpanel: reimplement backoff strategy
* #2363 source mixpanel: upd changelog
* #2363 source mixpanel: revert v0.1.32
* rework rate limits
* wait only when running CAT + increase timeouts
* #2363 return backoff time
* code format
* use env variable instead of a hidden field in the config
* review comments
* review fixes
* fix reading env var
* Resolve conflicts, Bump version
* Attempt to bump version to avoid caching
* split test config into to projects
* rollback version
* undo non related changes
* remove non related changes
* revert new line
* fix funnel slice patching
* fix funnel slice patching
* do not use stream state for generating request params!
* fix updating state
* Revert "do not use stream state for generating request params!"
This reverts commit 443781eaa9.
* revert prev commits
* fix timestamp filtering
---------
Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: maxi297 <maxime@airbyte.io>
* `destination-redshift` will fail syncs if records or properties are too large, rather than silently skipping records and succeding
* Bump version
* remove tests that don't matter any more
* more test removal
* more test removal
---------
Co-authored-by: Augustin <augustin@airbyte.io>
