1
0
mirror of synced 2026-01-05 21:02:13 -05:00
Commit Graph

4374 Commits

Author SHA1 Message Date
Edward Gao
9f6963ccfc Destination Bigquery 1s1t: handle cursor change (#28721)
* handle new cursor column

* sync2 is actualy weird, apparently

* logistics

* Automated Commit - Format and Process Resources Changes

---------

Co-authored-by: edgao <edgao@users.noreply.github.com>
2023-07-26 18:36:15 -06:00
Edward Gao
d9f2444c9e 1s1t: fix timestamps in T+D tests (#28749)
* fix timestamps in T+D tests

* lmao these tests have been wrong the entire time
2023-07-26 18:09:16 -05:00
Edward Gao
1aea368ce2 1s1t: move expected records into destination-bigquery (#28752) 2023-07-26 16:16:01 -06:00
Catherine Noll
dbb08ccfa5 Revert "pin stripe version in cloud to 3.13.0 (#28694)" (#28722)
This reverts commit 860e6af933.
2023-07-26 15:06:32 -04:00
Joe Reuter
94e8ef8b1e Langchain destination: Add Chroma support (#28605)
* add chroma support

* prepare release

* prepare release

* adjust docs

* adjust docs

* normalize metadata for Chroma
2023-07-26 19:29:15 +02:00
Jonathan Pearlin
3eb222fc89 Source MongoDB V2: Clean fieldPath before use in projection (#28733)
* Clean fieldPath before use in projection

* Update release notes

* Automated Commit - Format and Process Resources Changes

* Remove prefix $ from field name

---------

Co-authored-by: jdpgrailsdev <jdpgrailsdev@users.noreply.github.com>
2023-07-26 12:11:46 -05:00
Baz
343a45c3b7 🐛 Source Shopify: improve error messages, refactored code (#28700) 2023-07-26 15:44:52 +03:00
Serhii Lazebnyi
77251e445a 🐛 Source Hubspot: Add backoff for handler request (#28705)
* Add backoff for handler request

* Fixed code formating and expected records
2023-07-26 14:06:58 +02:00
Subodh Kant Chaturvedi
bcb5a4df8d mysql-cdc: implementation to construct initial debezium state (#28561)
* wip

* wip 2

* add tests + finalise implementation

* undo unwanted changes

* Automated Commit - Format and Process Resources Changes

* minor refactor

* add comment

* make method public

---------

Co-authored-by: subodh1810 <subodh1810@users.noreply.github.com>
2023-07-26 06:18:23 -05:00
Artem Inzhyyants
efd8bc2745 Source Zendesk Support: add to cloud (#28711) 2023-07-26 01:35:30 +02:00
Catherine Noll
b95def8889 Source Stripe: remove duplicate streams (#28709) 2023-07-25 19:11:47 -04:00
Edward Gao
a8b099c49e cython >.> (#28675) 2023-07-25 17:31:42 -05:00
Rodi Reich Zilberman
fb7cacb321 Pre release ctid (#28339)
* enable ctid feature flag for testing.
bump versions

* Removed PostgresSourceTest pre-ctid and turn CtidEnabledPostgresSourceTest back into PostgresJdbcSourceAcceptanceTest (#28341)

Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>

* Update PostgresSourceTest to no longer use Legacy AirbyteStateType

* Fix test.
Uncovered a bug in handling of money[] and float[]

* revert version change

* Orderly removal of ctid flag for cursor

* Handle correctly full refresh streams. (#28369)

* Handle correctly full refresh streams.
Handle xmin full refresh (currently blocked in ui).
remove unnecessary debug logs

* Test catalog with a mix of incremental and full refresh streams

* Filter out full refresh streams since they are not expected to be retuned by getIncrementalIterators()

* Filter out full refresh existing streams as well

* revert name to reflect incremental only streams

* 28505 xmin + ctid reclassify connection when ctid is not possible (#28548)

* reclassify to xmin stream we cannot read with ctid

* fix reclassify

* improve test

* Add initial load using xmin flow

* typo

* Added test for xmin initial load flow

* Improve testing for initial load with xmin flow

* sanity

* sanity

* bump versions and release notes

---------

Co-authored-by: Duy Nguyen <duy@airbyte.io>
2023-07-25 16:56:29 -05:00
Rodi Reich Zilberman
f4bf4a8292 Revert "source-postgres: enable ctid+cdc implementation (#28044)" (#28707)
This reverts commit 886ab6b09b.
2023-07-25 16:51:04 -05:00
Subodh Kant Chaturvedi
886ab6b09b source-postgres: enable ctid+cdc implementation (#28044)
* source-postgres: enable ctid+cdc implementation

* 100% ctid rollout for cdc
2023-07-25 16:31:27 -05:00
Jonathan Pearlin
28bedf9675 Source MongoDB V2: Handle statistics collection failure gracefully (#28692)
* Fix bug introduced by statistics collection

* Fix syntax

* Formatting

* Fix failing test

* Fix failing test

* Update release notes
2023-07-25 15:22:02 -05:00
Ella Rohm-Ensing
860e6af933 pin stripe version in cloud to 3.13.0 (#28694)
* pin stripe version in cloud to 3.13.0

* add ref
2023-07-25 15:18:41 -05:00
Serhii Lazebnyi
8f92798092 🐛 Source Zendesk Support: Add handle 404 error (#28397)
* Add handle 404 error

* Updated PR number

* Updated version
2023-07-25 19:20:56 +02:00
btkcodedev
66938e1e69 Source Stripe: New Stream - ShippingRates (#27217)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* Add stream ShippingRates

* Resolve conflicts, Bump version

* Fix unit tests

* Resolve conflict

* Fix unit_tests

* Remove unwanted files

* Run code formattingh

* Fix unit tests

* Fix unit_tests

* Fix unit_tests

* update expected records

* update expected records

* update expected records

* one more update for expected records

---------

Co-authored-by: Denys Davydov <davydov.den18@gmail.com>
2023-07-25 18:07:31 +03:00
Artem Inzhyyants
66b428bbda Source Zendesk Support: fix pagination for stream TicketMetricEvents (#28612)
* Source Zendesk Support: fix pagination for stream TicketMetricEvents

* Source Zendesk Support: update docs

* Source Zendesk Support: fix formatting

* Source Zendesk Support: add test for ticket_metric_events
2023-07-25 16:06:05 +02:00
Elliot T
9e1e198e18 🎉 New Source: Captain Data [Configuration based CDK] (#25230)
* initiate connector

* add doc

* fix schema

* add docs

* fix comments

* add logo

* add finished job filter

* suggested changes

* add metadata.yml

* add definition id

* update schemas

---------

Co-authored-by: Sajarin <sajarindider@gmail.com>
2023-07-25 08:03:09 -05:00
Efim Matytsin
d314dfd0ad 🐛 Source TikTok: fix default filters params - include deleted ad_groups, ad, campaigns to reports (#27910)
* SCNTALS-1949 fix filters for tiktok

* review fixes

* increment tiktok version in metadata file

* tiktok linter fix

* add changelog record

* small fix for changelog

* Added include_deleted param to TikTok source

* Update airbyte-integrations/connectors/source-tiktok-marketing/source_tiktok_marketing/streams.py

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* tiktok source: review fixes

* tiktok source, code format

---------

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2023-07-25 15:47:35 +03:00
Marcos Marx
53aade6333 🎉 Source Amazon Seller Partner: Add GET_FLAT_FILE_ACTIONABLE_ORDER_DATA_SHIPPING and GET_ORDER_REPORT_DATA_SHIPPING streams (#28578)
* add new streams

* update acceptancte test config

* format files

* fix format and update docs

* remove for

* update expected records

* update expected records
2023-07-24 17:01:20 -03:00
Edward Gao
df274b7f40 Destination snowflake: destinations v2 scaffolding (#28584)
* deps

* scaffolding

* logistics

* base-jdbc depends on base-td
2023-07-24 14:58:50 -05:00
Joe Bell
6159452f8f 🐛 Destination BigQuery - Limit Clustering Column Amount (#28625)
* Add limit to the number of clustering columns

* update bq version
2023-07-24 13:48:56 -06:00
Marcos Marx
6b588d04e2 Source Tiktok Marketing: add new fields to ad reports streams (#25423)
* add new fields to ad reports streams

* fix unit test

* fix test and check connection

* fix unit test

* add conversions too

* add additional fields

* fix: format python files with flake

* fix: add cta_conversion, vta_conversion to streams.py

* fix: python format

* fix: updated expected_records for ad_reports stream

---------

Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Augustin <augustin@airbyte.io>
2023-07-24 15:44:00 -04:00
Jonathan Pearlin
496678df5f Source MongoDB V2: Record statistics about MongoDB source connection (#28527)
* Record statistics about source database/collection

* Formatting

* Remove unnecessary log message

* Only record statistics during sync

* Emit structured Google log data

* Revert use of System.out.println

* Try using Log4j ObjectMessage

* Revert "Try using Log4j ObjectMessage"

This reverts commit d518c87aa8.

* Prepare release

* Update changelog

* Correct release version
2023-07-24 12:41:40 -05:00
Octavia Squidington III
b252a995d1 🤖 Add Gainsight Px connector to cloud (#28592)
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
2023-07-24 09:11:00 -07:00
Catherine Noll
7a18cc1535 Revert "Source Zendesk Support: remove version pin (#28587)" (#28611)
This reverts commit f402044dfb.
2023-07-24 08:37:33 -05:00
Denys Davydov
ccac29af1b 🚨 🚨 Source Amazon Ads: fix schemas (#27868)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #1955 source amazon ads: fix stream schemas

* source amazon ads: upd changelog

* source amazon ads: upd CAT config

* upd backwards incompatible version

* update metadata with breakingChanges entry

* fix CATs

* fix md
2023-07-24 16:29:42 +03:00
Roman Yermilov [GL]
7496163998 Source Google Ads: fix timezone (#28510) 2023-07-24 17:21:02 +04:00
Daryna Ishchenko
624bb815d0 increased timeout for incremental tests (#28559) 2023-07-24 14:05:03 +03:00
Daryna Ishchenko
a6cedb80cb increased timeout for discover test (#28560) 2023-07-24 14:03:44 +03:00
Daryna Ishchenko
d757c2ecf5 updated expected records (#28565) 2023-07-24 14:03:08 +03:00
Daryna Ishchenko
3f6b6eee8e updated expected records (#28595) 2023-07-24 14:01:57 +03:00
Daryna Ishchenko
36bb7aa0ee updated expected records (#28596) 2023-07-24 13:59:58 +03:00
Daryna Ishchenko
00ac8f44de fixed records comparison (#28597) 2023-07-24 13:58:59 +03:00
Artem Inzhyyants
f402044dfb Source Zendesk Support: remove version pin (#28587) 2023-07-24 12:13:50 +02:00
Edward Gao
adf8870de0 🐛 Destination bigquery 1s1t: respect dataset location (#28580)
* set dataset location during creation

* logistics
2023-07-21 16:54:45 -06:00
Davin Chia
b9a3c0817e 🐛 Release Snowflake Async State Bug. (#28581)
Release #28342 for Snowflake.
2023-07-21 15:38:54 -07:00
Artem Inzhyyants
a7b5a9c6b7 Source Zendesk Support: Fix cursor pagination (#28487)
* Source Zendesk Support: Remove extra page from params

* Source Zendesk Support: bump version

* Source Zendesk Support: Refactor + Fix pagination issues

* Source Zendesk Support: fix object count

* Source Zendesk Support: update timeout for tests

* Source Zendesk Support: update expected_records

* Source Zendesk Support: update expected_records

* Source Zendesk Support: update ignore pagination
2023-07-21 22:06:51 +02:00
Edward Gao
53da5baa7d Destination bigquery 1s1t: fix 1s1t schema change logic; extract TyperDeduper (#28490)
* rename for clarity

* fix cleanup method

* giant commit because I'm irresponsible

* rename constant

* better raw table creation

* fix build?

* move code around

* tweaks

* more code shuffling

* Automated Commit - Format and Process Resources Changes

* add tests

* minor tweak

* remove unimportant methods

* cleanup

* Automated Commit - Format and Process Resources Changes

* derp

* clean up tests

* some more fixes post-merge

* botched merge

* create NoopTyperDeduper

* try and update everything to work?

* tweak comment

* move suffix args to end of list

* fix exception message

* Automated Commit - Format and Process Resources Changes

* add sqlgenerator test for softReset

* only prepare once

* update log message

* do what intellij says

* implement one more test

* less indirection

* Automated Commit - Format and Process Resources Changes

* rename test

* use noop in test

* version bump + changelog

* use stringutils

* fix typo

* flip if-statement

* typo

* simplify logic

* fix schema change logic

* typo

* use spy for clarity

* Automated Commit - Format and Process Resources Changes

* better test teardown

* slightly better logs

* fix exception message

* softReset returns single string

* Automated Commit - Format and Process Resources Changes

* simplify if chain

---------

Co-authored-by: edgao <edgao@users.noreply.github.com>
2023-07-21 13:45:19 -06:00
Ben Church
da8596f907 Remove invalid public field from metadata.yaml (#28572) 2023-07-21 11:31:19 -06:00
Joe Reuter
6540fa7c91 Langchain destination: Support composite and nested primary keys for deduping (#28556)
* support composite primary keys for deduping

* prepare release

* format

* fix

---------

Co-authored-by: Augustin <augustin@airbyte.io>
2023-07-21 17:17:29 +02:00
Daryna Ishchenko
f6a294712d :bug:Source PostHog: fixed EventsSimpleRetriever declaration (#28461)
* fixed EventsSimpleRetriever declaration

* updated changelog
2023-07-21 14:32:57 +03:00
Daryna Ishchenko
b3d223e24e updated expected records (#28525) 2023-07-21 14:30:18 +03:00
Daryna Ishchenko
c8c03bd0be updated expected records (#28554) 2023-07-21 14:29:29 +03:00
Roman Yermilov [GL]
a8a2ba3011 Source Mixpanel: fix start/end date format (#28511) 2023-07-21 13:47:05 +04:00
Cynthia Yin
7e4797d90d Destinations V2: clean up AirbyteType code (#28430)
* general cleanup - move stuff around, add more comments

* guarantee `getAirbyteProtocolType` won't handle array values for `type`

* rename OneOf to Union

* simplify union ordering logic

* update testChooseUnion

* fix docs typos

* Automated Commit - Format and Process Resources Changes

* address comments

* Automated Commit - Format and Process Resources Changes

---------

Co-authored-by: cynthiaxyin <cynthiaxyin@users.noreply.github.com>
2023-07-20 14:59:52 -06:00
Edward Gao
225bfc4900 Destination Bigquery: improve test reliability by randomizing staging path in GCS tests (#28520) 2023-07-20 13:13:02 -07:00