1
0
mirror of synced 2025-12-30 12:04:43 -05:00
Commit Graph

2869 Commits

Author SHA1 Message Date
Jonathan Pearlin
3eb222fc89 Source MongoDB V2: Clean fieldPath before use in projection (#28733)
* Clean fieldPath before use in projection

* Update release notes

* Automated Commit - Format and Process Resources Changes

* Remove prefix $ from field name

---------

Co-authored-by: jdpgrailsdev <jdpgrailsdev@users.noreply.github.com>
2023-07-26 12:11:46 -05:00
Baz
343a45c3b7 🐛 Source Shopify: improve error messages, refactored code (#28700) 2023-07-26 15:44:52 +03:00
Serhii Lazebnyi
77251e445a 🐛 Source Hubspot: Add backoff for handler request (#28705)
* Add backoff for handler request

* Fixed code formating and expected records
2023-07-26 14:06:58 +02:00
Catherine Noll
b95def8889 Source Stripe: remove duplicate streams (#28709) 2023-07-25 19:11:47 -04:00
Rodi Reich Zilberman
fb7cacb321 Pre release ctid (#28339)
* enable ctid feature flag for testing.
bump versions

* Removed PostgresSourceTest pre-ctid and turn CtidEnabledPostgresSourceTest back into PostgresJdbcSourceAcceptanceTest (#28341)

Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>

* Update PostgresSourceTest to no longer use Legacy AirbyteStateType

* Fix test.
Uncovered a bug in handling of money[] and float[]

* revert version change

* Orderly removal of ctid flag for cursor

* Handle correctly full refresh streams. (#28369)

* Handle correctly full refresh streams.
Handle xmin full refresh (currently blocked in ui).
remove unnecessary debug logs

* Test catalog with a mix of incremental and full refresh streams

* Filter out full refresh streams since they are not expected to be retuned by getIncrementalIterators()

* Filter out full refresh existing streams as well

* revert name to reflect incremental only streams

* 28505 xmin + ctid reclassify connection when ctid is not possible (#28548)

* reclassify to xmin stream we cannot read with ctid

* fix reclassify

* improve test

* Add initial load using xmin flow

* typo

* Added test for xmin initial load flow

* Improve testing for initial load with xmin flow

* sanity

* sanity

* bump versions and release notes

---------

Co-authored-by: Duy Nguyen <duy@airbyte.io>
2023-07-25 16:56:29 -05:00
Jonathan Pearlin
28bedf9675 Source MongoDB V2: Handle statistics collection failure gracefully (#28692)
* Fix bug introduced by statistics collection

* Fix syntax

* Formatting

* Fix failing test

* Fix failing test

* Update release notes
2023-07-25 15:22:02 -05:00
Serhii Lazebnyi
8f92798092 🐛 Source Zendesk Support: Add handle 404 error (#28397)
* Add handle 404 error

* Updated PR number

* Updated version
2023-07-25 19:20:56 +02:00
btkcodedev
66938e1e69 Source Stripe: New Stream - ShippingRates (#27217)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* Add stream ShippingRates

* Resolve conflicts, Bump version

* Fix unit tests

* Resolve conflict

* Fix unit_tests

* Remove unwanted files

* Run code formattingh

* Fix unit tests

* Fix unit_tests

* Fix unit_tests

* update expected records

* update expected records

* update expected records

* one more update for expected records

---------

Co-authored-by: Denys Davydov <davydov.den18@gmail.com>
2023-07-25 18:07:31 +03:00
Artem Inzhyyants
66b428bbda Source Zendesk Support: fix pagination for stream TicketMetricEvents (#28612)
* Source Zendesk Support: fix pagination for stream TicketMetricEvents

* Source Zendesk Support: update docs

* Source Zendesk Support: fix formatting

* Source Zendesk Support: add test for ticket_metric_events
2023-07-25 16:06:05 +02:00
Elliot T
9e1e198e18 🎉 New Source: Captain Data [Configuration based CDK] (#25230)
* initiate connector

* add doc

* fix schema

* add docs

* fix comments

* add logo

* add finished job filter

* suggested changes

* add metadata.yml

* add definition id

* update schemas

---------

Co-authored-by: Sajarin <sajarindider@gmail.com>
2023-07-25 08:03:09 -05:00
Efim Matytsin
d314dfd0ad 🐛 Source TikTok: fix default filters params - include deleted ad_groups, ad, campaigns to reports (#27910)
* SCNTALS-1949 fix filters for tiktok

* review fixes

* increment tiktok version in metadata file

* tiktok linter fix

* add changelog record

* small fix for changelog

* Added include_deleted param to TikTok source

* Update airbyte-integrations/connectors/source-tiktok-marketing/source_tiktok_marketing/streams.py

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* tiktok source: review fixes

* tiktok source, code format

---------

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2023-07-25 15:47:35 +03:00
Baz
335f99fd8d 🐛 📄 Source Shopify-OAuth: remove public docs (#28660) 2023-07-25 14:26:06 +03:00
Rodi Reich Zilberman
ff9fd07675 fix changelog table (#28649) 2023-07-24 19:53:17 -05:00
Marcos Marx
53aade6333 🎉 Source Amazon Seller Partner: Add GET_FLAT_FILE_ACTIONABLE_ORDER_DATA_SHIPPING and GET_ORDER_REPORT_DATA_SHIPPING streams (#28578)
* add new streams

* update acceptancte test config

* format files

* fix format and update docs

* remove for

* update expected records

* update expected records
2023-07-24 17:01:20 -03:00
Marcos Marx
6b588d04e2 Source Tiktok Marketing: add new fields to ad reports streams (#25423)
* add new fields to ad reports streams

* fix unit test

* fix test and check connection

* fix unit test

* add conversions too

* add additional fields

* fix: format python files with flake

* fix: add cta_conversion, vta_conversion to streams.py

* fix: python format

* fix: updated expected_records for ad_reports stream

---------

Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Augustin <augustin@airbyte.io>
2023-07-24 15:44:00 -04:00
Jonathan Pearlin
496678df5f Source MongoDB V2: Record statistics about MongoDB source connection (#28527)
* Record statistics about source database/collection

* Formatting

* Remove unnecessary log message

* Only record statistics during sync

* Emit structured Google log data

* Revert use of System.out.println

* Try using Log4j ObjectMessage

* Revert "Try using Log4j ObjectMessage"

This reverts commit d518c87aa8.

* Prepare release

* Update changelog

* Correct release version
2023-07-24 12:41:40 -05:00
Denys Davydov
ccac29af1b 🚨 🚨 Source Amazon Ads: fix schemas (#27868)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #1955 source amazon ads: fix stream schemas

* source amazon ads: upd changelog

* source amazon ads: upd CAT config

* upd backwards incompatible version

* update metadata with breakingChanges entry

* fix CATs

* fix md
2023-07-24 16:29:42 +03:00
Roman Yermilov [GL]
7496163998 Source Google Ads: fix timezone (#28510) 2023-07-24 17:21:02 +04:00
Artem Inzhyyants
a7b5a9c6b7 Source Zendesk Support: Fix cursor pagination (#28487)
* Source Zendesk Support: Remove extra page from params

* Source Zendesk Support: bump version

* Source Zendesk Support: Refactor + Fix pagination issues

* Source Zendesk Support: fix object count

* Source Zendesk Support: update timeout for tests

* Source Zendesk Support: update expected_records

* Source Zendesk Support: update expected_records

* Source Zendesk Support: update ignore pagination
2023-07-21 22:06:51 +02:00
Daryna Ishchenko
f6a294712d :bug:Source PostHog: fixed EventsSimpleRetriever declaration (#28461)
* fixed EventsSimpleRetriever declaration

* updated changelog
2023-07-21 14:32:57 +03:00
Roman Yermilov [GL]
a8a2ba3011 Source Mixpanel: fix start/end date format (#28511) 2023-07-21 13:47:05 +04:00
Baz
7845dec681 🎉 Source Shopify-OAuth: release for Cloud Review (#28507) 2023-07-20 22:27:53 +03:00
Sherif A. Nada
ec87474d8a Google Ads: Make custom report queries a multiline string (#28535) 2023-07-20 11:48:38 -07:00
Daryna Ishchenko
28bc5dc17f Source Hubspot: add new fields to schema (#28349)
* added unexpected fields

* Update schema in forms stream

* Update schema for email_events, email_subscriptions, engagements, campaigns streams

* Update changelog

* Update schema for contact_lists and forms streams, update expected records

* Update version, remove fix step from dockerfile

* Add fields to goals and forms schema, add some fields to ignored_fields

* Add missing type declarations

---------

Co-authored-by: Arsen Losenko <20901439+arsenlosenko@users.noreply.github.com>
2023-07-20 12:57:18 +03:00
Daryna Ishchenko
67381f8678 :bug:Source Iterable: fix type error (#28457)
* added __iter__ to StreamSlice

* updated change log
2023-07-20 10:58:28 +03:00
Ella Rohm-Ensing
841942b0e6 Ship a breaking change to faker (#28485)
* bump the metadata version

* add changelog and docker tag

* add breakingChanges metadata

* add migration guide

* resolve qa check failure becuase of missing header

* update changelog
2023-07-19 16:37:18 -06:00
Christo Grabowski
f7980b0ce8 📝 Docs: Source SFTP documentation update (#28316)
* complete first draft of updated sftp source page

* add cloud-specific note

* fix link to puttygen, small edits

* update regex link

* update prereqs
2023-07-19 16:28:04 -04:00
btkcodedev
d57456b022 📝Source MailGun: Update documentation (#28452)
* Added documentation for mailgun

* Update example
2023-07-19 15:00:50 -05:00
Arsen Losenko
88ce714116 Source Google Analytics (UA): specify date format in spec (#28149)
* Source Google Analytics (Universal Analytics): specify date format in spec

* Update changelog

* Specify pattern in spec to pass CAT

* Update spec to support previous date format

* Leave validation only for YYYY-MM-DD dates, as the only valid option
2023-07-19 21:59:31 +03:00
Joe Reuter
b6152d1fbc source-metabase: Update CDK to 0.47.0 (#28470)
Fix breaking CDK change in metabase connector
2023-07-19 18:18:40 +02:00
Denys Davydov
a83774add1 Source Stripe: pin API version (#28466)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* Source Stripe: pin source API version

* #27772 update changelog
2023-07-19 16:33:16 +03:00
Akash Kulkarni
4c54e34bd7 Add back full_refresh to xmin syncs (#28336)
* Add back full_refresh to xmin syncs

* Dockerfile bumps + fix tests
2023-07-18 18:46:50 -05:00
Christo Grabowski
25636ab942 📝 Docs: Source Notion docs update (#28377)
* update setup steps for notion connector

* expand setup instructions for integration

* edits to field explanations

* complete setup steps on main docs page

* complete inapp setup instructions

* fix deprecated link

* add extra space at end of inapp setup

* fix link to full docs page

* add information on integration capabilities
2023-07-18 14:30:07 -04:00
Denys Davydov
e55e09be8f 🐛 Source Google Ads: include today in the date range (#28365)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #2393 Source Google Ads: include today in the date range

* update changelog
2023-07-18 14:44:14 +03:00
Denys Davydov
7ae4f8686b 🐛 Source Mixpanel: reimplement backoff strategy (#27752)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* Add stream ShippingRates

* #2363 source mixpanel: reimplement backoff strategy

* #2363 source mixpanel: upd changelog

* #2363 source mixpanel: revert v0.1.32

* rework rate limits

* wait only when running CAT + increase timeouts

* #2363 return backoff time

* code format

* use env variable instead of a hidden field in the config

* review comments

* review fixes

* fix reading env var

* Resolve conflicts, Bump version

* Attempt to bump version to avoid caching

* split test config into to projects

* rollback version

* undo non related changes

* remove non related changes

* revert new line

* fix funnel slice patching

* fix funnel slice patching

* do not use stream state for generating request params!

* fix updating state

* Revert "do not use stream state for generating request params!"

This reverts commit 443781eaa9.

* revert prev commits

* fix timestamp filtering

---------

Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: maxi297 <maxime@airbyte.io>
2023-07-18 14:02:21 +03:00
Joe Reuter
b6531ac120 🐛 Source apify-dataset: Fix broken sync (#28290)
* fix connector

* fix more things

* prepare release

* cleanup

* format

* revert back to wrapper object

* format
2023-07-18 11:28:23 +02:00
Ben Church
6fa755f81d Java Pipeline Bump patch bump all java connectors in july (#28345)
* patch bump all java connectors in july

* Bump changelog
2023-07-14 21:01:39 -05:00
btkcodedev
0a2cd5f757 Source Stripe: New Stream - Persons (#26208)
* Initial Commit

* EOF

* Update airbyte-integrations/connectors/source-stripe/source_stripe/streams.py

Co-authored-by: Mal Hancock <mhancock@archangelic.space>

* Resolve Comments

* fix unittest

* fix SAT

* fix schema

---------

Co-authored-by: Mal Hancock <mhancock@archangelic.space>
Co-authored-by: midavadim <midavadim@yahoo.com>
2023-07-14 23:12:28 +03:00
Richa Rochna
f5954caf0a 🐛 Source Datadog - Additional stream and query details in response (#28089)
* Source Datadog - Adding request query in response

* Source Datadog - Documentation update

* Source Datadog - metadata.yaml update
2023-07-14 11:44:02 -04:00
Duy Nguyen
633c939d46 [Source-postgres] Set default cursor value for cdc mode (#27442)
* use LSN as default cursor for postgres CDC

* Fixed static constant

* Set lsn default cursor value for postgres sync

* Bumped metadata and dockerfile versions

* Disable acceptance backwards compatibility discovery test as this is a breaking change

---------

Co-authored-by: Conor <cpdeethree@users.noreply.github.com>
2023-07-13 11:12:34 -05:00
Denys Davydov
c816d14fc1 Source Google Ads: add new streams (#28246)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* add new streams

* upd changelog

* update CAT config

* update expected records
2023-07-13 16:53:08 +03:00
btkcodedev
c6c3ee11b8 Source Twilio: New streams UserConversations with parent Users (#27221)
* New stream UserConversations with parent Users

* Update Docs

* EOF streams.py

* formatting fixes

* Add account_sid as primary key

* bump version

* fix merge errors

* fix catalog and expected records

---------

Co-authored-by: Mal Hancock <mallory@archangelic.space>
Co-authored-by: Daryna Ishchenko <darina.ishchenko17@gmail.com>
2023-07-13 16:09:42 +03:00
Roman Yermilov [GL]
aea5a9ec8e Source Amazon Ads: fix Nonetype error when recordId is missing (#28155)
* Source Amazon Ads: fix Nonetype error when recordId is missing

* Source Amazon Ads: bump version and update changelog

* Source Amazon Ads: unittest for recordId generation
2023-07-13 15:10:41 +04:00
Artem Inzhyyants
a0e152aa55 🚨🚨 Source PayPal Transactions: Rename field in Balances schema (#27916)
* Source PayPal Transactions: update spec

* Source PayPal Transactions: update docs

* Source PayPal Transactions: update test config

* Source PayPal Transactions: update test config

* Source PayPal Transactions: update test config

* Source PayPal Transactions: update balances schema

* Source PayPal Transactions: update test config

* Source PayPal: bump version

---------

Co-authored-by: Augustin <augustin@airbyte.io>
2023-07-13 11:23:47 +02:00
Anatolii Yatsuk
507ad698f3 Source Monday: Add incremental sync for Items and Boards stream (#27944)
* Add incremental sync for the Boards and Items streams using Activity logs

---------

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2023-07-13 12:13:47 +03:00
pnavarromuttdata
9f12062a35 🎉 Source Facebook Marketing - add action report time enum to adinsights class (#26345)
* chore: bump version

* feat: new action_report_time spec into custom_insights stream

* feat: airbyte-config new action_report_time enum spec into custom_insights stream FBMarketing

* feat: new action_report_time tests

* fix: update allOf instead of array json spec

* chore: update docs source connector

* update pr id into version history doc

* update metadata.yaml

* adding action report time into cloud registry and oss registry

* fix formatting of python files

* fix: update type action_report_time

* fix: update title action report time

* fix: format python files with flake8

* fix: update version to minor version

* fix: remove action report time default value in test_get_custom_insights_stream

* fix: keep old config for some unit tests

* fix: put missing action breakdowns

* fix: remove default value variable and set into class argument directly

* chore: bump version

* Delete changes from cloud_registry

* Removed action_report_time from oss registry

---------

Co-authored-by: Juan Marchese <juan.marchese@muttdata.ai>
Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
2023-07-12 20:50:09 +02:00
Christo Grabowski
56a7f07a92 📝 Docs: Source S3 documentation update (#28229)
* add detailed setup steps for s3 bucket

* complete s3 setup steps

* compress versioned setup steps

* update S3 Provider Settings section

* update CSV and Parquet sections

* update file format settings section

* final edits/fixes

* maintain typecase of True/False

* Update docs/integrations/sources/s3.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update docs/integrations/sources/s3.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update docs/integrations/sources/s3.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update docs/integrations/sources/s3.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* add example to escape character field description

---------

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2023-07-12 14:27:58 -04:00
Roman Yermilov [GL]
6f89af01d7 Source Google Ads: disable logging (#28230)
* Source Google Ads: disable logging

* Source Google Ads: bump version, update changelog
2023-07-12 21:05:25 +04:00
btkcodedev
6792753b86 New Source: Gainsight-px [Low Code CDK] (#26998)
* Initial commit

* Update source_definitions.yaml

* Update source_specs.yaml

* change manifest

* fix manifest

* fix schema

* fix conflict

* fix conflict

* fix schema, refactor manifest, add new streams

* add streams to docs

* add stream users

* add new stream survey_response

* update docs

* EOF, formatting

* Add new stream, Update docs

* Add new streams

* Refactor Schemas

* Change gainsight to gainsight-px

* Add spec

* Change spec

* rewrite spec

* Fix Schemas, metadata

* Add pagination, fix schemas

* Add icon

* Update gainsight-px.md

* Update supportUrl to documentationUrl

Co-authored-by: Mal Hancock <mhancock@archangelic.space>

* Rename to icon.svg

* fix metedata.yaml

---------

Co-authored-by: Mal Hancock <mhancock@archangelic.space>
2023-07-12 09:48:20 -07:00
Arsen Losenko
ead6bcfbea 🐛 Source Twilio: update expected records (#28216)
* Source Twilio: update expected records

* Add channel_metadata property to stream conversation_messages
2023-07-12 19:27:47 +03:00