1
0
mirror of synced 2026-01-07 09:05:45 -05:00
Commit Graph

5137 Commits

Author SHA1 Message Date
Arsen Losenko
0a9860bf9b 🐛 Source Pipedrive: update expected records (#27967) 2023-07-05 15:33:02 +03:00
Lake Mossman
810e4421ab Switch "request parameters" to "query parameters" in connector builder docs (#27896)
* Switch request parameters to query parameters in connector builder docsg

* fix more occurrences
2023-07-03 11:16:45 -07:00
Artem Inzhyyants
e387a5c0d6 Source Google Ads: handle Authorization Error (#27913)
* Source Google Ads: handle Authorization Error

* Source Google Ads: update docs

* Source Google Ads: bump version
2023-07-03 20:11:52 +02:00
Denys Davydov
e9fba26a99 🐛 Source Google Analytics Data API: limit custom report page size (#27909)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #2277 source google data analytics: limit page size of custom reports streams

* upd changelog

* limit page size only for check command
2023-07-03 20:19:00 +03:00
Denys Davydov
660d87e770 🐛 Source Google Ads: fix query template (#27835)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #2393 source google ads: fix query template

* source google ads: update changelog

* source google ads: add config and catalog to test click_view stream
2023-07-03 18:29:15 +03:00
Amélie
196fbbcf79 Update broken link in Meilisearch connector documentation (#27828) 2023-07-03 09:19:06 -05:00
Artem Inzhyyants
c734909b79 Source GitLab: add shared_runners_setting field to groups (#27869)
* Source GitLab: add `additionalProperties` to all schemas; add `shared_runners_setting` field to groups

* Source GitLab: update docs

* Source GitLab: remove additionalProperties

* Source GitLab: update docs

* Source GitLab: update expected records
2023-07-03 10:38:10 +02:00
Denys Davydov
a4e0f20bb2 🐛 Source Hubspot: use fully qualified name for custom objects (#27658)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #2294 Source Hubspot: use fully qualified name to access custom objects

* #27658 source hubspot: upd changelog

* #2294 source hubspot: review fixes

* source hubspot: test commit to reset ci cache

---------

Co-authored-by: Augustin <augustin@airbyte.io>
2023-07-03 02:27:43 -05:00
girarda
bdbe549a90 Bump Airbyte version from 0.50.5 to 0.50.6 2023-07-02 18:47:03 +00:00
Edward Gao
52b8cbe39d Revert "Destination Bigquery: Scaffolding for destinations v2 (#27268)" (#27891)
* Revert "Destination Bigquery: Scaffolding for destinations v2 (#27268)"

This reverts commit ba3e39bb0c.

* bump versions to 1.5.1 everywhere
2023-06-30 20:26:48 -04:00
Marcos Marx
62ee7026e6 Docs: update contributing guides (#27812)
* changes

* update how to pr and readme

* update contributeing guides

* update code of conduct text

* update submit new connector

* submit new connector doc

* make one line for submit new conn find request

* improve words
2023-06-30 15:00:15 -06:00
Marcos Marx
f27b8ba602 Collecting Metrics docs: improve words and add doc to sidebar (#27883)
* improve words and add doc to sidebar

* update words in collecting metrics docs'
2023-06-30 13:31:20 -06:00
Marcos Marx
4684b68334 add collecting metrics as paid feature doc (#27870) 2023-06-30 11:42:46 -06:00
Roman Yermilov [GL]
b51f285e2d Source Zendesk Support: fix missing headers (#27765)
* Source Zendesk Support: fix missing headers
2023-06-30 21:18:16 +04:00
Daryna Ishchenko
a08911e4ce Source Google Search Console: add new streams (#27831)
* Source Google Search Console: Add new streams

* added new streams

* updated changelog

* updated minor version

* updated SAT

---------

Co-authored-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
2023-06-30 14:55:21 +03:00
midavadim
957f21da43 :rotating_light:🚨 Source Hubspot - fixed incremental for engagement stream (#27161)
* fixed incremental for engagement stream

* added comments

* updated docs

* updated schema for email_events

* updated expected records and increased major connector version

* added backward_compatibility_tests_config

* added email_event record with value in browser.version field

* 🤖 Auto format source-hubspot code [skip ci]

* fixed expected records

* updated version if docker file

---------

Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
2023-06-29 18:12:29 -04:00
Maxime Carbonneau-Leclerc
e8ddda9709 🐛 Source Square: update following state management changes in the CDK (#27762)
* Update square stream_slices

* Fix unit tests

* Version bump and changelogs

* Update changelogs

* Format code

* Update CDK version to fix bug

* Fix unit tests

* Increase CDK version
2023-06-29 15:33:04 -04:00
Maxime Carbonneau-Leclerc
22f7445db6 🐛 Source Greenhouse: update following state management changes in the CDK (#27773)
* Update greenhouse custom component

* Force airbyte-cdk version

* Fix code, bump version and update changelogs

* Fix unit tests

* Fix tests

* Fix set_initial_state for GreenHouseSubstreamSlicer

* Reformat using black

* isort

* Update update_state to close_slice

* Simplify code and format

* Update considering fix

* Update CDK version to fix bug
2023-06-29 15:22:59 -04:00
Maxime Carbonneau-Leclerc
96aa72d3b2 🐛 Source Posthog: update following state management changes in the CDK (#27764)
* Update interface based on new Cursor interface

* Update version and changelogs

* Update unit tests

* Fix cursor assignation

* Update update_state to close_slice

* Update considering fix

* Update CDK version to fix bug
2023-06-29 15:03:25 -04:00
Maxime Carbonneau-Leclerc
baaabf4390 🐛 Source Zenloop: update CDK version to avoid bug introduced during data feed release (#27838)
* Update CDK version to fix bug

* Version bump
2023-06-29 14:45:17 -04:00
Evan Tahler
f455c1288d Java DB Destination connector licenses to Elv2 (#27781)
* Java DB Destination connector licenses to Elv2

* PR id for docs

* fix redshift tagging
2023-06-29 12:26:24 -05:00
Edward Gao
ba3e39bb0c Destination Bigquery: Scaffolding for destinations v2 (#27268)
* copy files from edgao branch

* start writing create table statement

* add basic unit test setup

* create a table, probably

* remove outdated todo

* derp, one more column

* ugh

* add partitioning+clustering

* use StringSubstitutor

* substitutions in updateTable

* wip generate update/insert statement

* split up into smaller methods

* handle json types correctly

* rename stuff

* more json_query vs _value stuff

* minor tweak

* super basic test setup

* laying foundation for type parsing

* more stuff

* tweaks

* more progress on type parsing

* fix json_value stuff?

* misc fixes in insert

* fix dedupFinalTable

* add testDedupRaw

* full e2e test

* type parsing: gave up and mirrored the dbt code structure to avoid bugs

* type parsing - more cleanup

* handle column name collisions

* handle tablename collisions...?

* comments

* remove original ns/name from quotedstream

* also javadoc

* remove redundant method

* fix table rename

* add incremental append test

* add full refresh append test

* comment

* call T+D sql in a reasonable location for standard inserts

* add config option

* use config option here

* type parsing - fix fromJsonSchema

* gate everything

* log query + runtime

* add spec option temporarily

* Raw Table Updates

* fix more stuff

* first big pass at toDialectType

* no quotes

* wrap everything in quotes

* resolve some TODOs

* log sql statement in tests

* overwriteFinalTable returns optional

* minor clean up

* add raw dataset override

* try to preserve the original namespace for t+d?

* write to the raw table correctly

* update todos

* write directly to raw table

this is kind of dumb because we're still trying to do tmp table operations,
and we still don't ack state until the end of the entire sync.

* standard inserts write to raw table correctly

* imports + log statements

* move logs + add comment

* explicitly create raw table

* move comment to better place

* Typing issues

* bash attempt

* formatting updates

* formatting updates

* write to the airbyte schema by default unless overriden by config options

* standard inserts truncate raw table at start of sync

* full refresh overwrite will overwrite correctly!

* fix avro record schema parsing

* better raw table recreate

* rename raw table to match standard inserts

* full refresh overwrite does tmp table things

* small clean up

* small clean up

* remove errors entry if no errors

* pull out destination config into singleton

* clean up singleton stuff

* make sure dest config exists when trying to do lookups

* avoid stringifying null

* quick thoughts on alter table

* add basic cdc testcase

* tweak cdc test setup

* rename raw table to match standard inserts

* minor tweak

* delete exact sql string assertions

* switch to JSON type

* minor cleanup

* sql whitespace changes

* explain cdc deletions

* GCS Staging Full Refresh create temp table

* assert schema

* first out of order cdc test

* add another cdc test case (currently failing)

* better test structure

* make this work

* oops, fix test

* stop trying to delete deletion records

* minor improvements to code+test

* enable concurrent test runs on integration test

* move stuff to static initializer

* extract utility method

* formatting

* Move conditional to the base java package, replace conditionals which did not use the typing and deduping flag but should have been.

* 🤖 Auto format destination-bigquery code [skip ci]

* 🤖 Auto format destination-gcs code [skip ci]

* switch back to empty list; write big assert

* minor wording tweaks

* 🤖 Auto format destination-bigquery code [skip ci]

* 🤖 Auto format destination-gcs code [skip ci]

* DestinationConfigTest

* 🤖 Auto format destination-bigquery code [skip ci]

* 🤖 Auto format destination-gcs code [skip ci]

* formatting

* remove ParsedType

* 🤖 Auto format destination-gcs code [skip ci]

* 🤖 Auto format destination-bigquery code [skip ci]

* tests verify every data type

* 🤖 Auto format destination-bigquery code [skip ci]

* 🤖 Auto format destination-gcs code [skip ci]

* full update with all data types

* 🤖 Auto format destination-bigquery code [skip ci]

* 🤖 Auto format destination-gcs code [skip ci]

* move stuff to new base lib

* 🤖 Auto format destination-gcs code [skip ci]

* Automated Commit - Formatting Changes

* 🤖 Auto format destination-bigquery code [skip ci]

* fix test

* 🤖 Auto format destination-bigquery code [skip ci]

* 🤖 Auto format destination-bigquery code [skip ci]

* 🤖 Auto format destination-gcs code [skip ci]

* asserts in dedupFinalTable

* better asserts in dedupRawTable

* [wip] test case for all data types

* 🤖 Auto format destination-gcs code [skip ci]

* 🤖 Auto format destination-bigquery code [skip ci]

* AirbyteTypeTest

* Automated Commit - Formatting Changes

* remove comments

* test chooseOneOf

* slightly better test output

* Automated Commit - Formatting Changes

* add some awful pretty print code

* more comment

* minor tweaks

* verify array/object type

* fix test

* handle deletions more correctly

* test toDialectType

* Destinations v2: better namespace handling (#27682)

* [wip] better namespace handling

* 🤖 Auto format destination-bigquery code [skip ci]

* wip also implement in gcs

* get gcs working (?)

* 🤖 Auto format destination-bigquery code [skip ci]

* remove duplicate method

* 🤖 Auto format destination-bigquery code [skip ci]

* fixed my code style settings

* make ci happy?

* 🤖 Auto format destination-bigquery code [skip ci]

* make ci happy?

* remove incorrect test

* blank line change

* initialize singleton

---------

Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>

* reset args correctly

* Automated Commit - Formatting Changes

* more bash stuff

* parse implicit structs

* initialize singleton in more tests

* Automated Commit - Formatting Changes

* I missed this namespace handling thing

* test more schemas

* fix singular types specified in arrays

* Automated Commit - Formatting Changes

* disable test for unimplemented feature

* initialize singleton

* remove spec options; changelogs+metadata

* randomize namespace

* also bump dockerfile

* unremove namespace sanitizing in legacy mode

* ... disable the correct test

* even more unit test fixes!

* move integration test to integration tests

---------

Co-authored-by: Cynthia Yin <cynthia@airbyte.io>
Co-authored-by: Joe Bell <joseph.bell@airbyte.io>
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: edgao <edgao@users.noreply.github.com>
Co-authored-by: cynthiaxyin <cynthiaxyin@users.noreply.github.com>
2023-06-29 08:44:37 -07:00
Maxime Carbonneau-Leclerc
074cddf690 🐛 Source Quickbooks: update following state management changes in the CDK (#27803)
* Update quickbooks following state management cleanup

* Update changelogs

* Update tests

* Ensure CDK version

* Update update_state to close_slice

* Update test to use partition end datetime

* Update considering fix

* renaming
2023-06-29 11:13:02 -04:00
Arsen Losenko
9ac7793b07 Source Typeform: Add OAuth support (#27240)
* Source Typeform: Add OAuth support

* Update spec to support OAuth

* Update spec structure and check implementation

* fix flake warnings

* Update spec.json

* Update unit tests

* Update spec and specify access_token path in check

* Keep structure for previous API token configs

* Remove backwards compatibility entries

* Get auth headers from authenticator, add forms_ids back to spec

* 🤖 Auto format source-typeform code [skip ci]

* Add major version bump

* Update unit tests

* Make "credentials" a required property

* Update version, update CAT config

* Skip previous version in spec and check tests

* Add OAuth information to docs, update setup description

* Update documentaion

* Make start_date optional

* Set start_date as optional in docs

* Apply suggestions from code review

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

---------

Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2023-06-29 17:46:52 +03:00
Maxime Carbonneau-Leclerc
9459b4ee1f 🐛 Source Zenloop: update following state management changes in the CDK (#27761)
* Update zenloop stream_slices

* Ensuring most recent CDK version

* Update version and changelogs

* Update changelogs

* Format

* flake8 cleanup
2023-06-29 08:45:55 -04:00
Artem Inzhyyants
c777ed8ab9 Source Close com: Update the Email Followup Tasks stream schema (#27776)
* Source Close: update expected_records

* Source Close: update schemas

* Source Close: bump version + docs update
2023-06-29 12:30:26 +02:00
Artem Inzhyyants
ad17938bc3 🚨🚨 Source MetaBase: Remove Activity Stream (#27777)
* Source MetaBase: update `activity` stream

* Source MetaBase: update docs

* Source MetaBase: Remove Activity Stream

* Source MetaBase: update test config

* Source MetaBase: bump version

* Source MetaBase: add allowedhosts

* Source MetaBase: remove activity stream refs
2023-06-29 12:30:09 +02:00
Denys Davydov
3d14fc721f 🐛 Source GA Data API: change page size when calling check() (#27718)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #2277 source GA: limit page size when calling check

* source GA: upd changelog

* code formatting

* #2277 review fixes
2023-06-29 10:52:04 +03:00
Denys Davydov
a9a98bc91d 🐛 Source Google Ads: refactor date slicing (#27711)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #2344 source google ads: refactor date slicing

* #2344 source google ads: upd changelog

* #2344 review fixes
2023-06-29 10:49:26 +03:00
Evan Tahler
9ce3cc6e79 Java DB Source connector licenses to Elv2 (#27737)
* Java DB Source connector licenses to Elv2

* Bump PR Ids
2023-06-28 18:52:02 -07:00
Evan Tahler
788a0ec7af Simplify faker offsets and limits (#27807) 2023-06-28 13:21:14 -07:00
Eric Schrock
620f8cb97f 🚨🚨 Source Okta: fix group members and resource sets streams (#27533)
---------

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2023-06-28 12:21:58 -07:00
Erica D'Souza
52040f9e98 Create getting-help.md (#27805)
* Create getting-help.md

* Adds support doc to sidebar.

renamed suport title

---------

Co-authored-by: Topher Lubaway <asimplechris@gmail.com>
2023-06-28 14:28:24 -04:00
Mal Hancock
9641d33773 Source Datadog: Multiple query streams and datadog site parameter (#27804)
* Source Datadog - Multiple query streams

* Source Datadog - Multiple query streams

* Source Datadog - Multiple query streams

* Source Datadog - Start date changes

* Source Datadog - Query date changes

* Source Datadog - Docs update

* Source Datadog - Review comment changes

* Source Datadog - Review comment changes

* formatting fixes

---------

Co-authored-by: richa-rochna <rochna.richa@gmail.com>
Co-authored-by: richar_backbase <richar@backbase.com>
Co-authored-by: Richa Rochna <99747757+richa-rochna@users.noreply.github.com>
2023-06-28 10:57:57 -07:00
Tim Roes
56c35d741b Fix Stripe title in menu (#27800) 2023-06-28 18:56:48 +02:00
Mal Hancock
e0be3945ef 🐛 Source Datadog: Fix schemas (#27784)
* update schemas and acceptance-test-config.yml

* disable incremental test

* bump version
2023-06-28 09:53:34 -07:00
Ben Church
5b183cbb7a Bnchrch/cat/backwards fail removed prop (#27685)
* Incorrect way to do this

* Working

* Make tests pretty

* Revert "Incorrect way to do this"

This reverts commit f8e29594c1b5fa07bad805806f2571af883d27fd.

* Add backwards compatibility docs

* bump version

* format

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
2023-06-28 08:21:27 -07:00
Evan Tahler
e03404bbd4 Google Sheets passes QA check (#27782)
* Google Sheets passes QA check

* bump PR id
2023-06-28 08:19:16 +03:00
Evan Tahler
4fb1f98221 Fix destination-s3 build (#27786)
* bump version

* PR id

* shh normalization, shh

* remove a bunch of arm64 deps?

* might as well match the dockerfile
2023-06-27 17:15:44 -07:00
Evan Tahler
fd34fe3e2b Google Sheets Destination connector licenses to Elv2 (#27780)
* Google Sheets Destination connector licenses to Elv2

* PR Id
2023-06-27 16:31:12 -05:00
Xiaohan Song
82b4961427 update docs for notification (#27733)
* update docs for notification

* Update docs/cloud/managing-airbyte-cloud/manage-airbyte-cloud-notifications.md

Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>

* Update docs/cloud/managing-airbyte-cloud/manage-airbyte-cloud-notifications.md

Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>

* Update docs/cloud/managing-airbyte-cloud/manage-airbyte-cloud-notifications.md

Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>

* Update docs/cloud/managing-airbyte-cloud/manage-airbyte-cloud-notifications.md

Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>

* Update docs/cloud/managing-airbyte-cloud/manage-airbyte-cloud-notifications.md

Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>

* Update docs/cloud/managing-airbyte-cloud/manage-airbyte-cloud-notifications.md

Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>

* Update docs/cloud/managing-airbyte-cloud/manage-airbyte-cloud-notifications.md

Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>

* Update docs/cloud/managing-airbyte-cloud/manage-airbyte-cloud-notifications.md

Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>

* use generic webhook

* rewording

---------

Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>
2023-06-27 13:55:17 -07:00
Evan Tahler
4dcb22d07a Stripe connector license to Elv2 (#27734)
* Stripe connector license to Elv2

* PR ID
2023-06-27 11:00:40 -05:00
Evan Tahler
62d9823cd8 Shopify connector license to Elv2 (#27732)
* Shopify connector license to Elv2

* PR id
2023-06-27 10:59:41 -05:00
Juan
14c0212d89 Update File source doc (#27763)
Included that AzBlob URL only needs the path to the file and not the base URL.
2023-06-27 11:46:21 -04:00
btkcodedev
569be8661d Source Twilio: New streams VerifyServices (#27231)
* Add new stream VerifyServices

* fix: format python files with flake8

* Override page_size

* find records with data_field

---------

Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: sh4sh <6833405+sh4sh@users.noreply.github.com>
2023-06-27 11:11:03 -04:00
btkcodedev
1716d2898c Source Zendesk Support: New Stream - Posts (#27156)
* Initial commit

* Update docs

* Update zendesk-support.md

* fix: add missing comma to unit test tuple

* Update unit_test.py

* 🤖 Auto format source-zendesk-support code [skip ci]

* increment expected_n_streams in unit tests

* increment one more thing for expected_n_streams in unit tests

* increment connector version

* fix unit test

---------

Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: sh4sh <6833405+sh4sh@users.noreply.github.com>
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
2023-06-27 11:10:26 -04:00
Artem Inzhyyants
16d7701742 Source Prestashop: Fix schema, remove empty datetime fields (#27716)
* Source Prestashop: update schema; remove empty datetime fields

* Source Prestashop: update expected_records

* Source Prestashop: update docs

* Source Prestashop: update schemas

* Source Prestashop: fix backward compatibility

* Source Prestashop: bump version

* Source Prestashop: update expected_records

* Source Prestashop: update test config

* Source Prestashop: update test config

* Source Prestashop: update expected records

* Source Prestashop: update expected_records
2023-06-27 17:09:03 +02:00
Denys Davydov
4da621f96a Source Zendesk Support: add stream Ticket Skips (#27436)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #16888 source zendesk support: add stream Ticket Skips

* source zendesk support: upd changelog

* fix typo after resolving merge conflict

* #16888 add expected recrods + set response_list_name

* update expected records

* upd expected records

* another expected records update
2023-06-27 15:15:10 +03:00
Denys Davydov
a7dc3af9b4 Source Zendesk Support: add Skills Based Routing streams (#27450)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #16887 source zendesk support: add skill based routing streams

* source endesk support: upd changelog

* #16887 source zendesk support: update expected records + schemas

* source zendesk: fix formatting+expected records

* zendesk support: upd schema + expected records

* zendesk support: update expected records

* upd expected records

* upd expected records

* upd tags in expected records

* another expected records update
2023-06-27 14:36:52 +03:00
Evan Tahler
037a522f8c Google Source connector licenses to Elv2 (#27738)
* Google Source connector licenses to Elv2

* bump PR Ids

* Update metadata.yaml
2023-06-27 03:38:13 -04:00