jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2026-02-03 01:02:02 -05:00
Code Issues Packages Projects Releases Wiki Activity
9,254 Commits 5,552 Branches 518 Tags
90108ca44d2d7535ee26d99e296ecc4e29b706d2
Commit Graph

2462 Commits

Author SHA1 Message Date
Sajarin
90108ca44d Source CoinAPI - add quote stream to coin api (#20600) 
* feat: add quote stream to coin api

* fix: restore configured_catalog.json

* docs: updated changelog with quotes stream

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-05 11:40:13 -05:00
Greg Solovyev
606e59eb0d Common Jsons: add flag to apply flatten to arrays (#20993) 
* add flag to apply flatten to arrays

* add additional unit test cases for array flattening

* add backward compatibility function

* bump dest-redshift version and add changelog

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Adam Bloom <adam-bloom@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>
 2023-01-05 08:26:43 -08:00
Artem Inzhyyants
cb84383bfb Source Facebook Marketing: Remove extra properties (#21057) 
* Source Facebook Marketing: Remove extra properties

* Source Facebook Marketing: Docs update

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-05 16:04:13 +01:00
Denys Davydov
62d219fb9a Source google ads: retry transient errors (#20855) 
Co-authored-by: Baz <oleksandr.bazarnov@globallogic.com>
 2023-01-05 16:07:34 +02:00
Denys Davydov
4897bbc983 Source Salesforce: remove ActivityMetric stream (#20886) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2023-01-05 15:51:33 +02:00
Serhii Chvaliuk
2381403b34 Source Harvest: skip FORBIDDEN for EstimateItemCategories stream (#20810) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2023-01-05 12:49:24 +02:00
Simon Späti
26053ef727 📝 Add warning for working with local files and Docker Desktop (/tmp and /private) (#21004) 2023-01-05 10:54:58 +01:00
JJ2788
50916e7c2b New Desination: Heap Analytics [python cdk] (#20785) 
- implement a heap client to load data via the server-side API:
  https://developers.heap.io/reference/server-side-apis-overview
- the connector supports a generic data source, and the api_type
    determines the output schema. The output schema is dynamic.
- users pick the columns that will be loaded to the destination
- Consequently, each configured catalog only includes one stream
 2023-01-04 19:37:31 -08:00
Artem Inzhyyants
6176ecbb46 Source Salesforce: fix tests (#20927) 
* Source Salesforce: fix tests

* Source Salesforce: update expected records

* Source Salesforce: update expected records

* Source Salesforce: update docs; bump version

* Source Salesforce: update acceptance test reason

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-04 22:06:18 +01:00
Artem Inzhyyants
f6cc98fd6e Source Monday: fix schema loader; delete old files (#20996) 
* Source Monday: fix schema loader; delete old files

* Source Monday: fix schema loader; delete old files

* Source Monday: fix tests

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-04 20:36:24 +00:00
Artem Inzhyyants
10d7bea363 Source File: Fix special characters bug (#21012) 
* Source File: Fix special characters bug

* Source File: bump version; update docs

* Source File: update SSH credentials in unit tests

* Source File: bump version (secure)

* Source File: update resources manually
 2023-01-04 20:58:42 +01:00
Serhii Chvaliuk
ec995959f7 Source Amazon Ads: fix bug with handling: "Report date is too far in the past." (#20662) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2023-01-04 20:09:00 +02:00
Artem Inzhyyants
f5b793d37f Source Snapchat Marketing: permission error (#20865) 
* Source Snapchat Marketing: permission error

* Source Snapchat Marketing: add test; docs update

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-04 13:53:46 +01:00
Artem Inzhyyants
abf3ef8e0e Source Google Analytics UA: fix check connection (#20858) 
* Source Google Analytics UA: fix check connection

* Source Google Analytics UA: docs update

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-04 12:52:38 +00:00
Artem Inzhyyants
b2da81e836 Source Facebook Pages: fix tests; data is not permanent (#20925) 
* Source Facebook Pages: fix tests; data is not permanent

* Source Facebook Pages: fix tests
 2023-01-04 13:15:08 +01:00
Rodi Reich Zilberman
6a68d1f4a7 Map number(integer) into an integer rather than a float (#20730) 
* Fix failing test

* Bigquery-denormalized update spec plus destination_definitions.yaml to fix integration tests

* sanity

* Update version number and release note

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-04 15:27:25 +05:30
Serhii Chvaliuk
6c1869bbfb Source Jira: Fixed pagination for streams issue_remote_links, sprints (#20859) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2023-01-04 10:59:13 +02:00
Subodh Kant Chaturvedi
7dfc38d5bc source-postgres-cdc: allow LSN to be acknowledged after sync (#20469) 
* WIP

* fix issues

* fix test

* format

* more tests

* one more test

* make lsn_commit_behaviour not required

* update expected spec

* update expected spec

* update expected spec

* PR comments

* PR comments

* upgrade version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-04 14:23:17 +05:30
Artem Inzhyyants
598d4e3909 Source Facebook Marketing: fix OC 1040 (#20736) 
* Source Facebook Marketing: fix OC 1040

* Source Facebook Marketing: docs update; bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-04 01:01:01 +00:00
Artem Inzhyyants
e5392eebcf Source Sentry: add incremental sync (#20709) 
* Source Sentry: add incremental sync

* Source Sentry: fix test config

* Source Sentry: bump version; update docs

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-04 00:32:03 +01:00
Amruta Ranade
cae63965bd Deployment docs and sidebar cleanup (#20965) 2023-01-03 19:18:35 +05:30
Ryan Fu
64254a44f6 Destination Redshift: Introduces configurable value for file buffer count (#20879) 
* Increased default buffer count and introduces configurable value for destination redshift

* Updates logic to ensure filebuffers are at minimum the default number

* Reverted changes to increase the file buffer default but allow for users to increase with adequate warnings

* Bumps version number and updates changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-02 19:16:42 -08:00
Evan Tahler
2022f7d2a8 [faker] decouple stream state (#20492) 
* [faker] decouple stream state

* add PR #

* commit Stream instantiate changes

* fixup expected record

* skip backward test for this version too

* Apply suggestions from code review

Co-authored-by: Augustin <augustin@airbyte.io>

* lint

* Create realistic datasets of 10GB, 100GB, and 1TB in size (#20558)

* Faker CSV Streaming utilities

* readme

* don't do a final pipe to jq or you will run out or ram

* doc

* Faker gets 250% faster (#20741)

* Faker is 250% faster

* threads in spec + lint

* pass tests

* revert changes to record helper

* cleanup

* update expected_records

* bump default records-per-slice to 1k

* enforce unique email addresses

* cleanup

* more comments

* `parallelism` and pass tests

* update expected records

* cleanup notes

* update readme

* update expected records

* auto-bump connector version

Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-02 18:04:44 -06:00
Joe Reuter
2654e050dc Fix snowflake destination spec (#20566) 
Fix spec to specify a const member for each possible object in a oneOf clause
 2023-01-02 15:52:44 +01:00
Artem Inzhyyants
54377e4eec Source File: increase timeout OC 1197 (#20740) 
* Source File: increase timeout OC 1197

* Source File: docs update; bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-21 14:28:40 -06:00
Evan Tahler
a55eb7df2d Fix CI Dependency Check Failures (#20666) 
* pardot

* plaid

* quickbooks

* appfollow

* appstore

* cloudtrail

* clickup

* clockify

* coda

* Coinmarketcap

* cooper

* dixa

* dv-360

* exchange-rates

* file

* gridly

* Hellobaton

* kustomer

* mailersend

* microsoft dataverse

* n8n

* PersistIq

* Survey Sparrow

* Twilio Taskrouter

* YouTube Analytics Business

* Younium

* Yahoo Finance Price

* Yandex Metrica

* Xero

* WooCommerce

* XKCD

* Webflow

* US Census API

* Qonto

* Pivotal Tracker

* KVDB

* Firestore

* Ignore even more connectors

* test run

* SFTP JSON

* cleanup

* move pardot changelog

* update links

* remove testing HACK

* Update docs/integrations/sources/dixa.md

Co-authored-by: Augustin <augustin@airbyte.io>

* Update docs/integrations/sources/kustomer-singer.md

Co-authored-by: Augustin <augustin@airbyte.io>

* Update docs/integrations/sources/pardot.md

Co-authored-by: Augustin <augustin@airbyte.io>

* Update docs/integrations/sources/kustomer-singer.md

Co-authored-by: Augustin <augustin@airbyte.io>

* Update docs/integrations/sources/pardot.md

Co-authored-by: Augustin <augustin@airbyte.io>

Co-authored-by: Augustin <augustin@airbyte.io>
 2022-12-20 16:57:25 -06:00
Catherine Noll
ddab3ce820 Update source-gnews manifest to use inline stream schemas (#20405) 2022-12-19 10:32:55 -05:00
Brian Lai
9f0bfa2e5d [source-tiktok] Remove audience reports hourly streams (#20598) 
* get rid of audience report streams using hourly dimension

* bump version and docs changelog

* fix tests

* update backwards compatibility tests since some streams are no longer available

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
 2022-12-19 04:21:56 -08:00
Roman Yermilov [GL]
06e1717554 🎉 New Source Braze: new source implemented in low-code (#20520) 
* 🎉 New Source Braze: new source implemented in low-code

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-17 19:16:22 +04:00
Artem Inzhyyants
dca4cf24c7 Source Square: docs fix (#20618) 2022-12-17 12:37:29 +01:00
midavadim
f7fc223bca 🎉 Source Posthog - migration to low code, from alpha to beta (#18993) 
* added projects stream, added slicing for all streams (based on projects), annotations stream does not support incremental syncs anymore (due to API)

* added schema for projects streams

* source switched to low code

* added low coe yaml file

* added customized components

* code cleanup

* fix SAT

* added insights stream

* fix schema

* fix catalog

* fix yaml

* added schema

* added test

* updated SAT

* updated docs, bumped version

* update docs

* for events stream: added CartesianProductStreamSlicer, added paginator

* added custom slicer

* updated SAT

* fixed formatting

* added unit tests

* updated page_size

* fixed unit tests

* removed obsolete streams.py file

* updated seed file

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-16 21:18:39 +02:00
Artem Inzhyyants
697b91a8ad 🎉 Source Square: migrate to Beta (YAML) (#19369) 
* 🐛 Source Square: migrate to Beta (YAML)

* Source Square: update CDK, reorder properties

* Source Square: update docs

* Source Square: add tests; change release stage to beta

* Source Square: fix tests

* Source Square: fix tests formatting

* Source Square: fix tests formatting

* Source Square: Add token_expiry_date_format to oauth authenticator

* Source Square: ref component

* Source Square: Refactor Slicer; inherit from datetime slicer

* Source Square: test fix

* Source Square: publish low-code rc1

* Source Square: update version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-16 19:58:13 +01:00
Serhii Chvaliuk
a5bbbc3005 Source Jira: Improved code to become BETA (#20128) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-12-16 20:35:23 +02:00
Denys Davydov
dcda815569 Source Harvest: fix build (#20572) 
* #17506 source Harvest: fix build

* #17506 source harvest: upd changelog

* #17506 source harvest: introduce replication_end_date in spec

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-16 20:34:07 +02:00
Henri Blancke
305b1eb7ef 🎉 Source Retently: Add campaigns, feedback, outbox, templates streams (#19456) 
* [ADD] missing streams

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] increase request limit

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] update customers schema

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] formatting, fix unit and acceptance tests

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] bump version

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] feedback additional questions answer can be string

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] docs

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] companies pagination limit

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] add all streams to configured catalog

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] update sample catalog

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] add newline

Co-authored-by: Haithem SOUALA <haithem.souala@woopit.fr>

* fix: modify schema files to pass SAT

* Update configured_catalog.json

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>
Co-authored-by: Haithem SOUALA <haithem.souala@woopit.fr>
Co-authored-by: Sajarin <sajarindider@gmail.com>
 2022-12-16 13:18:48 -05:00
Artem Inzhyyants
2de1dca801 🎉Source WooCommerce: migrate to low-code (#19903) 
* Source WooCommerce: migrate to low-code

* Source WooCommerce: docs update

* Source WooCommerce: update release stage

* Source WooCommerce: add expected records

* Source WooCommerce: test fix

* Source WooCommerce: docs update

* Source WooCommerce: fix

* Source WooCommerce: fix; refactor

* Source WooCommerce: fix check stream

* Source WooCommerce: fix tests

* Source Square: test fix

* Source Square: test fix

* Source WooCommerce: fix schemas

* Source WooCommerce: fix schemas

* Source WooCommerce: fix schemas

* Source WooCommerce: fix tests

* Source WooCommerce: update expected records

* Source WooCommerce: fix tests

* Source WooCommerce: fix slice gap: hardcode interval

* Source WooCommerce: schemas formatting

* Source WooCommerce: schemas formatting

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-16 17:18:50 +01:00
Artem Inzhyyants
4e9c4c635a 🎉Source Facebook Pages: Migrate to Low-code (#19788) 
* Source Facebook Pages: Migrate to Low-code

* Source Facebook Pages: update docs

* Source Facebook Pages: update release stage

* Source Facebook Pages: docs update

* Source Facebook pages: revert source definitions

* Source Facebook Pages: update schema

* Source Facebook pages: disable discovery backward compatibility

* Source Facebook pages: fix expected records

* Source Facebook pages: fix expected records

* Source Facebook pages: fix tests

* Source Facebook pages: fix test

* Source Facebook pages: fix expected records

* Source Facebook pages: fix test

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-16 14:48:34 +01:00
Alexandre Girard
2a44db69d1 source gitlab: revert availabilitystrategy and bump to latest version of the cdk (#20542) 
* bump

* revert AvailabilityStrategy changes

* auto-bump connector version

Co-authored-by: erohmensing <erohmensing@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-15 14:42:45 -08:00
Ella Rohm-Ensing
bc5b72ddb4 Update bigcommerce CDK dependency (#20540) 
* Bump bigcommerce

* Update PR number

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-15 17:21:52 -05:00
Greg Solovyev
a12ca28d72 Do not swallow GCS failure when checking BigQuery config (#20501) 
* Do not swallow GCS failure when checking BigQuery config

* bump version of BigQuery denormalized

* add changelog

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-15 12:56:47 -08:00
Ella Rohm-Ensing
25900b7578 bump CDK dependency of snapchat-marketing (#20537) 
* Update snapchat marketing

* use correct pr number

* seriously why are PRs and Issues the same object

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-15 15:44:04 -05:00
Ella Rohm-Ensing
e4a323745c bump CDK dependency of metabase (#20535) 
* Update metabase

* Fix typo

* Use correct PR number

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-15 15:26:38 -05:00
Alexandre Girard
0e85a977fc bump pinterest (#20532) 
* bump

* reset

* update

* retroactively update changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-15 11:39:26 -08:00
Alexandre Girard
4279902a46 Source monday: build with latest cdk version (#20533) 
* build with latest cdk version

* update readme

* Update docs/integrations/sources/monday.md

Co-authored-by: Ella Rohm-Ensing <erohmensing@gmail.com>

Co-authored-by: Ella Rohm-Ensing <erohmensing@gmail.com>
 2022-12-15 11:31:55 -08:00
Ella Rohm-Ensing
8bb41282c5 CDK 0.15.0 and source-github 0.3.10 -- revert AvailabillityStrategy changes (#20523) 
* Revert "source-github: move known error handling to GithubAvailabilityStrategy (#19978)"

This reverts commit f97db17ccc.

* Revert "🐛 Python CDK: fix `StopIteration` error for `check_availability` (#20429)"

This reverts commit 4e9b014277.

* Revert "CDK: `AbstractSource.read()` skips syncing stream if its unavailable (add `AvailabilityStrategy` concept) (#19977)"

This reverts commit 55a32886a3.

* Restore changelog entries

* bump CDK version

* Bump Github version

* Re-add removed dependencies

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-15 14:24:24 -05:00
Amruta Ranade
c9e6cb1cab Remove IP addresses from postgres and alloydb docs (#20524) 2022-12-15 12:18:22 -05:00
Yevhen Sukhomud
1b8376eb79 🎉 Update destination-s3 to handle the new data types protocol (#20088) 
* Update base-java-s3 to support V1 protocol

* Fix formatting

* Azur Blob test fix

* Destination GCS test fix

* java-base-s3 correct test dataset

* Destination GCS test fix

* Refactor tests comparators from epoch

* Remove bignumber from V1 dataset

* Clean up

* Update json-avro-converter version

* Bump version + changelog

* Base64 type should be decoded for s3 tests

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-15 23:27:18 +07:00
Denys Davydov
9a62139cfb #14152 source Gitlab: certify to beta (#20511) 2022-12-15 16:31:28 +02:00
Artem Inzhyyants
6bf244846a Source Facebook Marketing: remove unsupported fields (#20402) 
* Source Facebook Marketing: remove unsupported fields

* Source Facebook Marketing: fix

* Source Facebook Marketing: refactor; raise error

* Source Facebook marketing: docs update

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-15 14:59:55 +01:00
Philip Corr
25b0e7ae42 🎉 Source bigcommerce: add order_products stream (#20090) 
* Add product info to the orders stream

* Update docs

* Refactor order_products into its own stream

* formatting

* Update docs and docker version

* remove duplicate field

* Allow another empty stream and bump abnormal state id

* auto-bump connector version

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-15 06:25:41 -05:00