1
0
mirror of synced 2025-12-31 15:03:11 -05:00
Commit Graph

1202 Commits

Author SHA1 Message Date
Alexander Marquardt
e4d3d60ca8 🎉 New Source: Webflow (#13617)
* Added webflow code

* Updated readme

* Updated README

* Added webflow to source_definitions.yaml

* Enhanced documentation for the Webflow source connector

* Improved webflow source connector instructions

* Moved Site ID to before API token in Spec.yaml (for presentation in the UI)

* Addressed comments in PR.

* Changes to address requests in PR review

* Removed version from config

* Minor udpate to spec.yaml for clarity

* Updated to pass the accept-version as a constant rather than parameter

* Updated check_connection to hit the collections API that requires both site id and the authentication token.

* Fixed the test_check_connection to use the new check_connection function

* Added a streams test for generate_streams

* Re-named "autentication" object to "auth" to be more consistent with the way it is created by the CDK

* Added in an explict line to instantiante an "auth" object from WebflowTokenAuthenticator, to make it easier to describe in the blog

* Fixed a typo in a comment

* Renamed some classes to be more intuitive

* Renamed class to be more intuitive

* Minor change to an internal method name

* Made _get_collection_name_to_id_dict staticmethod

* Fixed a unit-test error that only appeared when running " python -m pytest -s unit_tests".
This was caused by Mocked settings from test_source.py leaking into test_streams.py

* format: add double quotes and remove unused import

* readme: remove semantic version naming of connector in build commands

* Updated spec.yaml

* auto-bump connector version

* format files

* add changelog

* update dockerfile

* auto-bump connector version

Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-06-22 08:44:27 -03:00
Tuhai Maksym
0d870bd37b 6339: error when attempting to use azure sql database within an elastic pool as source for cdc based replication (#13866)
* 6339: debug info

* 6339: not using 'USE' on Azure SQL servers

* 6339: cleanup

* 6339: cleanup2

* 6339: cleanup3

* 6339: versions/changelogs updated

* 6339: merge from master (consolidation issue)

* 6339: dev connector version (for testing in airbyte cloud)

* 6339: code review implementation

* 6339: apply formatting
2022-06-22 13:08:45 +03:00
Serhii Chvaliuk
f69a78c9a4 🎉 Source Github: break point added for workflows_runs stream (#13926)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-22 11:49:04 +03:00
Adam
a14fbda19f 📄 Postgres source: fix CDC setup order in docs (#13949)
* postgres source: fix CDC setup order docs

* Update docs/integrations/sources/postgres.md

Co-authored-by: Liren Tu <tuliren@gmail.com>
2022-06-21 11:52:06 -07:00
George Claireaux
e7dae06168 Enhance /publish to allow for multiple connectors and parallel execution (#13864)
* start

* revert

* azblob

* bq

* bq denorm

* megapublish baaaabyyyy

* fix needs

* matrix connectors

* auto-bump connector version

* dont failfast and max parallel 5

* multi runno

* minor

* testing matrix agents

* name

* testing multi agents

* tmp fix

* new multi agents

* multi test

* tryy

* let's do this

* magico

* fix

* label test

* couple more connector bumps

* temp

* things

* check this

* lets gooo

* more connectors

* Delete TEMP-testing-command.yml

* auto-bump connector version

* added comment describing bash part

* running single thread

* catch sentry cli

* auto-bump connector version

* destinations

* + snowflake

* saved

* auto-bump connector version

* auto-bump connector version

* java source bumps

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* remove twice-defined methods

* label things

* revert action

* using the new test action

* point at action

* wrong tag on action

* update pool label

* update to use new ec2-github-runner fork

* this needs to be more generic than publisher

* change publish to run on pool

* add comment about runner-pool usage

* updated publish command docs for multi & parallel connector runs

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* unbump failed publish versions

* missed dockerfiles

* remove failed docs

* mssql fix

* overhauled the git comment output

* bumping a test connector that should work

* slight order switcheroo

* output connectors properly in first message

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-21 18:40:33 +01:00
Amruta Ranade
c42576440d Update salesforce.md
Fixed broken link
2022-06-21 10:55:40 -04:00
Denis Davydov
e8a5c7fffb Source Marketo: process fail during creation of an export job (#13930)
* #9322 source Marketo: process fail during creation of an export job

* #9322 source marketo: upd changelog

* #9322 source marketo: fix unit test

* #9322 source marketo: fix SATs

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-21 16:15:03 +03:00
Anna Lvova
32b5ed736e 🐛 Source: Typeform - Update schema for Responses stream (#13935)
* Upd responses schema

* Upd docs

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-21 15:38:30 +03:00
Eugene
393397290f 🎉Source-mysql: aligned datatype test (#13945)
* [13607] source-mysql: aligned datatype tests for regular and CDC ways + added CHAR fix to CDC processing
2022-06-21 14:05:11 +03:00
George Claireaux
30c5d2d9a7 🎉 New Source Dockerhub (#13931)
* init

* implement working source + tests

* add docs

* add docs

* fix bad comments

* Update airbyte-integrations/connectors/source-dockerhub/acceptance-test-config.yml

* Update airbyte-integrations/connectors/source-dockerhub/Dockerfile

* Update airbyte-integrations/connectors/source-dockerhub/.dockerignore

* Apply suggestions from code review

* Update docs/integrations/sources/dockerhub.md

* Update airbyte-integrations/connectors/source-dockerhub/integration_tests/acceptance.py

Co-authored-by: George Claireaux <george@airbyte.io>

* address @Phlair's feedback

* address @Phlair's feedback

* each record is now a Docker image rather than response page

* format

* fix unit tests

* fix acceptance tests

* add icon, definition and generate seed spec

* add requests to requirements

Co-authored-by: sw-yx <shawnthe1@gmail.com>
2022-06-20 15:13:29 +01:00
George Claireaux
c5783aa799 connectors that published (#13932) 2022-06-20 12:42:56 +01:00
Andrii Leonets
8f602aee96 Source MsSql : incr ver to include changes #13854 (#13887)
* incr version

* put PR id

* docker ver
2022-06-20 11:54:40 +03:00
Ganpat Agarwal
b338014b83 🐛 Source Amazon Seller Partner: handle start date for financial stream (#13633)
* start and end date for finacial stream should not be more than 180 days apart

* improve unit tests

* make changes to start date for finance stream

* update tests

* lint changes

* update version to 0.2.22 for source-amazon-seller-partner
2022-06-18 17:27:04 +05:30
Amruta Ranade
37b7470139 Edited Amplitude, Mailchimp, and Zendesk Support docs (#13897) 2022-06-17 14:32:30 -04:00
Harshith Mullapudi
074adefcdc Harshith/test pr 12536 (#13874)
* 🎉 New Source: Firebolt (#1)

* fix: Boolean type cast

* test: Improve format testing and doc

* refactor: Move some db functionality

* docs: Adding types doc link in utils

* feat: Use future-proof Auth in SDK

* fix: integration tests are failing

* chore: update seed file

Co-authored-by: Petro Tiurin <93913847+ptiurin@users.noreply.github.com>
Co-authored-by: ptiurin <petro.tiurin@firebolt.io>
2022-06-17 17:47:08 +05:30
Denis Davydov
1d99b58eb6 Source Amplitude: try-catch BadZipFile exception (#13846)
* #272 oncall: source Amplitude - try-catch BadZipFile exception

* #272-oncall: upd changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-16 11:44:55 -07:00
Denis Davydov
723bf660d8 Source Hubspot - fix missing data issue (#13837)
* #278 oncall: source hubspot - fix missing data issue

* #278 oncall: upd changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-16 11:44:46 -07:00
Serhii Chvaliuk
62626ee690 🎉 Source Github: use GraphQL for pull_request_stats stream (#13763)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-16 20:18:49 +03:00
mutt-santiago-stachuk
ed5d83a44d source-facebook-marketing: 🎉 Add fields to the ads_set stream (#13623) 2022-06-16 18:00:27 +02:00
LiRen Tu
26f462327b 🎉 Postgres source: prepare for removing strict-encrypt variant (#13823)
* Bump postgres version

* Update doc

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-15 18:44:53 -07:00
Amruta Ranade
bc98dc0fd2 Edited Stripe, HubSpot, Intercom docs (#13810)
* Initial edits

* Updated the Stripe connector doc

* Updated the Stripe doc

* Updated HubSpot and Intercom docs

* Minor sidebar edit
2022-06-15 13:07:08 -04:00
Baz
330d32ecca 🐛 Source Zendesk Support: fixed 429 error for TicketMetrics stream (#13757) 2022-06-15 17:35:38 +03:00
Serhii Chvaliuk
dd8aa06664 🎉 Source Github: Fix API sorting, fix get_starting_point caching (#13707)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-15 16:50:42 +03:00
Bastien Gandouet
f015a9df07 🐛 Source Amplitude: Fix an infinite loop (#13638)
* Fix infinite loop when fetching Amplitude data

* Update changelog and version

* Address review

* fix: unit tests were failing

* chore: bump version in source definitions

* chore: update seed file

Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
2022-06-15 17:32:21 +05:30
LiRen Tu
6964fe2740 📝 Postgres source: add troubleshooting for postgres hot standby (#13784)
* Add troubleshooting section in postgres doc

* Update doc

* Add one more solution
2022-06-14 22:02:42 -07:00
Denis Davydov
81572a9cb2 Source Facebook marketing: fix not syncing any data issue (#13749)
* #253 oncall source fb marketing: fix  issue

* #253 oncall: upd changelog

* Update docs/integrations/sources/facebook-marketing.md

* auto-bump connector version

Co-authored-by: Evan Tahler <evan@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-14 15:28:19 -07:00
Charles
22b727c0ea Update Airbyte Protocol Docs (#13709) 2022-06-14 15:04:38 -07:00
VitaliiMaltsev
f5a6a28211 🐛 Postgres Source: fixed truncated precision if the value of the milliseconds or seconds is 0 (#13549)
* Postgres Source: fixed truncated precision if the value of the millisecond or second is 0

* check CI with 1.15.3 testcontainer

* check CI with 1.15.3 testcontainer

* returned latest version of testcontainer

* fixed checkstyle

* fixed checkstyle

* returned latest testcontainer version

* updated CHANGELOG

* bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-14 23:39:01 +03:00
Marcos Marx
669e6ed39d Source Salesforce: fix sync capped streams with more records than page size (#13658)
* change logic for counting records

* update doc

* correct unit test

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-14 09:08:38 -03:00
Greg Solovyev
2cd62002be Fallback to parsing datetime and time strings w/ and w/o timezones in case DateTimeParseException is thrown (#13745)
* Fall back to parsing w/ or w/o TZ if parsing a date or a time string fails
* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-13 23:28:46 -07:00
VitaliiMaltsev
a3ca3ab6b2 🐛 Postgres Source: fixed unsupported date-time datatypes during incremental sync (#13655)
* Postgres Source: fixed unsupposted date-time datatypes during incremental sync

* updated CHANGELOG

* add tests for incremental cursor check

* removed star import

* Postgres Source: fixed unsupposted date-time datatypes during incremental sync

* updated CHANGELOG

* add tests for incremental cursor check

* removed star import

* add timestamp datatype test

* Bump version in Dockerfile

* auto-bump connector version

Co-authored-by: grishick <greg@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-13 13:58:26 -07:00
Amruta Ranade
7ecfa13ee0 Updated BigQuery, Google Sheets, Facebook Marketing, Salesforce docs (#13717)
* initial changes

* Edited google Sheets doc

* More edits

* edited the intro and prereqs for BigQuery

* edited the data loading section

* more edits

* Grammatical edits

* Formatting edits
2022-06-13 14:25:41 -04:00
Luis Gomez
f4da234d18 source-notion: Updated docs as per new format (#13708) 2022-06-13 17:51:06 +02:00
Evan Tahler
edb74ec282 Faker Source emits times in proper ISO format (#13695)
* Faker Source emits times in ISO format

* fix product times

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-10 17:21:26 -07:00
Denis Davydov
5df20b7316 Source Hubspot: fix URI too long issue (#13691)
* #268: fix URI too long (HubSpot)

* #268 upd changelog

* #268 oncall: review fix

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-10 22:30:52 +03:00
Michael Nguyen
a30018b374 source-delighted: since parameter changed from unix to iso timestamp (#13439) 2022-06-10 16:07:14 +02:00
Evan Tahler
db67936111 Fix source-google-ads on M1 Macs by pinning protobuf==3.14 (#13624)
* Bump `source-google-ads` to build for both AMD and ARM

* pin protobuf==3.14

* update readme

* #263 oncall: bump google ads version 15.1.1, protobuf 3.20.0

* auto-bump connector version

Co-authored-by: Denys Davydov <davydov.den18@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-09 11:55:12 -07:00
Marcos Marx
fca7f0144e Source Facebook Marketing: fix tax_id to string canadian has letters (#13317)
* fix tax_id to string canadian has letters

* bump connector version

* auto-bump connector version

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-09 15:32:29 -03:00
wissevrowl
cc261de932 source Myhours: Udate API schema (#12964) 2022-06-09 14:42:13 +02:00
Denis Davydov
922bf46e8f Source Hubspot: fix for property_history stream (#13596)
* #259 on call: Source Hubspot - fix for the property_history stream which did not emit any records

* #259 Source Hubspot: upd changelog

* #259 oncall: hubspot review fixes

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-09 10:21:08 +03:00
Denis Davydov
60f5f4ce46 Source Google Ads: add missing dependency (#13618)
* Source Google Ads: add missing dependency

* Source Google Ads: upd changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-09 09:12:36 +03:00
Serhii Chvaliuk
9a4582ad7d 🐛 Source Github: enable caching only for parent streams (#13558)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-09 07:21:10 +03:00
Serhii Chvaliuk
38fa0fa8b3 🐛 Source Github: improve docs for GA (#13593)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-08 20:19:11 +03:00
Serhii Chvaliuk
5444317513 🐛 Source Github: fix different result from get_starting_point when reading by pages (#13531)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-08 20:06:06 +03:00
Baz
ec6a58c8a2 🐛 Source LinkedIn Ads: fixed issue with normalization on redshift (#13495) 2022-06-08 17:53:40 +03:00
LiRen Tu
fd2048da53 🎉 Postgres source: publish new version with JDBC fetch size v2 (#13576)
* Update postgres doc

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-07 19:01:50 -07:00
LiRen Tu
545a7a3eb6 🎉 JDBC source: adjust fetch size based on max memory and max row size (#13435)
* Switch to measure max row byte size

* Reduce fetch size change logs

* Update unit tests

* Determine jdbc buffer size based on max memory

* Bump postgres version

* Bump postgres version

* Bump mysql version

* Bump mssql version

* Format java code

* Increase hikari connection timeout

* Update data source default parameters

* auto-bump connector version

* Mark postgres 0.4.21 as not published

* Revert "Bump mysql version"

This reverts commit ad9135258c.

* Fix unit test

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-07 14:29:34 -07:00
Alexandre Girard
f1d79b201f Source Hubspot: report which scopes are missing to user (#13566)
* Report missing scopes

* simplify

* bump

* emit trace message

* remove print

* more careful error handling

* Add typing

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-07 14:10:03 -07:00
Ganpat Agarwal
1721827814 🎉 Source Amazon Seller Partner: add financial streams (#13364)
* add new schemas

* add catalog in integration tests

* add streams for ListFinancialEvents and ListFinancialEventGroups

* bump to version 0.2.21

* update docs

* add unit tests

* expand schema up to 2 levels for better normalisation

* fix unit test

* lint fix

* add missing attribute

* chore: bump version in source definitions

* chore: update seed file

Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
2022-06-07 18:26:51 +05:30
midavadim
0796b35e55 🎉 Source Stripe - to ga (#13449)
* Added incremental sync support (filtering by date - expired_at attr) for CheckourSessions stream and CheckoutSessionsLineItems
Added unittests

* updated docs

* updated connector version

* airbyte config

* auto-bump connector version

* updated connector version in config

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-07 01:55:09 +03:00