1
0
mirror of synced 2025-12-31 15:03:11 -05:00
Commit Graph

2745 Commits

Author SHA1 Message Date
Alexander Marquardt
e4d3d60ca8 🎉 New Source: Webflow (#13617)
* Added webflow code

* Updated readme

* Updated README

* Added webflow to source_definitions.yaml

* Enhanced documentation for the Webflow source connector

* Improved webflow source connector instructions

* Moved Site ID to before API token in Spec.yaml (for presentation in the UI)

* Addressed comments in PR.

* Changes to address requests in PR review

* Removed version from config

* Minor udpate to spec.yaml for clarity

* Updated to pass the accept-version as a constant rather than parameter

* Updated check_connection to hit the collections API that requires both site id and the authentication token.

* Fixed the test_check_connection to use the new check_connection function

* Added a streams test for generate_streams

* Re-named "autentication" object to "auth" to be more consistent with the way it is created by the CDK

* Added in an explict line to instantiante an "auth" object from WebflowTokenAuthenticator, to make it easier to describe in the blog

* Fixed a typo in a comment

* Renamed some classes to be more intuitive

* Renamed class to be more intuitive

* Minor change to an internal method name

* Made _get_collection_name_to_id_dict staticmethod

* Fixed a unit-test error that only appeared when running " python -m pytest -s unit_tests".
This was caused by Mocked settings from test_source.py leaking into test_streams.py

* format: add double quotes and remove unused import

* readme: remove semantic version naming of connector in build commands

* Updated spec.yaml

* auto-bump connector version

* format files

* add changelog

* update dockerfile

* auto-bump connector version

Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-06-22 08:44:27 -03:00
Tuhai Maksym
aa28d448d8 12708: Add an option to use encryption with staging in Redshift Destination (#13675)
* 12708: Add an option to use encryption with staging in Redshift Destination

* 12708: docs/docker configs updated

* 12708: merge with master

* 12708: merge fix

* 12708: code review implementation

* 12708: fix for older configs

* 12708: fix for older configs in check

* 12708: merge from master (consolidation issue)

* 12708: versions updated
2022-06-22 14:31:04 +03:00
Tuhai Maksym
0d870bd37b 6339: error when attempting to use azure sql database within an elastic pool as source for cdc based replication (#13866)
* 6339: debug info

* 6339: not using 'USE' on Azure SQL servers

* 6339: cleanup

* 6339: cleanup2

* 6339: cleanup3

* 6339: versions/changelogs updated

* 6339: merge from master (consolidation issue)

* 6339: dev connector version (for testing in airbyte cloud)

* 6339: code review implementation

* 6339: apply formatting
2022-06-22 13:08:45 +03:00
Serhii Chvaliuk
f69a78c9a4 🎉 Source Github: break point added for workflows_runs stream (#13926)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-22 11:49:04 +03:00
Octavia Squidington III
8c96a5e7f5 Bump Airbyte version from 0.39.22-alpha to 0.39.23-alpha (#13984)
Co-authored-by: pmossman <pmossman@users.noreply.github.com>
2022-06-21 12:46:58 -07:00
Adam
a14fbda19f 📄 Postgres source: fix CDC setup order in docs (#13949)
* postgres source: fix CDC setup order docs

* Update docs/integrations/sources/postgres.md

Co-authored-by: Liren Tu <tuliren@gmail.com>
2022-06-21 11:52:06 -07:00
Octavia Squidington III
5105c00a5d Bump Airbyte version from 0.39.21-alpha to 0.39.22-alpha (#13979)
Co-authored-by: Phlair <Phlair@users.noreply.github.com>
2022-06-21 19:37:05 +01:00
George Claireaux
e7dae06168 Enhance /publish to allow for multiple connectors and parallel execution (#13864)
* start

* revert

* azblob

* bq

* bq denorm

* megapublish baaaabyyyy

* fix needs

* matrix connectors

* auto-bump connector version

* dont failfast and max parallel 5

* multi runno

* minor

* testing matrix agents

* name

* testing multi agents

* tmp fix

* new multi agents

* multi test

* tryy

* let's do this

* magico

* fix

* label test

* couple more connector bumps

* temp

* things

* check this

* lets gooo

* more connectors

* Delete TEMP-testing-command.yml

* auto-bump connector version

* added comment describing bash part

* running single thread

* catch sentry cli

* auto-bump connector version

* destinations

* + snowflake

* saved

* auto-bump connector version

* auto-bump connector version

* java source bumps

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* remove twice-defined methods

* label things

* revert action

* using the new test action

* point at action

* wrong tag on action

* update pool label

* update to use new ec2-github-runner fork

* this needs to be more generic than publisher

* change publish to run on pool

* add comment about runner-pool usage

* updated publish command docs for multi & parallel connector runs

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* unbump failed publish versions

* missed dockerfiles

* remove failed docs

* mssql fix

* overhauled the git comment output

* bumping a test connector that should work

* slight order switcheroo

* output connectors properly in first message

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-21 18:40:33 +01:00
Amruta Ranade
c42576440d Update salesforce.md
Fixed broken link
2022-06-21 10:55:40 -04:00
Denis Davydov
e8a5c7fffb Source Marketo: process fail during creation of an export job (#13930)
* #9322 source Marketo: process fail during creation of an export job

* #9322 source marketo: upd changelog

* #9322 source marketo: fix unit test

* #9322 source marketo: fix SATs

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-21 16:15:03 +03:00
Anna Lvova
32b5ed736e 🐛 Source: Typeform - Update schema for Responses stream (#13935)
* Upd responses schema

* Upd docs

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-21 15:38:30 +03:00
Eugene
393397290f 🎉Source-mysql: aligned datatype test (#13945)
* [13607] source-mysql: aligned datatype tests for regular and CDC ways + added CHAR fix to CDC processing
2022-06-21 14:05:11 +03:00
Octavia Squidington III
689064e373 Bump Airbyte version from 0.39.20-alpha to 0.39.21-alpha (#13938)
Co-authored-by: alafanechere <alafanechere@users.noreply.github.com>
2022-06-20 18:04:15 +02:00
George Claireaux
30c5d2d9a7 🎉 New Source Dockerhub (#13931)
* init

* implement working source + tests

* add docs

* add docs

* fix bad comments

* Update airbyte-integrations/connectors/source-dockerhub/acceptance-test-config.yml

* Update airbyte-integrations/connectors/source-dockerhub/Dockerfile

* Update airbyte-integrations/connectors/source-dockerhub/.dockerignore

* Apply suggestions from code review

* Update docs/integrations/sources/dockerhub.md

* Update airbyte-integrations/connectors/source-dockerhub/integration_tests/acceptance.py

Co-authored-by: George Claireaux <george@airbyte.io>

* address @Phlair's feedback

* address @Phlair's feedback

* each record is now a Docker image rather than response page

* format

* fix unit tests

* fix acceptance tests

* add icon, definition and generate seed spec

* add requests to requirements

Co-authored-by: sw-yx <shawnthe1@gmail.com>
2022-06-20 15:13:29 +01:00
VitaliiMaltsev
c283d9d159 Deprecate PART_SIZE_MB in connectors using S3/GCS storage (#13753)
* Removed part_size from connectors that use StreamTransferManager

* fixed S3DestinationConfigTest

* fixed S3JsonlFormatConfigTest

* upadate changelog and bump version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* upadate changelog and bump version for Redshift and Snowflake destinations

* auto-bump connector version

* fix GCS staging test

* fix GCS staging test

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-20 14:56:33 +03:00
George Claireaux
c5783aa799 connectors that published (#13932) 2022-06-20 12:42:56 +01:00
Andrii Leonets
8f602aee96 Source MsSql : incr ver to include changes #13854 (#13887)
* incr version

* put PR id

* docker ver
2022-06-20 11:54:40 +03:00
Rajakavitha Kodhandapani
ec8e5bccf2 Docs: fixed the broken links (#13915) 2022-06-19 19:32:49 -03:00
Serhii Chvaliuk
49d181a198 Normalization: Fix incorrect jinja2 macro json_extract_array call (#13894)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-19 13:13:49 +03:00
Ganpat Agarwal
b338014b83 🐛 Source Amazon Seller Partner: handle start date for financial stream (#13633)
* start and end date for finacial stream should not be more than 180 days apart

* improve unit tests

* make changes to start date for finance stream

* update tests

* lint changes

* update version to 0.2.22 for source-amazon-seller-partner
2022-06-18 17:27:04 +05:30
Amruta Ranade
9c1fbea3dd deleting SUMMARY.md since we don't need it for docusaurus builds (#13901) 2022-06-17 15:58:07 -04:00
Amruta Ranade
37b7470139 Edited Amplitude, Mailchimp, and Zendesk Support docs (#13897) 2022-06-17 14:32:30 -04:00
Jonathan Pearlin
abe3024511 Update airbyte-protocol.md (#13892)
* Update airbyte-protocol.md

* Fix typo

* Fix prose

* Add protocol reviewers for protocol documentation

* Remove duplicate
2022-06-17 14:06:53 -04:00
Charles
e89846cba9 update api for per stream (#13835) 2022-06-17 10:59:41 -07:00
George Claireaux
e63458fabb Publishing java connectors with better formatted stacktrace in Trace messages (#13852)
* start

* revert

* azblob

* bq

* bq denorm

* megapublish baaaabyyyy

* fix needs

* matrix connectors

* auto-bump connector version

* dont failfast and max parallel 5

* multi runno

* minor

* testing matrix agents

* name

* testing multi agents

* tmp fix

* new multi agents

* multi test

* tryy

* let's do this

* magico

* fix

* label test

* couple more connector bumps

* temp

* things

* check this

* lets gooo

* more connectors

* Fix md and lg spacing scss vars (#13886)

* Delete TEMP-testing-command.yml

* auto-bump connector version

* Delete publish-command.yml

* added comment describing bash part

* running single thread

* catch sentry cli

* auto-bump connector version

* Update _variables.scss

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>
2022-06-17 16:15:24 +01:00
Harshith Mullapudi
074adefcdc Harshith/test pr 12536 (#13874)
* 🎉 New Source: Firebolt (#1)

* fix: Boolean type cast

* test: Improve format testing and doc

* refactor: Move some db functionality

* docs: Adding types doc link in utils

* feat: Use future-proof Auth in SDK

* fix: integration tests are failing

* chore: update seed file

Co-authored-by: Petro Tiurin <93913847+ptiurin@users.noreply.github.com>
Co-authored-by: ptiurin <petro.tiurin@firebolt.io>
2022-06-17 17:47:08 +05:30
Amruta Ranade
96a4d7515f Updated connector catalog (#13862) 2022-06-16 17:32:01 -04:00
Denis Davydov
1d99b58eb6 Source Amplitude: try-catch BadZipFile exception (#13846)
* #272 oncall: source Amplitude - try-catch BadZipFile exception

* #272-oncall: upd changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-16 11:44:55 -07:00
Denis Davydov
723bf660d8 Source Hubspot - fix missing data issue (#13837)
* #278 oncall: source hubspot - fix missing data issue

* #278 oncall: upd changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-16 11:44:46 -07:00
Serhii Chvaliuk
62626ee690 🎉 Source Github: use GraphQL for pull_request_stats stream (#13763)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-16 20:18:49 +03:00
mutt-santiago-stachuk
ed5d83a44d source-facebook-marketing: 🎉 Add fields to the ads_set stream (#13623) 2022-06-16 18:00:27 +02:00
Edward Gao
5f843415db 📚 update MSSQL docs to reflect that we support normalization (#13826) 2022-06-16 08:16:00 -07:00
LiRen Tu
26f462327b 🎉 Postgres source: prepare for removing strict-encrypt variant (#13823)
* Bump postgres version

* Update doc

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-15 18:44:53 -07:00
Octavia Squidington III
aed9244696 Bump Airbyte version from 0.39.19-alpha to 0.39.20-alpha (#13829)
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
2022-06-15 17:19:28 -07:00
Augustin
e8146e5ec2 Normalization: Upgrade MySQL to dbt 1.0.0 (#11470) 2022-06-15 15:05:49 -07:00
Octavia Squidington III
7d9f8c97c5 Bump Airbyte version from 0.39.18-alpha to 0.39.19-alpha (#13820)
Co-authored-by: edgao <edgao@users.noreply.github.com>
2022-06-15 13:14:50 -07:00
Alexander Marquardt
6a78aec852 Update 4-connection-checking.md (#13812)
* Update 4-connection-checking.md

As discussed in https://airbytehq-team.slack.com/archives/C02UQ9MJ4GG/p1655312684160609 - this example is misleading, as it does not actually connect to any API to validate connectivity.

* Update 4-connection-checking.md

Added a link to OneSignal check_connection, and move the "Note" to below the example
2022-06-15 21:41:15 +02:00
Amruta Ranade
fb0b5a3694 Fix broken link (#13815)
* fixing broken links

* docusaurus updates
2022-06-15 13:57:46 -04:00
Amruta Ranade
bc98dc0fd2 Edited Stripe, HubSpot, Intercom docs (#13810)
* Initial edits

* Updated the Stripe connector doc

* Updated the Stripe doc

* Updated HubSpot and Intercom docs

* Minor sidebar edit
2022-06-15 13:07:08 -04:00
Edward Gao
897522cf51 Add some dev-facing normalization docs (#13780) 2022-06-15 08:21:14 -07:00
Baz
330d32ecca 🐛 Source Zendesk Support: fixed 429 error for TicketMetrics stream (#13757) 2022-06-15 17:35:38 +03:00
Serhii Chvaliuk
dd8aa06664 🎉 Source Github: Fix API sorting, fix get_starting_point caching (#13707)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-15 16:50:42 +03:00
Bastien Gandouet
f015a9df07 🐛 Source Amplitude: Fix an infinite loop (#13638)
* Fix infinite loop when fetching Amplitude data

* Update changelog and version

* Address review

* fix: unit tests were failing

* chore: bump version in source definitions

* chore: update seed file

Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
2022-06-15 17:32:21 +05:30
LiRen Tu
6964fe2740 📝 Postgres source: add troubleshooting for postgres hot standby (#13784)
* Add troubleshooting section in postgres doc

* Update doc

* Add one more solution
2022-06-14 22:02:42 -07:00
Denis Davydov
81572a9cb2 Source Facebook marketing: fix not syncing any data issue (#13749)
* #253 oncall source fb marketing: fix  issue

* #253 oncall: upd changelog

* Update docs/integrations/sources/facebook-marketing.md

* auto-bump connector version

Co-authored-by: Evan Tahler <evan@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-14 15:28:19 -07:00
Charles
22b727c0ea Update Airbyte Protocol Docs (#13709) 2022-06-14 15:04:38 -07:00
Edward Gao
61ce03a436 🐛 Normalization correctly propagates deletions to the final tables (#12846) 2022-06-14 14:56:18 -07:00
Sophia Wiley
da60cd4f4a Update getting-started-with-airbyte-cloud.md (#13741)
* Update getting-started-with-airbyte-cloud.md

Edited Getting Started with Airbyte Cloud guide to match the updated Cloud UI.

* Update getting-started-with-airbyte-cloud.md

Updated based on Amruta's suggestions.
2022-06-14 17:32:04 -04:00
VitaliiMaltsev
f5a6a28211 🐛 Postgres Source: fixed truncated precision if the value of the milliseconds or seconds is 0 (#13549)
* Postgres Source: fixed truncated precision if the value of the millisecond or second is 0

* check CI with 1.15.3 testcontainer

* check CI with 1.15.3 testcontainer

* returned latest version of testcontainer

* fixed checkstyle

* fixed checkstyle

* returned latest testcontainer version

* updated CHANGELOG

* bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-14 23:39:01 +03:00
Octavia Squidington III
50f2a340d6 Bump Airbyte version from 0.39.17-alpha to 0.39.18-alpha (#13759)
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
Co-authored-by: Benoit Moriceau <benoit@airbyte.io>
2022-06-14 11:17:28 -07:00