1
0
mirror of synced 2025-12-25 02:09:19 -05:00

source freshdesk, hubspot, mixpanel, paypal-transaction, salesforce, zendesk-support: adding fixtures to mock time.sleep for connectors that explicitly sleep (#12335)

* adding fixtures to mock time.sleep for connectors that explicitly sleep

* bump connector versions

* update changelog doc for each connector

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* remove version bump for freshdesk because connector tests are in a bad state

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
This commit is contained in:
Brian Lai
2022-04-28 12:21:45 -07:00
committed by GitHub
parent 4a9f663565
commit bb2da42fde
27 changed files with 121 additions and 50 deletions

View File

@@ -473,7 +473,7 @@
- name: Mixpanel
sourceDefinitionId: 12928b32-bf0a-4f1e-964f-07e12e37153a
dockerRepository: airbyte/source-mixpanel
dockerImageTag: 0.1.12
dockerImageTag: 0.1.13
documentationUrl: https://docs.airbyte.io/integrations/sources/mixpanel
icon: mixpanel.svg
sourceType: api
@@ -563,7 +563,7 @@
- name: Paypal Transaction
sourceDefinitionId: d913b0f2-cc51-4e55-a44c-8ba1697b9239
dockerRepository: airbyte/source-paypal-transaction
dockerImageTag: 0.1.4
dockerImageTag: 0.1.5
documentationUrl: https://docs.airbyte.io/integrations/sources/paypal-transaction
icon: paypal.svg
sourceType: api
@@ -695,7 +695,7 @@
- name: Salesforce
sourceDefinitionId: b117307c-14b6-41aa-9422-947e34922962
dockerRepository: airbyte/source-salesforce
dockerImageTag: 1.0.5
dockerImageTag: 1.0.6
documentationUrl: https://docs.airbyte.io/integrations/sources/salesforce
icon: salesforce.svg
sourceType: api
@@ -862,7 +862,7 @@
- name: Zendesk Support
sourceDefinitionId: 79c1aa37-dae3-42ae-b333-d1c105477715
dockerRepository: airbyte/source-zendesk-support
dockerImageTag: 0.2.6
dockerImageTag: 0.2.7
documentationUrl: https://docs.airbyte.io/integrations/sources/zendesk-support
icon: zendesk.svg
sourceType: api

View File

@@ -5024,7 +5024,7 @@
path_in_connector_config:
- "credentials"
- "client_secret"
- dockerImage: "airbyte/source-mixpanel:0.1.12"
- dockerImage: "airbyte/source-mixpanel:0.1.13"
spec:
documentationUrl: "https://docs.airbyte.io/integrations/sources/mixpanel"
connectionSpecification:
@@ -6122,7 +6122,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-paypal-transaction:0.1.4"
- dockerImage: "airbyte/source-paypal-transaction:0.1.5"
spec:
documentationUrl: "https://docs.airbyte.io/integrations/sources/paypal-transactions"
connectionSpecification:
@@ -7346,7 +7346,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-salesforce:1.0.5"
- dockerImage: "airbyte/source-salesforce:1.0.6"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/salesforce"
connectionSpecification:
@@ -9328,7 +9328,7 @@
path_in_connector_config:
- "credentials"
- "client_secret"
- dockerImage: "airbyte/source-zendesk-support:0.2.6"
- dockerImage: "airbyte/source-zendesk-support:0.2.7"
spec:
documentationUrl: "https://docs.airbyte.io/integrations/sources/zendesk-support"
connectionSpecification:

View File

@@ -14,6 +14,7 @@ MAIN_REQUIREMENTS = [
TEST_REQUIREMENTS = [
"pytest==6.1.2",
"pytest-mock~=3.6",
"requests_mock==1.8.0",
"source-acceptance-test",
]

View File

@@ -5,11 +5,18 @@
from pathlib import Path
from pytest import fixture
from source_freshdesk.client import Client
HERE = Path(__file__).parent.absolute()
@fixture(autouse=True)
def time_sleep_mock(mocker):
time_mock = mocker.patch("time.sleep", lambda x: None)
yield time_mock
def test_client_backoff_on_limit_reached(requests_mock):
"""Error once, check that we retry and not fail"""
responses = [

View File

@@ -14,6 +14,7 @@ MAIN_REQUIREMENTS = [
TEST_REQUIREMENTS = [
"pytest==6.1.2",
"pytest-mock~=3.6",
"requests_mock==1.8.0",
"source-acceptance-test",
]

View File

@@ -29,6 +29,12 @@ NUMBER_OF_PROPERTIES = 2000
logger = logging.getLogger("test_client")
@pytest.fixture(autouse=True)
def time_sleep_mock(mocker):
time_mock = mocker.patch("time.sleep", lambda x: None)
yield time_mock
def test_check_connection_ok(requests_mock, config):
responses = [
{"json": [], "status_code": 200},

View File

@@ -33,6 +33,12 @@ from source_hubspot.streams import (
from .utils import read_full_refresh, read_incremental
@pytest.fixture(autouse=True)
def time_sleep_mock(mocker):
time_mock = mocker.patch("time.sleep", lambda x: None)
yield time_mock
def test_updated_at_field_non_exist_handler(requests_mock, common_params, fake_properties_list):
stream = ContactLists(**common_params)

View File

@@ -13,5 +13,5 @@ ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
LABEL io.airbyte.version=0.1.12
LABEL io.airbyte.version=0.1.13
LABEL io.airbyte.name=airbyte/source-mixpanel

View File

@@ -41,6 +41,12 @@ def patch_incremental_base_class(mocker):
mocker.patch.object(IncrementalMixpanelStream, "__abstractmethods__", set())
@pytest.fixture(autouse=True)
def time_sleep_mock(mocker):
time_mock = mocker.patch("time.sleep", lambda x: None)
yield time_mock
def test_url_base(patch_base_class):
stream = MixpanelStream(authenticator=MagicMock())

View File

@@ -12,5 +12,5 @@ RUN pip install .
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
LABEL io.airbyte.version=0.1.4
LABEL io.airbyte.version=0.1.5
LABEL io.airbyte.name=airbyte/source-paypal-transaction

View File

@@ -11,6 +11,7 @@ MAIN_REQUIREMENTS = [
TEST_REQUIREMENTS = [
"pytest~=6.1",
"pytest-mock~=3.6",
"source-acceptance-test",
]

View File

@@ -6,9 +6,16 @@ from datetime import datetime, timedelta
from airbyte_cdk.sources.streams.http.auth import NoAuth
from dateutil.parser import isoparse
from pytest import fixture
from source_paypal_transaction.source import Balances, PaypalTransactionStream, Transactions
@fixture(autouse=True)
def time_sleep_mock(mocker):
time_mock = mocker.patch("time.sleep", lambda x: None)
yield time_mock
def test_get_field():
record = {"a": {"b": {"c": "d"}}}

View File

@@ -13,5 +13,5 @@ RUN pip install .
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
LABEL io.airbyte.version=1.0.5
LABEL io.airbyte.version=1.0.6
LABEL io.airbyte.name=airbyte/source-salesforce

View File

@@ -7,7 +7,7 @@ from setuptools import find_packages, setup
MAIN_REQUIREMENTS = ["airbyte-cdk", "vcrpy==4.1.1", "pandas"]
TEST_REQUIREMENTS = ["pytest~=6.1", "requests_mock", "source-acceptance-test", "pytest-timeout"]
TEST_REQUIREMENTS = ["pytest~=6.1", "pytest-mock~=3.6", "requests_mock", "source-acceptance-test", "pytest-timeout"]
setup(
name="source_salesforce",

View File

@@ -485,6 +485,7 @@ class Describe(Stream):
Stream of sObjects' (Salesforce Objects) describe:
https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_sobject_describe.htm
"""
name = "Describe"
primary_key = "name"

View File

@@ -24,6 +24,12 @@ from source_salesforce.streams import (
)
@pytest.fixture(autouse=True)
def time_sleep_mock(mocker):
time_mock = mocker.patch("time.sleep", lambda x: None)
yield time_mock
def test_bulk_sync_creation_failed(stream_config, stream_api):
stream: BulkIncrementalSalesforceStream = generate_stream("Account", stream_config, stream_api)
with requests_mock.Mocker() as m:
@@ -482,7 +488,8 @@ def test_forwarding_sobject_options(stream_config, stream_names, catalog_stream_
"flag1": True,
"queryable": True,
}
for stream_name in stream_names if stream_name != "Describe"
for stream_name in stream_names
if stream_name != "Describe"
],
},
)

View File

@@ -9,6 +9,12 @@ from source_salesforce.api import DATE_TYPES, LOOSE_TYPES, NUMBER_TYPES, STRING_
from source_salesforce.exceptions import TypeSalesforceException
@pytest.fixture(autouse=True)
def time_sleep_mock(mocker):
time_mock = mocker.patch("time.sleep", lambda x: None)
yield time_mock
@pytest.mark.parametrize(
"streams_criteria,predicted_filtered_streams",
[

View File

@@ -11,6 +11,12 @@ from conftest import generate_stream
from source_salesforce.streams import BulkIncrementalSalesforceStream
@pytest.fixture(autouse=True)
def time_sleep_mock(mocker):
time_mock = mocker.patch("time.sleep", lambda x: None)
yield time_mock
@pytest.mark.parametrize(
"n_records, first_size, first_peak",
(

View File

@@ -25,5 +25,5 @@ COPY source_zendesk_support ./source_zendesk_support
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
LABEL io.airbyte.version=0.2.6
LABEL io.airbyte.version=0.2.7
LABEL io.airbyte.name=airbyte/source-zendesk-support

View File

@@ -7,7 +7,7 @@ from setuptools import find_packages, setup
MAIN_REQUIREMENTS = ["airbyte-cdk~=0.1.36", "pytz", "requests-futures~=1.0.0", "pendulum~=2.1.2"]
TEST_REQUIREMENTS = ["pytest~=6.1", "source-acceptance-test", "requests-mock==1.9.3"]
TEST_REQUIREMENTS = ["pytest~=6.1", "pytest-mock~=3.6", "source-acceptance-test", "requests-mock==1.9.3"]
setup(
version="0.1.0",

View File

@@ -21,6 +21,12 @@ STREAM_ARGS: dict = {
}
@pytest.fixture(autouse=True)
def time_sleep_mock(mocker):
time_mock = mocker.patch("time.sleep", lambda x: None)
yield time_mock
@pytest.mark.parametrize(
"records_count,page_size,expected_futures_deque_len",
[

View File

@@ -69,6 +69,12 @@ STREAM_RESPONSE: dict = {
TEST_STREAM = TicketComments(**STREAM_ARGS)
@pytest.fixture(autouse=True)
def time_sleep_mock(mocker):
time_mock = mocker.patch("time.sleep", lambda x: None)
yield time_mock
def test_str2datetime():
expected = datetime.strptime(DATETIME_STR, DATETIME_FORMAT)
output = BaseSourceZendeskSupportStream.str2datetime(DATETIME_STR)

View File

@@ -51,11 +51,11 @@ Please read [How to find your API key](https://support.freshdesk.com/support/sol
## Changelog
| Version | Date | Pull Request | Subject |
|:--------|:-----------| :--- |:-------------------------------------------------------------------------------|
| 0.2.11 | 2021-12-14 | [8682](https://github.com/airbytehq/airbyte/pull/8682) | Migrate to the CDK |
| 0.2.10 | 2021-12-06 | [8524](https://github.com/airbytehq/airbyte/pull/8524) | Update connector fields title/description |
| 0.2.9 | 2021-11-16 | [8017](https://github.com/airbytehq/airbyte/pull/8017) | Bugfix an issue that caused the connector not to sync more than 50000 contacts |
| 0.2.8 | 2021-10-28 | [7486](https://github.com/airbytehq/airbyte/pull/7486) | Include "requester" and "stats" fields in "tickets" stream |
| 0.2.7 | 2021-10-13 | [6442](https://github.com/airbytehq/airbyte/pull/6442) | Add start_date parameter to specification from which to start pulling data. |
| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:-------------------------------------------------------------------------------|
| 0.2.11 | 2021-12-14 | [8682](https://github.com/airbytehq/airbyte/pull/8682) | Migrate to the CDK |
| 0.2.10 | 2021-12-06 | [8524](https://github.com/airbytehq/airbyte/pull/8524) | Update connector fields title/description |
| 0.2.9 | 2021-11-16 | [8017](https://github.com/airbytehq/airbyte/pull/8017) | Bugfix an issue that caused the connector not to sync more than 50000 contacts |
| 0.2.8 | 2021-10-28 | [7486](https://github.com/airbytehq/airbyte/pull/7486) | Include "requester" and "stats" fields in "tickets" stream |
| 0.2.7 | 2021-10-13 | [6442](https://github.com/airbytehq/airbyte/pull/6442) | Add start_date parameter to specification from which to start pulling data. |

View File

@@ -59,6 +59,7 @@ Select the correct region \(EU or US\) for your Mixpanel project. See detail [he
| Version | Date | Pull Request | Subject |
|:---------|:-----------|:---------------------------------------------------------|:-----------------------------------------------------------------------------------------------------|
| `0.1.13` | 2022-04-27 | [12335](https://github.com/airbytehq/airbyte/pull/12335) | Adding fixtures to mock time.sleep for connectors that explicitly sleep |
| `0.1.12` | 2022-03-31 | [11633](https://github.com/airbytehq/airbyte/pull/11633) | Increase unit test coverage |
| `0.1.11` | 2022-04-04 | [11318](https://github.com/airbytehq/airbyte/pull/11318) | Change Response Reading |
| `0.1.10` | 2022-03-31 | [11227](https://github.com/airbytehq/airbyte/pull/11227) | Fix cohort id always null in the cohort_members stream |

View File

@@ -55,11 +55,12 @@ Transactions sync is performed with default `stream_slice_period` = 1 day, it me
## Changelog
| Version | Date | Pull Request | Subject |
| :--- | :--- | :--- | :--- |
| 0.1.4 | 2021-12-22 | [9034](https://github.com/airbytehq/airbyte/pull/9034) | Update connector fields title/description |
| 0.1.3 | 2021-12-16 | [8580](https://github.com/airbytehq/airbyte/pull/8580) | Added more logs during `check connection` stage |
| 0.1.2 | 2021-11-08 | [7499](https://github.com/airbytehq/airbyte/pull/7499) | Remove base-python dependencies |
| 0.1.1 | 2021-08-03 | [5155](https://github.com/airbytehq/airbyte/pull/5155) | fix start\_date\_min limit |
| 0.1.0 | 2021-06-10 | [4240](https://github.com/airbytehq/airbyte/pull/4240) | PayPal Transaction Search API |
| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:------------------------------------------------------------------------|
| 0.1.5 | 2022-04-27 | [12335](https://github.com/airbytehq/airbyte/pull/12335) | Adding fixtures to mock time.sleep for connectors that explicitly sleep |
| 0.1.4 | 2021-12-22 | [9034](https://github.com/airbytehq/airbyte/pull/9034) | Update connector fields title/description |
| 0.1.3 | 2021-12-16 | [8580](https://github.com/airbytehq/airbyte/pull/8580) | Added more logs during `check connection` stage |
| 0.1.2 | 2021-11-08 | [7499](https://github.com/airbytehq/airbyte/pull/7499) | Remove base-python dependencies |
| 0.1.1 | 2021-08-03 | [5155](https://github.com/airbytehq/airbyte/pull/5155) | fix start\_date\_min limit |
| 0.1.0 | 2021-06-10 | [4240](https://github.com/airbytehq/airbyte/pull/4240) | PayPal Transaction Search API |

View File

@@ -121,7 +121,8 @@ Now that you have set up the Salesforce source connector, check out the followin
## Changelog
| Version | Date | Pull Request | Subject |
|:--------|:-----------| :--- |:---------------------------------------------------------------------------------------------------------------------------------|
|:--------|:-----------|:---|:---------------------------------------------------------------------------------------------------------------------------------|
| 1.0.4 | 2022-04-27 | [12335](https://github.com/airbytehq/airbyte/pull/12335) | Adding fixtures to mock time.sleep for connectors that explicitly sleep |
| 1.0.3 | 2022-04-04 | [11692](https://github.com/airbytehq/airbyte/pull/11692) | Optimised memory usage for `BULK` API calls |
| 1.0.2 | 2022-03-01 | [10751](https://github.com/airbytehq/airbyte/pull/10751) | Fix broken link anchor in connector configuration |
| 1.0.1 | 2022-02-27 | [10679](https://github.com/airbytehq/airbyte/pull/10679) | Reorganize input parameter order on the UI |

View File

@@ -72,24 +72,25 @@ The Zendesk connector should not run into Zendesk API limitations under normal u
### CHANGELOG
| Version | Date | Pull Request | Subject |
|:---------|:-----------| :----- |:-------------------------------------------------------|
| Version | Date | Pull Request | Subject |
|:---------|:-----------|:---------------------------------------------------------|:------------------------------------------------------------------------|
| `0.2.7` | 2022-04-27 | [12335](https://github.com/airbytehq/airbyte/pull/12335) | Adding fixtures to mock time.sleep for connectors that explicitly sleep |
| `0.2.6` | 2022-04-19 | [12122](https://github.com/airbytehq/airbyte/pull/12122) | Fixed the bug when only 100,000 Users are synced [11895](https://github.com/airbytehq/airbyte/issues/11895) and fixed bug when `start_date` is not used on user stream [12059](https://github.com/airbytehq/airbyte/issues/12059).
| `0.2.5` | 2022-04-05 | [11727](https://github.com/airbytehq/airbyte/pull/11727) | Fixed the bug when state was not parsed correctly
| `0.2.4` | 2022-04-04 | [11688](https://github.com/airbytehq/airbyte/pull/11688) | Small documentation corrections
| `0.2.3` | 2022-03-23 | [11349](https://github.com/airbytehq/airbyte/pull/11349) | Fixed the bug when Tickets stream didn't return deleted records
| `0.2.2` | 2022-03-17 | [11237](https://github.com/airbytehq/airbyte/pull/11237) | Fixed the bug when TicketComments stream didn't return all records
| `0.2.1` | 2022-03-15 | [11162](https://github.com/airbytehq/airbyte/pull/11162) | Added support of OAuth2.0 authentication method
| `0.2.0` | 2022-03-01 | [9456](https://github.com/airbytehq/airbyte/pull/9456) | Update source to use future requests |
| `0.1.12` | 2022-01-25 | [9785](https://github.com/airbytehq/airbyte/pull/9785) | Add additional log messages |
| `0.1.11` | 2021-12-21 | [8987](https://github.com/airbytehq/airbyte/pull/8987) | Update connector fields title/description |
| `0.1.9` | 2021-12-16 | [8616](https://github.com/airbytehq/airbyte/pull/8616) | Adds Brands, CustomRoles and Schedules streams |
| `0.1.8` | 2021-11-23 | [8050](https://github.com/airbytehq/airbyte/pull/8168) | Adds TicketMetricEvents stream |
| `0.1.7` | 2021-11-23 | [8058](https://github.com/airbytehq/airbyte/pull/8058) | Added support of AccessToken authentication |
| `0.1.6` | 2021-11-18 | [8050](https://github.com/airbytehq/airbyte/pull/8050) | Fix wrong types for schemas, add TypeTransformer |
| `0.1.5` | 2021-10-26 | [7679](https://github.com/airbytehq/airbyte/pull/7679) | Add ticket_id and ticket_comments |
| `0.1.4` | 2021-10-26 | [7377](https://github.com/airbytehq/airbyte/pull/7377) | Fix initially_assigned_at type in ticket metrics |
| `0.1.3` | 2021-10-17 | [7097](https://github.com/airbytehq/airbyte/pull/7097) | Corrected the connector's specification |
| `0.1.2` | 2021-10-16 | [6513](https://github.com/airbytehq/airbyte/pull/6513) | Fixed TicketComments stream |
| `0.1.1` | 2021-09-02 | [5787](https://github.com/airbytehq/airbyte/pull/5787) | Fixed incremental logic for the ticket_comments stream |
| `0.1.0` | 2021-07-21 | [4861](https://github.com/airbytehq/airbyte/pull/4861) | Created CDK native zendesk connector |
| `0.2.5` | 2022-04-05 | [11727](https://github.com/airbytehq/airbyte/pull/11727) | Fixed the bug when state was not parsed correctly |
| `0.2.4` | 2022-04-04 | [11688](https://github.com/airbytehq/airbyte/pull/11688) | Small documentation corrections |
| `0.2.3` | 2022-03-23 | [11349](https://github.com/airbytehq/airbyte/pull/11349) | Fixed the bug when Tickets stream didn't return deleted records |
| `0.2.2` | 2022-03-17 | [11237](https://github.com/airbytehq/airbyte/pull/11237) | Fixed the bug when TicketComments stream didn't return all records |
| `0.2.1` | 2022-03-15 | [11162](https://github.com/airbytehq/airbyte/pull/11162) | Added support of OAuth2.0 authentication method |
| `0.2.0` | 2022-03-01 | [9456](https://github.com/airbytehq/airbyte/pull/9456) | Update source to use future requests |
| `0.1.12` | 2022-01-25 | [9785](https://github.com/airbytehq/airbyte/pull/9785) | Add additional log messages |
| `0.1.11` | 2021-12-21 | [8987](https://github.com/airbytehq/airbyte/pull/8987) | Update connector fields title/description |
| `0.1.9` | 2021-12-16 | [8616](https://github.com/airbytehq/airbyte/pull/8616) | Adds Brands, CustomRoles and Schedules streams |
| `0.1.8` | 2021-11-23 | [8050](https://github.com/airbytehq/airbyte/pull/8168) | Adds TicketMetricEvents stream |
| `0.1.7` | 2021-11-23 | [8058](https://github.com/airbytehq/airbyte/pull/8058) | Added support of AccessToken authentication |
| `0.1.6` | 2021-11-18 | [8050](https://github.com/airbytehq/airbyte/pull/8050) | Fix wrong types for schemas, add TypeTransformer |
| `0.1.5` | 2021-10-26 | [7679](https://github.com/airbytehq/airbyte/pull/7679) | Add ticket_id and ticket_comments |
| `0.1.4` | 2021-10-26 | [7377](https://github.com/airbytehq/airbyte/pull/7377) | Fix initially_assigned_at type in ticket metrics |
| `0.1.3` | 2021-10-17 | [7097](https://github.com/airbytehq/airbyte/pull/7097) | Corrected the connector's specification |
| `0.1.2` | 2021-10-16 | [6513](https://github.com/airbytehq/airbyte/pull/6513) | Fixed TicketComments stream |
| `0.1.1` | 2021-09-02 | [5787](https://github.com/airbytehq/airbyte/pull/5787) | Fixed incremental logic for the ticket_comments stream |
| `0.1.0` | 2021-07-21 | [4861](https://github.com/airbytehq/airbyte/pull/4861) | Created CDK native zendesk connector |