1
0
mirror of synced 2026-01-06 06:04:16 -05:00
Commit Graph

46 Commits

Author SHA1 Message Date
Arsen Losenko
2ded988944 🐛 Source Pinterest: update expected records (#27964) 2023-07-05 15:33:27 +03:00
Artem Inzhyyants
f68255fe31 Source Pinterest: update expected records (#27841) 2023-06-29 19:59:16 +02:00
Serhii Lazebnyi
da67a60b7c Source Confluence, Source Greenhouse, Source Hubspot, Source Stripe, Source Close com, Source Klaviyo, Source Notion, Source Pinterest, Source Snapchat Marketin, Source S3, Source Airtable and Source Posthog: fix builds (#27135)
* Fix SAT tests for confluence, greenhouse, hubspot, stripe

* Fix CAT for close, klaviyo, notion, pinterest and snapchat marketing

* Fix CAT for source s3

* Fix CAT for airtable and posthog

* Bump posthog version
2023-06-09 03:05:44 +02:00
Roman Yermilov [GL]
0a63c454b5 Source Pinterest: fix expected records (#26949)
* Source Pinterest: fix expected records

* Source Pinterest: updated schema for board_pins

* Source Pinterest: bump version

* Source Pinterest: update changelog
2023-06-02 22:04:09 +04:00
Ben Church
1dabc6208e Metadata: add tags field (#26320)
* Add optional tags field

* Remove duplicate icons

* Add programming tags to all

* Update docs

* supportUrl -> documentationUrl

* Ensure one language tag is applied

* Add keyvalue check

* rebase and fix tests

* Format

* Add cache buster

* Improve test

* Automated Commit - Formatting Changes

* Update error

* Fix missing tags

* Fix scaffold

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-05-26 16:13:09 -07:00
Ben Church
248bbf94c1 HACKDAY: Icon CDN (#26158)
* Move icons to connector folder

* Delete old icons

* Update upload logic

* Add icon url to definitions

* Update registry model

* Populate cdn url

* DNC butcher the pipeline

* Low hanging fruit fixes

* Fix bucket name

* Merge old and new approaches

* Fix metadata upload step

* Format

* Fix test
2023-05-24 17:25:41 -07:00
Roman Yermilov [GL]
5a4c2c44f5 Source Pinterest: fix expected records (#26356)
* Source Pinterest: fix expected records

* Source Pinterest: update schema for missing field
2023-05-23 14:15:17 +04:00
Artem Inzhyyants
ffad67dd55 Source Pinterest: Add pattern for start_date (#25984)
* Source Pinterest: Add pattern for start_date

* Source Pinterest: update docs

* Source Pinterest: fix backward compatibility

* Source Pinterest: handle wrong date format

* Source Pinterest: add unit test

* Source Pinterest: fix test

---------

Co-authored-by: artem1205 <artem1205@users.noreply.github.com>
2023-05-22 21:13:37 +00:00
Denys Davydov
5575d86f68 🐛 Source Hubspot, Gitlab, Salesloft, Pinterest: fix builds (#26188)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* connector health: fix builds for gitlab, salesloft, pinterest, hubspot

* conenctor health: update changelog

* Automated Change

* connector health - fix prop type

* connector health: upd expected records

* Automated Change

* empty commit to trigger ci

---------

Co-authored-by: davydov-d <davydov-d@users.noreply.github.com>
2023-05-17 20:03:50 +03:00
Denys Davydov
d756940092 🐛 Source Pinterest, Source Strava, Source Sentry, Source Xero: fix builds (#26112)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* Connector health: fix builds for sentry, strava, xero, pinterest

* connector health: upd changelog

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-05-16 11:39:45 +03:00
Denys Davydov
ac154b1db7 🐛 Source Mixpanel, Source Pinterest, Source Freshdesk: fix builds (#25915)
* #connector health: fix builds for pipedrive, mixpanel, freshdesk, pinterest

* Source Pipedrive: another fix

* Source Pipedrive: another fix

* Source Pipedrive: another fix

* Source Pipedrive: update expected records

* connector health: upd changelogs

* connector health: exclude pipedrive changes for now

* connector health: exclude pipedrive changes for now

* connector health: exclude pipedrive changes for now

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-05-11 08:17:34 +03:00
Augustin
7310494846 qa-checks: check metadata version matches dockerfile version (#25661) 2023-05-04 16:08:19 -07:00
Baz
3e6c8a37b0 🐛 Source Pinterest: fix boards schema with correct format for date-time (#25548)
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-04-26 17:17:41 +03:00
Serhii Lazebnyi
5bde434895 🐛Source Pinterest: Update AMOUNT_OF_DAYS_ALLOWED_FOR_LOOKUP to 89 days (#25313)
* Update `AMOUNT_OF_DAYS_ALLOWED_FOR_LOOKUP` to 90 days

* Redced AMOUNT_OF_DAYS_ALLOWED_FOR_LOOKUP to 89 days

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-04-24 22:05:29 +00:00
Ben Church
5563179782 Dagster: rename catalog to registry (#25254)
* rename catalog to registry in metadata service

* rename catalog to registry in metadata files

* Run generate models

* Fix missed renames

* Add github personal access token

* Run black

* Automated Change

---------

Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-04-18 22:15:11 +02:00
Augustin
ebc907cdf7 create metadata files for all connectors (#24964) 2023-04-13 07:45:04 +02:00
Artur Siepietowski
63850e5804 Source Pinterest - Adding missing columns for analytics streams (#23457)
* Adding missing metrics for pinterest source

* Fixing formatting

* Adding new version to docs

* update schemas

* remove minimal config catalog for tests

* removed expected file

* rollback test strictness level to high

* auto-bump connector version

---------

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-04-03 15:15:59 -03:00
Ella Rohm-Ensing
9cfc647fa5 Ella/fail on extra fields (#23985)
* Pass argument along, add test that should pass and test that should fail

* Add tests with additionalProperties

* Set additionalproperties=false when not set|

* Parametrize test cases

* Make the behavior 'optional'

* Fix parametrization for all combinations

* Improve CI credentials README and rename param

* Update naming to be clearer about columns only

* record_has_unexpected_field > record_has_unexpected_column

* Automated Change

* Add bypass for beta + ga connectors that failed

* Update docs and TODOs

* Update changelog and dockerfile

* Update TODO

* Update a few neglected connectors

* Remove uploaded file

* Update dockerfile after merge conflict

---------

Co-authored-by: marcosmarxm <marcosmarxm@users.noreply.github.com>
2023-03-23 22:22:44 +00:00
Augustin
bad5bce8ce CDK: remove unexpected error swallowing on abstract source's check method (#24240) 2023-03-23 13:04:51 +00:00
Serhii Lazebnyi
0b9ae8e209 Connector Health: Fixed SAT for GitHub, Google Analytics v4 (UA), Google Search Console, Hubspot, Recharge, Sendgrid, Greenhouse, Chargebee, Survey Monkey, Twitter, Zenloop, Zendesk-support, Pinterest, Slack, Sentry (#23946)
* Fix SAT source-google-search-console, source-hubspot, source-instagram, source-recharge, source-zendesk-support

* Fix github, sendgrid, google-analytics-v4, survey-monkey, twitter

* Added a newline at the end

* Fix pinterest, twitter expected records

* Fix hubspot and slack

* Fix sentry and GAv4(UA) allowed hosts

* Fix github and sendgrid

* Fix google analytics UA expected records

* Increase timeout for read records

* Fix chargebee and greenhouse

* Fix sentry expected records
2023-03-15 23:44:27 +01:00
Baz
29a450c0d1 🐛 Source Pinterest: fix for HTTP - 400 Bad Request when requesting data >= 90 days. (#23649) 2023-03-02 11:27:23 +02:00
Serhii Lazebnyi
3d8f81a3e4 Connector Health: Fixed SAT for Amazon Ads, Faker, Hubspot, Instagram, Pinterest, Sentry, Zendesk Talk (#23600)
* Fixed SAT for some connecotrs

* Fix unittest pinterest

* add allowedHosts

* Updated unittest for instagram

* Fix expected records instagram
2023-03-01 10:09:19 +01:00
Catherine Noll
7da6a3bb77 Run CATs with local CDK (#23084)
Scripts to 

* Run CATs against the local CDK for one connector

* Run CATs against the local CDK for multiple connectors

* Create a connecter image with the local CDK

---------

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2023-02-24 16:13:42 -05:00
Augustin
fbd6dbf091 python-connectors: remove explicit dependencies to connector-acceptance-test build (#23332) 2023-02-22 21:21:38 +01:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594)
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
2023-02-08 13:01:16 -08:00
Joe Reuter
6a10ae3e05 Rename source acceptance test to connector acceptance test (#21846)
Rename source acceptance test to connector acceptance test
2023-02-02 11:38:19 +01:00
Ella Rohm-Ensing
a1d1f2a202 Turn off default httpavailabilitystrategy for source-pinterest (GA) (#22020)
* Turn off default for source-pinterest (GA)

* Update source.py

* Update pinterest.md

* Update pinterest.md

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-01-28 02:30:13 +00:00
Arsen Losenko
aa22a53e67 Source Pinterest: enable SAT high strictness (#21477) 2023-01-18 20:32:00 +02:00
Alexandre Girard
0e85a977fc bump pinterest (#20532)
* bump

* reset

* update

* retroactively update changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-12-15 11:39:26 -08:00
Baz
88c652e060 🐛 Source Pinterest: added TypeTransformer to keep the schemas safe (#20242) 2022-12-14 03:12:37 +02:00
Eugene
a4ea907094 Fixed checkstyle by running the "./gradlew --no-daemon format --scan". NO changes in logic (#19214)
* Fixed checkstyle by running the "./gradlew --no-daemon format --scan". NO changes in logic
2022-11-09 19:36:37 +02:00
drewrasm
431ed0f475 🎉 Source Pinterest: filters for entity statuses (#15074)
* solve conflicts

* solve conflict in json schema

* bump to version 0.1.8 for the changes of this pr

* change ad account id in the schemas

* query to include data plane attributes (#18531)

* query to include data plane attributes

* rename functions

* fix java build

* more renaming fix

* Fix unit tests in source relational db (#18789)

* Fix unit tests

* Add extra test case for record count > 1

* Store record count in variable

* ci: use custom test-reporter action to upload job results (#18004)

* ci: use custom action to upload job results

* Correct coinmarket spec (#18790)

* correct coinmarket spec

* remove duplicate support normalization from source spec

* rollback coinmarketcap version in source def seed

* update connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* Parameterize test_empty_streams and test_stream_with_1_airbyte_column by destination (#18197)

* Remove lines that always add Postgres to list of destinations
* Parameterize all tests in test_ephemeral by destination

* 🐛 Source Facebook Marketing: reduce request limit after specific error (#18734)

* 🐛 Source Facebook Marketing: reduce request limit after specific error

* 🐛 Source Facebook Marketing: bump version; update docs

* 🐛 Source Facebook Marketing: add test

* 🐛 Source Facebook Marketing: increase timeout

* [charts/airbyte-cron] Cleanup env vars (#18787)

* [charts/airbyte-cron] Cleanup env vars

* Remove unused env var

* Use equalsIgnoreCase (#18810)

* Bump helm chart version reference to 0.40.40 (#18815)

Co-authored-by: perangel <perangel@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>

* 🐛Destination Google Sheets: Fix empty headers list (#18729)

* Fix empty headers list

* Updated PR number

* Bumped version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* 🐛Source Exchange Rates: Fix handling error during check connection (#18726)

* Fix handling error during check connection

* Updated PR number

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* Add normalization changelog and bump normalization version in platform (#18813)

* Remove ConfigPersistence usage from SecretsMigrator (#18747)

* remove config persistence from seeding logic (#18749)

* Remove the bulk actions from ConfigPersistence (#18800)

* hide ConfigPersistence inside ConfigRepository to discourage use (#18803)

* ci: add job and run id to test reports (#18832)

* Bump Airbyte version from 0.40.17 to 0.40.18 (#18827)

Co-authored-by: grishick <grishick@users.noreply.github.com>

* 🪟🔧 Remove styled components (round 1) (#18766)

* refactor EditorHeader (untested)

* refactor BaseClearView

* delete unused Subtitle

* refactor ConfirmationModal

* refactor Arrow

* refactor BulkHeader

* refactor CatalogTreeSearch

* refactor StreamFieldTable

* refactor StreamHeader

* refactor ConnectorIcon

* refactor TreeRowWrapper

* refactor DeleteBlock

* refactor EmptyResourceBlock

* revert unintended element change

* fixed acceptance tests (#18699)

* 🪟🔧 Reactor Breadcrumbs component to use anchors (#18764)

* refactor breadcrumbs to use actual links

* PR comments on styles

* increase test timeout for some webapp tests to prevent flakes (#18807)

* Remove "Filters and Segments" from Google Analytics v4 (#18508)

Filters and Segments info was incorrectly added to the Google Analytics v4 connector instead of the Google Analytics (Universal Analytics) Connector.

* Add notes about EU OAUth (#18835)

EU OAuth is not fully tested so adding a note to account for that.

* 🪟🐛 Fix: visual regression in ConnectorIcon (#18849)

* fix visual regression

* remove unused prop

* Add links to demo page (#18828)

* mysql-source:fix tinyint unsigned handling (#18619)

* mysql-source:fix tinyint unsigned handling

* update doc

* format

* upgrade version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

* 🪟 🎉 Allow environment specific sections in docs (#18829)

* Allow environment specific sections in docs

* Change syntax to lower case

* ci: replace GITHUB_OUTPUT with GITHUB_ENV on multiline variables (#18809)

* ci: replace GITHUB_OUTPUT with GITHUB_ENV on multiline variables

* ci: replace github set-ouput with new syntax in ./tools/bin/

* Add connection ID to span (#18852)

* edited connector docs (#18855)

* 🪟 🔧 Upgrade husky to 8.0.1 (#18719)

* Upgrade Husky

* Upgrade Husky

* Upgrade Husky

* Upgrade Husky

* Upgrade Husky

* ci: replace GITHUB_OUTPUT with GITHUB_ENV for multiline variables (#18857)

* Avoid NPE when adding connection ID to trace (#18856)

* Filter exit errors by operation name (#18850)

* add label

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
Co-authored-by: Liren Tu <tuliren@gmail.com>
Co-authored-by: Conor <cpdeethree@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Greg Solovyev <grishick@users.noreply.github.com>
Co-authored-by: Artem Inzhyyants <36314070+artem1205@users.noreply.github.com>
Co-authored-by: perangel <perangel@gmail.com>
Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: perangel <perangel@users.noreply.github.com>
Co-authored-by: Kyryl Skobylko <xpuska513@gmail.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: Charles <charles@airbyte.io>
Co-authored-by: Octavia Squidington III <90398440+octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com>
Co-authored-by: darynaishchenko <80129833+darynaishchenko@users.noreply.github.com>
Co-authored-by: Michael Siega <109092231+mfsiega-airbyte@users.noreply.github.com>
Co-authored-by: Tyler B <104733644+TBernstein4@users.noreply.github.com>
Co-authored-by: Yowan Ramchoreeter <26179814+YowanR@users.noreply.github.com>
Co-authored-by: Tim Roes <tim@airbyte.io>
Co-authored-by: Subodh Kant Chaturvedi <subodh1810@gmail.com>
Co-authored-by: Volodymyr Pochtar <v.pochtar@gmail.com>
Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com>
2022-11-08 11:55:43 +02:00
Serhii Chvaliuk
e7bf365d6d Source pinterest: Fix type of start_date (#18285)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-10-22 18:17:32 +03:00
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585)
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
2022-10-11 11:04:23 -04:00
Arsen Losenko
6c94e30bd9 Source Pinterest: Set start_date dynamically based on API restrictions for lookup (#17387)
* Set start_date dynamically based on API restrictions for lookup

* Update changelog and bump connector version

* Parse start_date from config

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-30 16:55:15 +03:00
Augustin
ff4ea3961a Republish connectors using CDK 0.1.88 to 0.1.89 (#17304) 2022-09-28 18:18:59 +02:00
Brian Lai
d0386249de bump notion, pipedrive, and pinterest to the latest CDK including per-stream changes (#16799) 2022-09-21 18:17:42 -05:00
Luis Gomez
50014742f3 🎉 Source Pinterest: Added backoff strategy for rate-limit errors (#16161) 2022-09-06 17:06:45 +03:00
Arsen Losenko
46112cfaa1 Source Pinterest: support OAuth (#16271)
* Source Pinterest: Initial setup of OAuth flow

* Remove previously added class and method for auth

* Update Java part of OAuth flow, update spec

* Update spec and add additional methods to Java OAuth flow

* Add backwards compatibility for old config structure

* Add missing imports

* Revert previous changes in source_specs.yaml

* Cleanup in imports and source_specs

* Add missing imports

* Add missing imports

* Remove 'subdomain' logic from Java OAuth flow

* Update docs

* Update docs accordingly to comments in PR

* Refactor credentials variable

* Fix typo

* Update acceptance-test-config.yml

* Specify integer type for AD_ACCOUNT_ID value in schemas

* updated SAT tests, fixed Max Rate Limit error handling

* updated unit_tests

* updated schemas, added caching for Boards and AdAccounts stream to reduce API Calls for child streams, commented out Incremental and Full refresh SAT tests

* auto-bump connector version [ci skip]

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-06 13:16:19 +03:00
Baz
8737aedd4a 🎉 Source Pinterest: increase unit_test coverage up to 90% (#16164) 2022-09-01 14:52:07 +03:00
Marcos Marx
dca2256a7c Bump 2022 license version (#13233)
* Bump year in license short to 2022

* remove protocol from cdk
2022-05-26 15:00:42 -03:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763)
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
2022-04-11 20:51:37 -07:00
Ronald Fortmann
fd747691bf Source Pinterest: fix typo in schema fields (#10223) 2022-02-24 15:36:37 -03:00
Iryna Grankova
04704ffa8e Update fields in source-connectors specifications: pinterest (#9043)
* Files title/description update for issue # 8913

* Version update for issue # 8913

* Changelogs update for PR #9043

* Update docs/integrations/sources/pinterest.md

* update test_stream_slices

* bump version

Co-authored-by: Anna Lvova <37615075+annalvova05@users.noreply.github.com>
Co-authored-by: alvova <annalvova05@gmail.com>
2021-12-28 17:26:00 +02:00
Davin Chia
7f6d12a7da Davinchia/tracking client refactor (#7595)
* Add doc string to tracking interface.

* Add doc string with minor refactor.

* Update airbyte-analytics/src/main/java/io/airbyte/analytics/SegmentTrackingClient.java

* Format javadoc. Add user_id trait.

* Better formatting javadocs.

* Turn on spotless.

* Format before checking again.

* Clean this up.
2021-11-11 00:21:33 +08:00
Serhii Lazebnyi
2d2965b879 🎉 New Source: Pinterest (#7493)
* Add Pinterest implementation

* Updated to review

* Added rate limits and PR number to doc

* Fixed unit tests and data slicing bug

* Update airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/connectors/source-pinterest/build.gradle

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Updated to review

* Updated window in days

* Fixed unit test

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-11-09 12:54:12 +02:00