1
0
mirror of synced 2025-12-22 19:38:29 -05:00
Commit Graph

387 Commits

Author SHA1 Message Date
Vladimir remar
06eff7d8ca 🎉 Source Hubspot: Add tickets associations to engagements streams. (#11423) 2022-03-31 18:57:08 +02:00
Christophe Duong
316180c186 format code (#11399) 2022-03-25 09:55:11 +01:00
Serhii Lazebnyi
6528926b12 🐛 Source Hubspot: Fix non exists updated at field error (#11321)
* Fix updated at field non exists issue

* Updated PR number

* Change state default value to self._state

* Bumped seed version
2022-03-24 17:21:49 +02:00
Marcos Marx
f8fc057c54 Hubspot bump connector and update doc (#11332)
* Fixed engagement stream pagination

* Added unit test

* Removed unused import

* Fixed issue if incremental engagements attempts to get more than 10K records

* Fixed comment

* Merged import statement in unit test

* bump connector and update doc

Co-authored-by: lgomezm <luis@calixa.io>
2022-03-22 22:15:00 -03:00
Luis Gomez
6cf1f09771 🐛 Source Hubspot: Fixed engagements pagination (#11266)
* Fixed engagement stream pagination

* Added unit test

* Removed unused import

* Fixed issue if incremental engagements attempts to get more than 10K records

* Fixed comment

* Merged import statement in unit test
2022-03-22 22:12:53 -03:00
Serhii Lazebnyi
103dd2a2ae Increased unit test coverage to 90 (#11317) 2022-03-22 20:15:57 +02:00
girarda
b74e72e9c1 Improve hubspot documentation (#11218)
* typo

* remove hint block

* Move required scopes to top level header

* Revert "remove hint block"

This reverts commit c7d070abbd.

* update

* anchor link

* update other links too

* reset

* anchor link

* Update doc

* com -> io

* Update docs/integrations/sources/hubspot.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* only update doc

* Update changelog

* anchor link

* Bump version

* Update specs

* Capitalize

* link to where to find credentials

* update header level

* Capitalize

* Capitalize

* not all caps

* API Key

* Update cloud setup instructions

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2022-03-18 14:52:24 -07:00
Davin Chia
5a575e42e0 Format connectors. (#11231) 2022-03-17 19:08:01 +08:00
Anna Lvova
fd49f92ebc 🐛 Source Hubspot: fix casting of float numbers (#11105)
* upd documentation

* fix comma in float number

* bump version

* add unit test for float casting

* bump version
2022-03-16 14:44:02 +02:00
girarda
b19f2415a9 🎉 Source HubSpot: Set Primary keys for streams with an identifier (#11121)
* Set partition key on streams with id field

* reset to master

* Update readme with primary key

* This can be incremental

* I think this can also support incremental

* incremental streams

* incremental

* Missing comma

* Everything can be incremental

* set pk

* Add a primary key

* Add missing pk

* format

* Update doc

* Bump version

* Not everything can be incremental

* fix field

* Update pk

* Update source_specs
2022-03-15 15:06:28 -07:00
Marcos Marx
669a1220ef Source Hubspot bump connector version and format files
* Handled search queries that would output more than 10K records

* Getting CRM search objects in ascending chronological ortder

* Fixed stream

* Fixed rebase

* Fixed condition

* Added unit test

* Removed unused import

* Started a new query when reached 10K records

* Moved comment

* bump connector docker version

* bump connector version

* correct dockerfile version

* change doc version

Co-authored-by: lgomezm <luis@calixa.io>
2022-03-14 22:45:43 -03:00
Luis Gomez
710543a9ab 🐛 Source Hubspot: Handled 10K+ search-endpoint queries (#10700)
* Handled search queries that would output more than 10K records

* Getting CRM search objects in ascending chronological ortder

* Fixed stream

* Fixed rebase

* Fixed condition

* Added unit test

* Removed unused import

* Started a new query when reached 10K records

* Moved comment
2022-03-14 22:17:11 -03:00
Christophe Duong
744e0d5f13 Refactor Snowflake internal Staging as a base class for other staging classes (#10865)
* Refactor Snowflake internal Staging as model to share staging abilities in jdbc destinations
2022-03-11 15:29:12 +01:00
Luis Gomez
d39b63674e 🐛 Source Hubspot: Removed stage history from deals stream (#10707)
* Removed stage history from Hubspot deal stream

* Fixed stream slices on CrmSearchStream

* Removed unused DealStageHistoryStream class

* Bumped connector version
2022-03-05 09:07:42 +02:00
Marcos Marx
47c4f233f3 🎉 add associations companies to deals, ticket and contacts stream (from PR 9027) (#10631)
* Added associations to some CRM Object streams in Hubspot connector

* Added associations in the relevant schemas

* fix eof

* bump connector version

Co-authored-by: ksoenandar <kevin.soenandar@gmail.com>
2022-02-24 15:44:04 -03:00
augan-rymkhan
7b3fd6ba4a Source Hubspot: cast timestamp to date/datetime (#10576)
* cast timestamp to date

* change test name

* fix corner cases

* fix corner cases 2

* format code

* changed method name

* add return typing

* bump version

* updated spec and def yaml

Co-authored-by: auganbay <auganenu@gmail.com>
2022-02-24 19:09:45 +06:00
augan-rymkhan
9c9dfb25f1 Source Hubspot: add "format": "date-time" (#10492)
* add "format": "date-time"

* add "format": "date-time" to feedback_submissions

* add "format": "date-time" to deals

* bump version

* updated spec and def yaml

Co-authored-by: auganbay <auganenu@gmail.com>
2022-02-22 20:46:36 +06:00
augan-rymkhan
2282a4ae02 🎉 Source Hubspot: Migrate to CDK (#10177)
* migrate SourceHubspot to cdk

* refactor discover method

* change method name

* deleted Client class

* remove comment

* added get_updated_state

* fix setting initial state

* fix stream_state dict key

* fix cursor_field

* change check test case status

* refactor streams method

* remove comment

* remove TODOs

* remove comments

* fix get_updated_state

* refactor chunk_read

* override _read_incremental

* fix unit tests

* remove comments

* fix test_check_connection_backoff_on_server_error

* fix test_check_connection_backoff_on_server_error 2

* fix test_check_connection_backoff_on_limit_reached

* fix unit tests

* clear comments

* override read method on Source

* added comments to overriding methods

* some improvements

* reafactor overridden _read_incremental

* format code

* refactor discovery

* remove discover

* format code 2

* added return types

* refactor template stream classes

* remove comments

* remove _name field

* rename api.py to streams.py

* move to HttpStream

* refactor FormSubmissions

* refactor Campaings

* refactor ContactsListMemberships

* CRMSearchStream refactor

* CRMSearchStream refactor 2

* CRMObjectStream refactor

* DealStageHistoryStream refactor

* Deals refactor

* Engagements refactor

* path method refactor

* refactor authentication

* fix check_connection

* fix call parse_response

* fix Engagements stream

* fix CRMSearchStream

* fix CRMObjectIncremental stream

* override _read_incremental

* remove commented codes

* format code

* update cdk version

* fix cursor field

* fix unit tests

* removed client

* clear comments

* clear comments 2

* clear comments 3

* clear comments 4

* override backoff_time

* remove comment

* format code

* backoff_time modified

* refactor backoff_time

* format code

* added return typing

* format code

* removed cursor_paths

* bump version

* updated spec and def yaml

Co-authored-by: auganbay <auganenu@gmail.com>
2022-02-21 22:03:05 +06:00
Vladimir remar
fb647ad1e1 :tada:Source hubspot: add associations to tickets stream. (#10142)
* update: client add associations to tickets stream

* update: tickets json schema add deals and contacts

* bump connector version

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2022-02-15 16:08:24 -03:00
augan-rymkhan
b22efc03a1 Source Hubspot: fix "quotes" key error exception (#10055)
* check if stream exists in source

* check if stream exists in source, added comment

* test skipping reading quotes stream

* format code

* airbyte-cdk version

* added __init__.py to unit_tests

* fix importing airbyte models

* bump the version

* update spec and def yamls

Co-authored-by: auganbay <auganenu@gmail.com>
2022-02-10 10:39:50 +06:00
Vladimir remar
983d6ae2ba :tada:Source hubspot add new streams for engagements(calls, emails, meetings, notes and tasks) (#9786) 2022-02-04 20:09:29 +01:00
LiRen Tu
a094142825 Format code (#9892) 2022-01-29 19:56:00 -08:00
gergelylendvai
1c421a9791 Source Hubspot: Fix getting submissions for all forms (#9555)
Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2022-01-26 23:24:57 -03:00
Tino Merl
b40b303638 🎉Source HubSpot: Adds form_submission and property_history streams (#7787)
* Began working on HubSpot Form Submission Connector

* Added Property History Stream

* Added form_guid to as value to form_submissions_stream.

* Finalized the Form Submission Stream

* Added documentation and test config

* Corrected Version Number

* updated version number to 0.1.25

* removed or none worked on tests

* Changed code due to review comments & merges

* readded Propertyhistory after merging

* bump connector version

Co-authored-by: Tino Merl <tino.merl@park-sieben.com>
Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2022-01-26 00:34:15 -03:00
Vladimir remar
d616279bea Source hubspot: add Feedback Submissions stream and update Ticket Pipelines stream (#9081)
* update: on api.py update url for ticket_pipelines

* update: on client.py add new entries for feedback_submissions streams and ticket_pipelines

* update: add to shchemas feedback_submissions and ticket_pipelines

* update: to integration_tests_abnormal_state add new value for feedback_submissions

* update: in sample_files add new entries for feedback_submissions and ticket_pipelines

* update: bump connector version

* fix: in feedback submission replace timestamp to updatedAt

* update version in source_definitons and Dockerfile

* run format

* config seed

* bump connector version in config file

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2022-01-24 23:06:27 -03:00
Serhii Chvaliuk
c524e0af64 🎉 Source HubSpot: Add more fields for email_events stream (#9641)
* email_events.json updated
* bugfix: "date-time" -> "date"

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-01-21 19:33:17 +02:00
Luis Gomez
25fb7e7fd7 Source Hubspot: Some incremental CRM objects and engagements (#8887) 2022-01-17 11:54:31 +01:00
gergelylendvai
9cc2560935 Source Hubspot: adding form_submissions stream (#8011)
* Source Hubspot: adding form_submissions stream

* Adding form_id to form_submissions output

* Adding form_id to form_submissions output and schema

* Removing field deletions and id generation

* Adding new line to form_submissions.json

* Fixing form_submissions.json schema

* add form_submissions as empty_stream to run ci airbyte

* add forms_submissions to empty_stream

* bump connector seed version

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2022-01-13 21:06:51 -03:00
Serhii Chvaliuk
9909cd69f2 🎉 Source HubSpot: engagements stream, remove auto-generated properties (#9385)
* extend schemas/engagements.json

* fix unix newlines

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-01-13 16:49:32 +02:00
vitaliizazmic
7c4ae13473 🎉 Source HubSpot: support list memberships in Contacts stream (#9129)
* Source Hubspot #8477 - separate stream for contacts list memberships

* Source Hubspot #8477 - reformat

* Source Hubspot #8477 - fix basic read tests: add contacts_list_memberships to empty streams

* Source Hubspot #8477 - bump version and update documentation
2022-01-10 22:26:25 +02:00
Baz
102994541a 🎉 Source Fecebook-Marketing: add asset_feed_spec to ad creatives stream (#8962) 2021-12-21 15:46:52 +02:00
Serhii Lazebnyi
212c1ff22c 🎉 Source Hubspot: Support incremental sync on all possible streams (#8699)
* Add incremental sync support for companies, contact_lists, contacts, deals, line_items, products, quotes, tickets streams

* Updated PR number

* Fix formating

* Fix typo

* Commented unsupported tests

* Updated to review

* Updated abnormal state file

* Deleted comment

* Updated version in docker

* Updated version in spec yaml and seed
2021-12-17 15:48:47 +02:00
ycherniaiev
2ad4ca1d5c Update fields in source-connectors specifications: drift, harvest, hubspot, intercom, monday, clickhouse, bing-ads, snapchat-marketing, microsoft-teams (#8429)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Co-authored-by: Sergey Chvalyuk <grubberr@gmail.com>
2021-12-16 17:52:37 +02:00
Serhii Chvaliuk
cde71578c9 🐛 Source Hubspot: additionalProperties: true (#8658)
* additionalProperties: true in spec

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2021-12-15 18:38:20 +02:00
Serhii Chvaliuk
757650f9de 🐛 Source Hubspot: remove 'skip_dynamic_fields' config param (#8329)
* ignored fields added
* remove 'skip_dynamic_fields'

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2021-11-30 23:28:26 +02:00
Serhii Chvaliuk
0d7e69e760 🐛 Source Hubspot: unset 'hs_time_in_*' properties (#8216)
* 'skip_dynamic_fields' added

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2021-11-26 21:35:31 +02:00
Christophe Duong
86ca36c5c0 Format code (#7978) 2021-11-15 14:51:10 +01:00
Serhii Chvaliuk
1b38fc7b79 bugfix 'Hubspot' -> 'HubSpot' (#7683)
* bugfix 'Hubspot' -> 'HubSpot'

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2021-11-09 20:43:17 +02:00
Dmytro
c918f52f16 Fix Hubspot and Asana oAuth flow. (#7730) 2021-11-08 14:22:20 +02:00
Yevhenii
90451eb075 🎉 Source Hubspot: Migrate Hubspot source to CDK structure (#7562)
* Migrate Hubspot source to CDK structure
2021-11-03 09:57:56 +02:00
Yevhenii
906aa75bc9 🐛 Source Hubspot: hotfix import urllib package (#7405)
* Source Hubspot: hotfix import urllib package
2021-10-27 14:30:27 +03:00
Yevhenii
85851b7e0f 🐛 Hotfix Hubspot: add the length of separator symbol to check (#7393)
* hotfix for split_properties function
2021-10-26 23:27:04 +03:00
Yevhenii
928aa6280c 🐛 Source Hubspot: Fix issue with getting 414 HTTP error for streams (#6954)
* Source Hubspot: Fix issue with getting 414 HTTP error for streams

* update code and schemas

* bump version
2021-10-26 15:37:07 +03:00
Marcos Marx
58d81b2500 Publish PR 5840: hubspot connector add stream (#7220)
* Add new marketing emails (with statistics) stream

* Update docs and changelog

* bump version in config files

* run format

Co-authored-by: n0rritt <dev.n0rritt@gmail.com>
2021-10-20 21:02:18 -03:00
Hai To
fae4d00c8f Source Hubspot: Add new marketing emails (with statistics) stream (#5840)
* Add new marketing emails (with statistics) stream

* Update docs and changelog
2021-10-20 20:37:48 -03:00
Yevhenii
6c4ae4aac4 Source Hubspot: update SAT config (#6995)
* Update discover method: disable quotes stream when using OAuth config
2021-10-14 13:37:49 +03:00
Vadym
5a5b3c2571 🎉 Source Hubspot: Bump docker version (#6840)
* Bump Hubspot docker version

* spec.json oauth2Specification rootObject fixed
2021-10-06 18:05:44 +03:00
Vadym
351e998394 🎉 Source Hubspot: Support OAuth (#6465)
* Add required scopes to hubspot.md docs

* Update spec with oauth creds.
Update acceptance test config with oauth tests.
Add oauth test creds configs.

* Fix spec.js `oneOf` usage

* Add CDK authenticators usage.
Update source acceptance tests.
Update spec.json.

* Fix source unit tests

* Update hubspot.md scopes docs

* Add spec field titles.
Update comments.

* Update spec.json oauthFlowOutputParameters.

* Update spec.json to pass SAT.
2021-10-04 17:11:34 +03:00
Michel Tricot
1773e41e47 Shorten our headers + adds contributors file (#6478) 2021-09-27 10:45:50 -07:00
Yevhenii
aac98b9b9b 🐛 Source Hubspot: Use correct schema for 'owners' stream (#6374)
* Source Hubspot: Use correct schema for 'owners'  stream

* Update builds.md with correct data
2021-09-23 15:03:03 +03:00