1
0
mirror of synced 2025-12-21 19:11:14 -05:00
Commit Graph

167 Commits

Author SHA1 Message Date
Serhii Chvaliuk
aab03770c9 🐛 Source SalesForce: better detect API type (#16086)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-08-31 01:06:26 +03:00
Edward Gao
ab98441488 Source Salesforce: exclude KnowledgeArticleVersion from bulk api (#16119)
* exclude KnowledgeArticleVersion from bulk api

* version bump + changelog

* add other affected KnowledgeArticle streams

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* auto-bump connector version [ci skip]

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-08-29 19:49:59 -07:00
Jagruti Tiwari
8288c16485 fix: replace airbyte oss with airbyte open source (#15885)
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2022-08-24 01:01:53 -03:00
Liren Tu
35b0b1be4b 🐛 Salesforce source: exclude knowledge-article from bulk api (#15901)
* Exclude KnowledgeArticle from bulk api

* Bump version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-08-23 19:49:53 -07:00
Baz
959d862e5b 🐛 Source SalesForce: changed DEFAULT_WAIT_TIMEOUT_SECONDS to 24-hour limit (#15444) 2022-08-10 19:20:22 +03:00
Topher Lubaway
9c6c092a22 Revert "Improving docusaurus sidebar generation (#1927) (#14369)" (#14596)
This reverts commit a2c194a11f.
2022-07-11 15:27:14 -05:00
Mykyta Serbynevskiy
a2c194a11f Improving docusaurus sidebar generation (#1927) (#14369)
* Improving docusaurus sidebar generation (#1927)

* Added "Career & open positions" folder to sidebar, adjusted "Project overview" folder

* Deleted "career-and-open-positions" folder from sidebar
2022-07-08 14:18:27 -05:00
Amruta Ranade
d4071a0ae5 source-salesforce,source-google-sheet,destination-snowflake: Updated field descriptions(#13729) 2022-07-07 15:55:52 +02:00
Amruta Ranade
c42576440d Update salesforce.md
Fixed broken link
2022-06-21 10:55:40 -04:00
Marcos Marx
669e6ed39d Source Salesforce: fix sync capped streams with more records than page size (#13658)
* change logic for counting records

* update doc

* correct unit test

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-14 09:08:38 -03:00
Amruta Ranade
7ecfa13ee0 Updated BigQuery, Google Sheets, Facebook Marketing, Salesforce docs (#13717)
* initial changes

* Edited google Sheets doc

* More edits

* edited the intro and prereqs for BigQuery

* edited the data loading section

* more edits

* Grammatical edits

* Formatting edits
2022-06-13 14:25:41 -04:00
Pedro S. Lopez
816fece247 bump GA connectors to latest version of python cdk (#12685)
* bump GA connectors to latest version of python cdk

* update changelogs

* fix changelogs

* more changelog fixes

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-06 20:52:59 -04:00
Alexandre Girard
22b67d8528 source salesforce: use utf8 by default and iso as fallback (#12576)
* use utf8 by default and iso as fallback

* test both

* add comment

* Bump version
2022-05-04 11:04:18 -07:00
Alexandre Girard
90162b6e76 Encoding is ISO-8859-1 (#12552)
* Encoding is ISO-8859-1

* rename test

* bump

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-03 19:23:41 -07:00
Brian Lai
bb2da42fde source freshdesk, hubspot, mixpanel, paypal-transaction, salesforce, zendesk-support: adding fixtures to mock time.sleep for connectors that explicitly sleep (#12335)
* adding fixtures to mock time.sleep for connectors that explicitly sleep

* bump connector versions

* update changelog doc for each connector

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* remove version bump for freshdesk because connector tests are in a bad state

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-28 12:21:45 -07:00
Baz
644ace4036 🐛 [On call / 171] Source Salesforce: fixed the bug when Bulk fetch took all memory of kube pods (#11692) 2022-04-13 20:35:10 +03:00
Topher Lubaway
17407da1e6 Add docusaurus tool not pipeline logic (#11716)
* Adds docusaurs tooling

tooling works
broken links still exist

needs the pipeline to work with GH pages
merging sooner means that we won't have forked problems from
the relatively frequently updated docs folder

* Updates readme

* Update docusaurus/README.md

Co-authored-by: Davin Chia <davinchia@gmail.com>

* Update docusaurus/README.md

Co-authored-by: Davin Chia <davinchia@gmail.com>

* Update docusaurus/README.md

Co-authored-by: Davin Chia <davinchia@gmail.com>

* Update docusaurus/README.md

Co-authored-by: Davin Chia <davinchia@gmail.com>

* Update docusaurus/docusaurus.config.js

Co-authored-by: Davin Chia <davinchia@gmail.com>

Co-authored-by: Davin Chia <davinchia@gmail.com>
2022-04-06 10:32:36 -07:00
Topher Lubaway
03ed894917 Updated GA connector docs and connector (#11640)
pipeline trigger because gitbook is sad

This reverts commit 230134068f.
2022-03-31 15:29:34 -05:00
Topher Lubaway
230134068f Revert "Updated GA connector docs and connector " (#11639)
gitbook pipelines failed.  reverting to trigger gitbook pipeline again
This reverts commit 6aa7e4ce79.
2022-03-31 15:24:17 -05:00
Amruta Ranade
6aa7e4ce79 Updated GA connector docs and connector stages (#11628)
* updated GA connector docs and README

* minor edit
2022-03-31 10:07:03 -04:00
Sherif Nada
ed10c700d6 Fix typo in salesforce docs 2022-03-01 14:57:01 -08:00
Sherif Nada
b3546be317 fix salesforce docs markdown formatting 2022-03-01 14:55:36 -08:00
Sherif A. Nada
df47987cd6 📕 source salesforce: fix broken page anchor in spec.json & add guide for adding read only user (#10751) 2022-03-01 10:42:37 -08:00
Sherif A. Nada
9e941f3818 Improve note in salesforce docs about creating a RO user 2022-02-27 20:53:40 -08:00
Sherif A. Nada
5691434073 📖 improve salesforce docs & reorder properties in the spec (#10679) 2022-02-27 20:08:46 -08:00
Maksym Pavlenok
2bba529803 🎉 Source Salesforce: speed up discovery >20x by leveraging parallel API calls (#10516) 2022-02-27 19:03:39 -08:00
Anna Lvova
ce5c007540 🎉 Salesforce Source: Support "Replicate Incremental Deletes" (#10454) 2022-02-21 15:03:50 +02:00
Maksym Pavlenok
5bc8ec205e 🐛 Source salesforce: processing of failed jobs (#10141) 2022-02-10 19:49:08 +02:00
Serhii Chvaliuk
84d7323d43 🐛 Source Salesforce: increase CSV field_size_limit (#10012)
* Increase CSV field_size_limit

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-02-04 21:06:33 +02:00
augan-rymkhan
cb69017ec8 Source Salesforce: finish the sync with success if rate limit is reached (#9499)
* finish the sync with success if rate limit is reached

* todo

* added rate_limit_handler

* added rate_limit_handler decorator

* removed TODO

* deleted rate_limit_hanlder decorator

* unit tests for rate limit is reached

* format code

* bump version

* format code

* updated spec and def yaml

Co-authored-by: auganbay <auganenu@gmail.com>
2022-01-28 11:17:31 +06:00
Serhii Chvaliuk
2629098390 🐛 Source Salesforce: parse CSV with "unix" dialect (#9757)
* improve csv parsing
* test_csv_reader_dialect_unix added

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-01-25 19:27:25 +02:00
Iryna Grankova
579923d2f8 Update fields in source-connectors specifications: posthog, recurly, redshift, salesforce, salesloft (#8617)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
Co-authored-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-01-25 16:14:02 +02:00
Serhii Lazebnyi
0ed20bca61 🐛 Source Salesforce: Fix error during generating schema (#9478)
* Add filtering by queryable flag for available streams

* Updated PR number and version

* add logs for failed bulk jobs

* Added logs for collect stream objects options if code 404

* Fixed unittests

* Fixed integration tests

* Bumped docker version

* Update version in seed

* Update streams.py

* Added logger for print not queriable streams

Co-authored-by: antixar <antixar@gmail.com>
2022-01-24 10:54:24 +02:00
augan-rymkhan
0a3713a5a5 Source Salesforce: Deprecate API Type parameter (#9302)
* use BULK for the first sync, REST for incremental sync

* if stream contains compound data or/and base64 use always REST

* fix get stream state from connector state

* fix integration test

* refactor catalog name

* format code

* refactor unit tests

* refactor unit tests 2

* format code 2

* Set additionalProperties to true not to break test temporarily

* fix unit test and remove unnecessary filtering fields

* bump version

* updated spec and def yaml

Co-authored-by: auganbay <auganenu@gmail.com>
2022-01-19 16:23:54 +06:00
augan-rymkhan
62c433eeb1 Source Salesforce: fix pagination in REST API streams (#9151)
* fix next_page_token

* fix BULK API

* fix BUlk incremental stream

* added unit test and comments

* format code

* bump version

* updated spec and def yaml

Co-authored-by: auganbay <auganenu@gmail.com>
2022-01-18 20:42:21 +06:00
Yevhenii
791f39aee9 🐛 Salesforce Connector: Correcting the presence of an extra 'else' handler in the error handling (#9409)
* Correcting the presence of an extra 'else' handler in the error handling
2022-01-11 12:45:22 +02:00
Yevhenii
b5d24106a3 🐛 Salesforce Connector: handling 400 error, while sobject doesn't support query or queryAll requests (#9386)
* Salesforce Connector: handling 400 error, while sobject doesn't support query or queryAll requests
2022-01-11 12:08:54 +02:00
vitaliizazmic
f466986f34 🎉 Source Salesforce: Support oauth'ing into sandbox environment, switching to advanced_auth (#8797)
* Source Salesforce #7722 - change spec to use advanced_auth

* Source Salesforce #7722 - update oauth flow

* Source Salesforce #7722 - reformat

* Source Salesforce #7722 - fixing get environment in oauth flow

* Source Salesforce #7722 - improve spec

* Source Salesforce #7722 - reformat

* Source Salesforce #7722 - bump version and update docs
2022-01-11 11:36:19 +02:00
Yevhenii
432d9f77ed Source Salesforce: add note about possible to creating custom user for using Airbyte (#9185) 2021-12-29 17:30:48 +02:00
Yevhenii
ea578e8f10 🎉 Source Salesforce: Add the ability to filter streams (#8871)
* Source Salesforce: Add the ability to filter streams (version 0.1.11)
* Source Salesforce: fix examples for new field in specification (version 0.1.12)
2021-12-23 13:54:18 +02:00
Maksym Pavlenok
a4be492111 🐛 Source Salesforce: 400 error for non-queryable streams (#9005) 2021-12-23 02:42:23 +02:00
Serhii Chvaliuk
dfdd2c5fb5 🐛 Source Salesforce: filter 'null' byte(s) in HTTP responses (#8405)
* filter_null_byte added

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2021-12-07 19:20:28 +02:00
augan-rymkhan
4e44166d0f Salesforce make start_date optional and change format to YYYY-MM-DD (#8191)
* Make `start_date` optional

* start_date can take format `2021-07-25`

* added title to all spec input properties and changed version in Dockerfile

* added a new item into salesforce.md changelogs

* fixed case when start_date is None

* formatted code

* changed comments in unit test fixtures

* changed spec title fields

* updated source and spec yaml

Co-authored-by: Auganbay <auganenu@gmail.com>
2021-12-03 14:38:47 +06:00
Yevhenii
81d66a6388 Source Salesforce: Handling 400 error when using BULK API (#8206)
* Source Salesforce: Handling 400 error when using BULK API
2021-11-24 15:42:10 +02:00
Maksym Pavlenok
9a3a32711c 🐛 Source salesforce: failed async bulk jobs (#8009)
* update dockerfile

* update version

* update changelogs

* remove test config

* fix after flake8

* bump versio to 0.1.6

* remove secrets from config

* update source specs
2021-11-18 01:45:35 +02:00
Yevhenii
d9135b0f13 🎉 Source Salesforce: Add Transform for output records (#7885)
* Source Salesforce: Add Transform for output records
2021-11-15 15:41:29 +02:00
Yevhenii
3d4f730a9a 🐛 Source Salesforce: Fix types for anyType fields (#7778)
* Fix types for anyType fields
2021-11-09 16:08:37 +02:00
Yevhenii
2efef3c04d 🐛 Source Salesforce: Fix getting anyType fields using BULK API (#7592)
* Source Salesforce: Fix getting anyType fields using BULK API
2021-11-06 00:42:06 +02:00
Abhi Vaidyanatha
ae32ecbb27 GitBook: [master] 186 pages and 77 assets modified 2021-10-08 21:17:47 +00:00
Yevhenii
65d62eb560 🎉 Declare Oauth init parameters for 4 connector (#6438)
* Annotate Oauth init parameters for connectors: Facebook Marketing/Pages, Instagram, Salesforce
2021-09-30 18:29:51 +03:00