jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-22 19:38:29 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,117 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

387 Commits

Author SHA1 Message Date
LiRen Tu
548a3a3063 Format code (#6381) 2021-09-22 09:30:14 -07:00
Sherif A. Nada
f5c8ce2ebc publish PR 5693 (#6299) 
Co-authored-by: vladimir-remar <vladimir.remar@gmail.com>
 2021-09-19 18:52:35 -07:00
Vladimir remar
f560ae18cf :tada:Source Hubspot: Add contacts associations to Deals stream. (#5693) 2021-09-19 18:52:18 -07:00
Yaroslav Dudar
5e3b24ff4e 🐛 Source Hubspot: fix array fields in schema with missed items property (#5834) 
fixed array schemas in companies
 2021-09-07 22:17:03 +03:00
Dmytro
60fd9d5ab9 🐛 Fix hubspot datetime empty string (#5798) 
* Fix hubspot datetime empty string
 2021-09-06 10:11:48 +03:00
Marcos Marx
e873129fe4 remove all date-time schemas (#5685) 2021-08-26 21:59:23 -03:00
Baz
bc89f85bd6 🐛 📚 Source Hubspot: Fix fail on reading stream using API Key without required permissions (#5463) 
Fix fail on reading stream using `API Key` without required permissions (#5463) https://github.com/airbytehq/airbyte/pull/5463

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
 2021-08-18 12:52:27 +03:00
Davin Chia
07d854f564 Set up Shortio integration test and release connector. (#5418) 2021-08-16 12:27:53 +08:00
Baz
6f56833473 🐛 Source HubSpot: Fix empty string inside number / float datatype (#5334) 
#5293 - Source Hubspot fails in normalization step

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
 2021-08-12 18:47:50 +03:00
Sherif A. Nada
8e74551703 🐛 Source hubspot: correctly use logger exception printing (#5250) 2021-08-06 13:41:14 -07:00
Sherif A. Nada
6b07d3014f publish PR 4550 (#5219) 
Co-authored-by: Rodrigo Parra <rodpar07@gmail.com>
 2021-08-04 23:10:39 -07:00
Arthur Galuza
25c04db295 🎉 Source Hubspot: update user-defined custom field schema generation (#4913) 
* Handle field types, convert unknown to string with warning

* Bump version

* Upd version in source definitions

* Add json field format for date fields

* Add a stub integration test to bypass CI/CD test stage

* Replace cached_property with lru_cache to support Python < v3.8

* Extract the field format converting logic, cover it with a unit test

* Set default field type to `string`

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Upd changelog

* Cleanup

* Fix tests

* Downgrade version tag

* Remove caching for the custom properties field

* Bump version tag

* Add type casting for malformed fields

* Upd schemas

* Disable connection tests

* Fix docs, add custom properties cache

* Upd tests, check failed state

* Disable invalid config test

* Add starte_date param into config

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-08-03 17:52:52 +03:00
Eugene Kulak
9de53688e3 SAT: remove deprecated validate_output_from_all_streams from configs (#5010) 
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-07-27 03:02:32 -04:00
Sherif A. Nada
b86ebf63fb Automatically pull latest SAT & build connector image in SAT runner (#4969) 2021-07-26 11:56:19 -07:00
Yevhenii
5971983004 Source HubSpot: Add dealstage history retrieval to DealStream (#4131) 
* Source HubSpot: Add dealstage history retrieval to DealStream

* Apply code review feedback, handle missing attrs

* Fix URL and updated_at field

* Apply CR feedback: impl _transform to parse nested fields

* Only yield records having a populated dealstage field

* update SAT for hubspot connector

* bump version

* update handling Rate Limiting for Hubspot

* code clean up

Co-authored-by: Jacob Avelino <jacobavelino@berkeley.edu>
Co-authored-by: Jacob Avelino <jacob7227@yahoo.com>
Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
 2021-06-16 16:12:35 +03:00
Jared Rhizor
b4793b2510 add AIRBYTE_ENTRYPOINT for kubernetes support (#3973) 
* add AIRBYTE_ENTRYPOINT for kubernetes support

* bump versions

* bump version in seed

* Update generic template

* keep scaffold sources at 0.1.0

* add missing newline

* handle python base versions correctly

* re-bump mysql and postgres sources

* re-bump snowflake destination

* add skip tests option

* switch to running tests

* reverse conditional to make it safer

* fix publish to include the test running

* fix iterable version

* fix file generation

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-06-09 13:01:45 -07:00
vovavovavovavova
123355bd04 secondary PR: update acceptance config variable (#3930) 2021-06-08 09:09:35 +03:00
Vadym
cea80b451e Hubspot: Added deals associations for company and contacts (#3850) 
* Added deals associations for company and contacts

* Fixing the format under client.py

Co-authored-by: Vadym <vege1wgw@gmail.com>

* Update deals schema.
Update deals expected records.

Co-authored-by: Stanley Ivan <salexand001@e.ntu.edu.sg>
Co-authored-by: stanleyivan96 <51161692+stanleyivan96@users.noreply.github.com>
 2021-06-07 15:58:30 +03:00
Dmytro
a1035e007b 📚 Update all Python connectors READMEs with minimum required version (#3760) 2021-06-02 12:35:29 +03:00
Eugene Kulak
c61fc3df07 SAT: migrate to CDK (#3640) 
* move SAT to CDK and fix image tag

* fix publish

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-05-28 04:32:15 -04:00
Charles
0df53170c9 Stop formatting python with spotless (#3388) 2021-05-13 17:46:34 -07:00
Eugene Kulak
752fe9172a add colors to tests when running inside docker (#3193) 
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-05-04 08:25:14 -04:00
Eugene Kulak
41ad3a9211 Standard Tests: fix incremental test and docs (#3161) 
* fix cursor comparision in the tests

+ update docs and templates

* format

* format

* fix colors

* update acceptance-test-docker.sh script

* fix colors in docker

* Update airbyte-integrations/connector-templates/source-python-http-api/integration_tests/acceptance.py

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/utils/json_schema_helper.py

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* address comments from @sherifnada

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-05-03 13:43:10 -04:00
Eugene Kulak
7f66569436 Standard Tests: allow specifying exact record matches #2186 (#2960) 
* records matching

+ extra_fields
+ exact_order
+ extra_records

* fix serialization before comparision

* fix pydantic validation

* fix reporting and enable for hubspot

* format

* update docs

* update examples

* fix template

* Update docs/contributing-to-airbyte/building-new-connector/source-acceptance-tests.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update docs/contributing-to-airbyte/building-new-connector/source-acceptance-tests.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-04-30 04:21:35 -04:00
Eugene Kulak
8b1ac13c7a Standard Tests: hotfix - upgrade PyYAML (#3123) 
* upgrade PyYAML

* format

* format java files too

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-04-28 19:04:25 -04:00
Eugene Kulak
88b77aa9f3 Standard tests: update template (#2937) 
* update template for python sources
* update templates and their docs
* update docs, rename folders, add acceptance script
* adding hooks for custom setup and teardown
+ improve plugin behaviour, enable it explicitly
+ add acceptance.py to run acceptance tests together with integration tests
+ remove helper acceptance-test-python.sh
+ fix gradle command for acceptance tests
+ update docs

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-04-28 18:11:48 -04:00
Sherif A. Nada
3f6126be6f Fix argument error in log call (#3087) (#3091) 
* Fix argument error in log call (#3087)

When attempting to use an OAuth access token with the Hubspot source,
the logging call would cause the process to crash with an error:

```
TypeError: info() takes 2 positional arguments but 3 were given
```

* bump

Co-authored-by: Nathan Yergler <nathan@yergler.net>
 2021-04-27 15:47:28 -07:00
Davin Chia
f660b0a946 Add template generation for Santa aka CDK. (#3034) 
Template generation for new Source using the Santa CDK - provide basic scaffolding for someone implementing a new source.

General approach is to buff up comments in the original SDK, and add TODOs with secondary comments in the generated stub methods, as well as links to existing examples (e.g. Stripe or ExchangeRate api) users can look at.

Checked in and added tests for the generated modules.
 2021-04-25 18:02:33 +08:00
Charles
f445fdb5b2 match styling for spotlessApply and format (#3017) 
* as a java developer I want to be able to run spotlessApply without changing styles in python code
 2021-04-23 09:21:41 -07:00
Eugene Kulak
1755847c5e Standard Tests: create modular standard test suite v2 scaffolding (#2702) 
* Standard Tests: Implement v2 test suites #2181 (#2792)
* rename standard tests (v2) to source_acceptance_test
 2021-04-20 15:26:16 -04:00
Sherif A. Nada
2d2d0173d6 Update codegen template READMEs (#2873) 2021-04-13 11:41:52 -07:00
Jared Rhizor
0609011e05 fix connector base versioning (#2726) 
* fix connector base versioning

* only run integration tests in non-bases directories

* upgrade datamodel-code-generator which coincidentally has already fixed this issue with generating List of Lists

* bump forgotten hubspot version

* fix build
 2021-04-02 22:46:06 -07:00
Christophe Duong
bc1547f185 Fix integration tests of connectors related to destination_sync_mode and sync_mode being required (#2688) 2021-04-01 09:58:38 +02:00
Sherif Nada
77b72dcf5b Revert "Revert "Merge branch 'master' of github.com:airbytehq/airbyte"" 
This reverts commit de66bf010d.
 2021-03-23 15:12:01 -07:00
Sherif Nada
de66bf010d Revert "Merge branch 'master' of github.com:airbytehq/airbyte" 
This reverts commit a3a27f2fa2, reversing
changes made to 5917feee28.
 2021-03-23 15:09:44 -07:00
Eugene Kulak
6436b460a6 Source Hubspot: hotfix client tests (#2577) 
* fix tests that try invalid API token

* format

* add error codes constants

* format

* bumping

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-03-23 14:13:48 -03:00
Eugene K
0b9b038ca4 🎉 New source: Native Hubspot connector (#2215) 
* #2150 Issue: created native connector with schema folder populated

* #2150 Issue: make format code

* first version

* fix few issues

* fix issues

* fix read issue

* format

* docs

* docker tags

* extend configured catalog for testing

* fix source definitions

* format

* fix call rate issue, add backoff for retry after

* add general backoff

* write secrets for new connector

* drop singer connector registration

* refactor streams, resolve properties in schemas at runtime

* replace deprecated endpoint for company contacts

* replace deprecated pipeline endpoint

* update comments

* update docs

* fix typo

* fix stream contact lists

* fix pagination and forms result fetching

* fix health_check

* format and update catalog

* revert changes

* drop singer based hubspot

* fix company contacts substream

* move deals to separate test

* fix deals tests

* remove dynamic fields from records

* move deals to catalog again

* extend CRMObjectStream with associations

* format

* update schemas with updated field, change engagement layout

* fix Campaign stream

* remove custom tests

* remove dependency

* remove oauth

* Source Hubspot: add incremental streams (#2425)

* add incremental

* add incremental

* polishing

* update docs

* fix docstring

* clean up

* fix incremental bookmark access

* fix incremental tests

* clean up

* add custom test for incremental, improve logging

* format

* Update airbyte-integrations/connectors/source-hubspot/source_hubspot/api.py

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Source Hubspot: best practices (#2537)

* fix error reporting and add unit tests

* fix test and refactor cursor fields

* format

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>

* restored configured_catalog.json

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
Co-authored-by: Eugene Kulak <widowmakerreborn@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-03-20 06:18:55 -03:00