1
0
mirror of synced 2026-01-09 15:05:02 -05:00
Commit Graph

67 Commits

Author SHA1 Message Date
Jared Rhizor
1affb14827 even more link fixes (#7148)
* even more link fixes

* add remaining exchangerates items
2021-10-20 12:56:55 -07:00
Michel Tricot
1773e41e47 Shorten our headers + adds contributors file (#6478) 2021-09-27 10:45:50 -07:00
Dmytro
60fd9d5ab9 🐛 Fix hubspot datetime empty string (#5798)
* Fix hubspot datetime empty string
2021-09-06 10:11:48 +03:00
Yaroslav Dudar
8ddce6f355 🎉 Python CDK: Allow to ignore http status errors and override retry parameters (#5363)
added auto_fail_on_errors, max_retries, retry_factor properties to python cdk
2021-08-25 10:31:24 +03:00
LiRen Tu
2906ec287a CI: Add action to check broken doc links (#5254)
* Add action to check broken doc links

* Ignore localhost

* Update config

* Fix broken links

* Use quiet mode

* Ignore PR link

* Fix more broken links

* Fix more broken links

* Fix more broken links

* Verify pattern

* Fix more broken links

* Separate full and pr check

* Update pattern

* Test invalid link

* Remove invalid link
2021-08-07 14:28:02 -07:00
Sherif A. Nada
45ee6234f2 Only run pytest if test files are found in the test directories (#5012) 2021-07-27 08:33:51 -07:00
Davin Chia
71637a27c8 Improve Python Build. (#4540)
* Remove old pip build command. Deconflict Pydantic.

* Run test on ec2 instance.
2021-07-05 16:25:33 +08:00
Marcos Marx
d269f91871 correct url (#4293) 2021-06-24 21:56:08 -03:00
Dmytro
d5d0c8b252 Replace npm install + generate -> ./generate.sh in docs (#4048) 2021-06-11 02:17:35 +03:00
Jared Rhizor
b4793b2510 add AIRBYTE_ENTRYPOINT for kubernetes support (#3973)
* add AIRBYTE_ENTRYPOINT for kubernetes support

* bump versions

* bump version in seed

* Update generic template

* keep scaffold sources at 0.1.0

* add missing newline

* handle python base versions correctly

* re-bump mysql and postgres sources

* re-bump snowflake destination

* add skip tests option

* switch to running tests

* reverse conditional to make it safer

* fix publish to include the test running

* fix iterable version

* fix file generation

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-06-09 13:01:45 -07:00
Dmytro
66d96967df 🎉 Containerize connector code generator (#3763) 2021-06-08 12:25:27 +03:00
Dmytro
a1035e007b 📚 Update all Python connectors READMEs with minimum required version (#3760) 2021-06-02 12:35:29 +03:00
Sherif A. Nada
137845a28e rename toy connector tutorial to "Build a connector the hard way" (#3421) 2021-05-17 13:31:25 -07:00
Sherif A. Nada
6beb4eb7e7 add npm install before all npm run generates' (#3442) 2021-05-17 09:39:37 -07:00
Davin Chia
9460cb4422 Update all of Pydantic to 1.6.2 per Dependabot. (#3408)
* Update all to 1.6.2.

* Publish new airbyte-cdk version.

* Use repr instead of str for exceptions.

* Use rc.

* Edit test.

* Bump for SAT.

* Format.
2021-05-14 16:44:49 +08:00
Charles
0df53170c9 Stop formatting python with spotless (#3388) 2021-05-13 17:46:34 -07:00
Davin Chia
ced243881b Add CDK publishing setup. (#3156)
Allow us to publish CDK versions on PyPi via Github workflows.

This lets us:
- Version CDK moving forward
- Publish via Github actions to both PyPi and test PyPi for test releases.

Note, this will not work on this branch as Github only detects new workflows after they are checked into master, so I was forced to develop and test this from a fork. Browse this PR or this PR to see this command in action.

Added publishing instructions to the Release Document.
2021-05-03 08:38:47 +08:00
Abhi Vaidyanatha
d940c78307 Add Python CDK Alpha Tutorial to Docs (#3147)
* Fix Transformations article link

* Add Python CDK Alpha documentation.

* Make requested changes, add to SUMMARY.md

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-04-30 11:10:04 -07:00
Davin Chia
fc00d36d0c Add Primary Key to CDK. (#3105)
Hold off on publishing the version and updating various downstream consumers for now since a major refactor is happening.
2021-04-30 14:49:19 +08:00
Davin Chia
fd0d253373 Release CDK version. (#3130) 2021-04-29 07:27:28 -07:00
Davin Chia
bd99b5ba8c Hotfix: CDK make sure to export OAuth2Authenticator. (#3129) 2021-04-29 22:01:47 +08:00
Sherif A. Nada
80ce46f357 Base-python: fix exception when reading schemas from shared/ folder (#3127) 2021-04-29 00:38:54 -07:00
Davin Chia
72c237d164 Fix build. (#3104) 2021-04-28 14:14:37 +08:00
Davin Chia
2188201c7c Upgrade to Pip 21.1. (#3070)
Upgrade to Pip 21.1 in preparation to upgrade DBT 19.

The latest Pip has

- a stricter resolver
- access to more up to date Python wheels (e.g. #2267 is solved since as any version after Pip 21 has access to ready-compiled Python wheels, see https://issues.apache.org/jira/browse/ARROW-11835).
- in theory faster pip installs (https://github.com/pypa/pip/issues/9187#issuecomment-826037738 makes me think the current Pip version is stable enough be consumed)

Also relax the constraint for `requests` and `backoff` since both were actually specifying incompatible versions that the old Pip resolver was silently swallowing.

Test this locally first by running git clean -dfx to remove all the existing venv files. The two different Pip versions do not work well.
2021-04-28 06:40:31 +08:00
Davin Chia
52df90b458 Proof read CDK Tutorial docs. (#3085) 2021-04-27 09:36:15 -07:00
Sherif A. Nada
4c96499ffd Add README.md for base-python (#3080) 2021-04-27 00:30:04 -07:00
Sherif A. Nada
4e9f4bdd06 CDK: tutorial for implementing an HTTP source (#3079) 2021-04-27 00:28:10 -07:00
Davin Chia
9f5719f3a8 CDK Conceptual Docs (#3048)
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-04-26 15:33:42 -07:00
Sherif A. Nada
bf1ac1ec3a document how to set stream schemas in CDK 2021-04-26 14:52:12 -07:00
Davin Chia
f660b0a946 Add template generation for Santa aka CDK. (#3034)
Template generation for new Source using the Santa CDK - provide basic scaffolding for someone implementing a new source.

General approach is to buff up comments in the original SDK, and add TODOs with secondary comments in the generated stub methods, as well as links to existing examples (e.g. Stripe or ExchangeRate api) users can look at.

Checked in and added tests for the generated modules.
2021-04-25 18:02:33 +08:00
Sherif Nada
49296ef657 format 2021-04-23 16:12:55 -07:00
Sherif A. Nada
6bec66ed8d santa: rename sdk --> cdk (#3040) 2021-04-23 15:07:27 -07:00
Charles
f445fdb5b2 match styling for spotlessApply and format (#3017)
* as a java developer I want to be able to run spotlessApply without changing styles in python code
2021-04-23 09:21:41 -07:00
Sherif A. Nada
3827235af0 Santa: Implement OAuth (#3000)
Basic OAuthAuthenticator interface.

Co-authored-by: Davin Chia <davinchia@gmail.com>
2021-04-23 10:12:13 +08:00
Davin Chia
3d2d20b87e Implement basic ExchangeRate API using Santa. (#2942)
Take Santa for a test drive.

Slight modification to Santa framework.
2021-04-21 17:09:39 +08:00
Eugene Kulak
1755847c5e Standard Tests: create modular standard test suite v2 scaffolding (#2702)
* Standard Tests: Implement v2 test suites #2181 (#2792)
* rename standard tests (v2) to source_acceptance_test
2021-04-20 15:26:16 -04:00
Sherif A. Nada
8d1a027c86 CDK: Add stream slicing and Slack source (#2924) 2021-04-19 21:25:07 -07:00
Yevhenii
6236ff0a05 Initialize packages inside base_python with bumping (#2882)
Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
2021-04-14 18:30:55 +03:00
Yevhenii
07748236c7 bump base-python version to 0.1.2 (#2872)
Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
2021-04-13 21:43:11 +03:00
Davin Chia
a7b9cd5b7a Remove unused schema parameter and cleanup. (#2780) 2021-04-07 19:58:11 +08:00
Sherif A. Nada
94093f0df9 Project Santa v0.1 (#2634) 2021-04-07 00:21:25 -07:00
Jared Rhizor
0609011e05 fix connector base versioning (#2726)
* fix connector base versioning

* only run integration tests in non-bases directories

* upgrade datamodel-code-generator which coincidentally has already fixed this issue with generating List of Lists

* bump forgotten hubspot version

* fix build
2021-04-02 22:46:06 -07:00
jrhizor
1eadbb0326 hotfix: formatting 2021-04-02 09:12:34 -07:00
Nathan Nowack
317e8b44ff raise FAIL after stream-read exception (#2695) 2021-04-01 14:47:14 -07:00
dependabot[bot]
60d7db9413 Bump pyyaml from 5.3.1 to 5.4 to avoid security warning. (#2621) 2021-03-27 01:29:36 +08:00
Sherif A. Nada
e87468d4bc improve instructions in templates & publish base docker images to dockerhub (#2441) 2021-03-15 02:16:36 -07:00
Eugene K
a7761e792e Source Freshdesk: handle rate limiting #1972 (#2060)
Co-authored-by: Sherif Nada <snadalive@gmail.com>
2021-02-17 09:00:28 -08:00
Eugene K
a45dbd3097 Source Freshdesk: support incremental sync and add conversations and satisfaction ratings streams (#2052)
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-02-16 10:55:03 -08:00
Sherif A. Nada
8dc6c7c234 Source Facebook Marketing: support insights streams via async queries (#1990) 2021-02-15 01:23:34 -08:00
Eugene K
9b099942d8 Source FB Marketing: add incremental support (#1699)
Co-authored-by: Sherif Nada <snadalive@gmail.com>
2021-01-22 15:34:06 -08:00