1
0
mirror of synced 2025-12-21 19:11:14 -05:00
Commit Graph

182 Commits

Author SHA1 Message Date
Jared Rhizor
b4793b2510 add AIRBYTE_ENTRYPOINT for kubernetes support (#3973)
* add AIRBYTE_ENTRYPOINT for kubernetes support

* bump versions

* bump version in seed

* Update generic template

* keep scaffold sources at 0.1.0

* add missing newline

* handle python base versions correctly

* re-bump mysql and postgres sources

* re-bump snowflake destination

* add skip tests option

* switch to running tests

* reverse conditional to make it safer

* fix publish to include the test running

* fix iterable version

* fix file generation

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-06-09 13:01:45 -07:00
Dmytro
a1035e007b 📚 Update all Python connectors READMEs with minimum required version (#3760) 2021-06-02 12:35:29 +03:00
Davin Chia
9460cb4422 Update all of Pydantic to 1.6.2 per Dependabot. (#3408)
* Update all to 1.6.2.

* Publish new airbyte-cdk version.

* Use repr instead of str for exceptions.

* Use rc.

* Edit test.

* Bump for SAT.

* Format.
2021-05-14 16:44:49 +08:00
Charles
0df53170c9 Stop formatting python with spotless (#3388) 2021-05-13 17:46:34 -07:00
Davin Chia
f660b0a946 Add template generation for Santa aka CDK. (#3034)
Template generation for new Source using the Santa CDK - provide basic scaffolding for someone implementing a new source.

General approach is to buff up comments in the original SDK, and add TODOs with secondary comments in the generated stub methods, as well as links to existing examples (e.g. Stripe or ExchangeRate api) users can look at.

Checked in and added tests for the generated modules.
2021-04-25 18:02:33 +08:00
Charles
f445fdb5b2 match styling for spotlessApply and format (#3017)
* as a java developer I want to be able to run spotlessApply without changing styles in python code
2021-04-23 09:21:41 -07:00
Sherif A. Nada
643a2fc6bf Bump & Publish Google sheets connector (#2994) 2021-04-20 10:47:04 -07:00
Aneesh Makala
33c20b817c 🎉 Google sheets bugfix: handle duplicate sheet headers (#2905) 2021-04-20 09:50:21 -07:00
Sherif A. Nada
2d2d0173d6 Update codegen template READMEs (#2873) 2021-04-13 11:41:52 -07:00
Jared Rhizor
0609011e05 fix connector base versioning (#2726)
* fix connector base versioning

* only run integration tests in non-bases directories

* upgrade datamodel-code-generator which coincidentally has already fixed this issue with generating List of Lists

* bump forgotten hubspot version

* fix build
2021-04-02 22:46:06 -07:00
Christophe Duong
bc1547f185 Fix integration tests of connectors related to destination_sync_mode and sync_mode being required (#2688) 2021-04-01 09:58:38 +02:00
Christophe Duong
8a29584125 ☝🏼Destinations supports destination sync mode (#2460)
* Handle destination sync mode in destinations

* Source & Destination sync modes are required (#2500)

* Provide Migration script making sure it is always defined for previous sync configs
2021-03-26 20:23:48 +01:00
dependabot[bot]
60d7db9413 Bump pyyaml from 5.3.1 to 5.4 to avoid security warning. (#2621) 2021-03-27 01:29:36 +08:00
Deividas J
0c603db296 Fixed: typo in google sheets spec.json (#2604) 2021-03-24 16:26:50 +01:00
Sherif A. Nada
02a5b9b8a8 depend on published base singer/python images in existing connectors (#2469) 2021-03-15 18:21:05 -07:00
Sherif A. Nada
fe22594bb9 update all python Readmes to describe dev process without gradle (#2442) 2021-03-15 18:05:38 -07:00
Sherif A. Nada
2139a9656b Cleanup Python packages (#2440) 2021-03-15 01:24:21 -07:00
Christophe Duong
070575ffdf Protocol allows future / unknown properties (#2238)
* Allow new extra properties in validation
* Create migration script to upgrade all connectors versions
* Bumpversion of all connectors
2021-03-09 13:36:36 +01:00
Sherif A. Nada
ac0bcf7d01 Add comment at the top of all requirements.txt files clarifying they should not be used for dependencies (#2366) 2021-03-08 19:01:47 -08:00
Yevhenii
c2dab064dd #1740 Issue: fix failing Google Sheets Source with large spreadsheet (#1762)
Co-authored-by: Sherif Nada <snadalive@gmail.com>
2021-01-21 14:44:52 -08:00
Yevhenii
0eea5bc765 adopt Best practice for Google Sheets Source (#1668)
Co-authored-by: Sherif Nada <snadalive@gmail.com>
2021-01-18 10:24:55 -08:00
Yevhenii
d16ef618e5 Resolving issue #1353: implement backoff in Google Sheets (#1438)
* Issue #1353: implement backoff for integration tests

* update code for backoff HTTP error while read Google Sheet

* create Client class for Google Sheets with backoff all methods

* update Google Sheets Source after review #2

* update docker version for google_sheets_source
2020-12-30 17:13:20 -08:00
Christophe Duong
e92e9aa8c4 Refactor python sources (#1331)
* Make ConfigContainer internal to SingerSource only
2020-12-16 17:52:17 +01:00
Anurag870
f51c7a72ff Update documentation url in google sheets spec.json (#1292) 2020-12-11 09:02:14 -08:00
Sherif A. Nada
e8a332ae65 Standard source incremental tests (#1175) 2020-12-04 09:54:10 -08:00
Charles
02819a4b87 Incremental Docs and Data Model Update (#1021) 2020-11-19 22:07:32 -08:00
Sherif A. Nada
f4c3ac70f9 annotate secret fields (#1012) 2020-11-19 15:13:23 -08:00
Charles
e7edb2c858 Adding incremental to the catalog data model (#998)
* Add ConfiguredAirbyteCatalog and ConfiguredAirbyteStream
2020-11-18 14:15:59 -08:00
Charles
aafce4eadf google sheets manual test (#922) 2020-11-11 20:15:13 -08:00
Charles
148d6592bd fix syntax error in hrefs in specs (#921) 2020-11-11 16:28:08 -08:00
Jared Rhizor
e1d7add50a mostly incremental builds (#817)
* support cached builds for base -> base-java -> snowflake

* use plugin for image building

* fix matching on COPY from

* remove docker.gradle

* tmp commit

* update connectors

* finish rest of build files

* fix ide errors

* more build fixes

* clean up

* clean up for new soruces

* fix spotless

* fix flake problems

* add recommended empty file

* python caching

* fixes upon review

* clean up docker and build test files

* clean up python

* clean up

* fix integration test dependencies

* fix standard tests

* fix

* remove symlink

* re-add requirements to fix normalizatioon build

* fix symlink

* fix dumbest build problem of all

* add missing integration test def

* fix missing dep

* remove class exclusion

* move trim so null source versions are allowed

* rename map

* fix hardcoded value

* remove unnecessary dep

* use dashes for salesforce package name

* fix typo

* DRY and fix test image name

* Fix edit

* assert string is not empty

* build integration test image only for integrationTest

* move code generator to tools and rename docker build tasks

* make source test depend on integration test build, not the other way

* remove guard because the docker build should exist before the integrationtest is applied

* remove comment

* DRY up airbyte-source-test

* fix plugin compilation

* add missing dependency

* rename getTaggedImage to getDevTaggedImage

* fix test vs main docker build bug
2020-11-09 11:00:11 -08:00
Sherif A. Nada
f408bce2dc Add gsheets source (#695) 2020-10-29 08:50:12 -07:00