jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2026-01-09 06:03:17 -05:00
Code Issues Packages Projects Releases Wiki Activity
23,725 Commits 5,232 Branches 518 Tags
dfdfbb0ebbdd94d84cbbbb20d5cbc39f4e7edb0b
Commit Graph

96 Commits

Author SHA1 Message Date
Aldo Gonzalez
ad08bcf99c Python Connector Tutorial: Update some instructions and libraries (#38828) 2024-09-21 18:13:12 -06:00
Natik Gadzhi
cb80e6922a [tools] prettier rules for .md + formatting cleanup 2024-05-07 08:19:33 -07:00
Przemysław Dąbek
66394a80c0 [connector-template] macro to use current year for new connectors (#37397) 2024-04-18 17:15:04 -07:00
Natik Gadzhi
3b50b0d00c connector-templates: Unpinning pytest and upgrading to recent base image version (#36596) 2024-03-27 22:13:27 -07:00
Natik Gadzhi
0e9bdf314d connector-templates: move generated code from src so airbyte-ci builds it (#36428) 
### What

For newly generated python and low-code connectors, move sources from `src` to `source_%CONN_NAME%` so `airbyte-ci build` can pick them up correctly. Closes #35893.

### What changed?

- Moved the sources from `src`
- Updated package paths in `pyproject.toml` templates

### How to test?

You can make a new connector with `./generate.sh` and verify that it works with Poetry and that it builds with `airbyte-ci build --name source-test`

---
 2024-03-25 10:01:30 -07:00
Augustin
62ba6c9e2a update connector generators to use poetry (#35411) 2024-02-29 18:55:00 +01:00
Joe Reuter
264043f4c2 Connector templates: Publish by default (#34766) 2024-02-06 19:29:16 +01:00
Mostafa Kamal
29852cf081 add run method in run.py (#34241) 2024-01-15 15:41:49 +01:00
Joe Reuter
7fe72581b2 Prepare Google Sheets, Google Drive, Apify dataset and connector templates for airbyte-lib (#33414) 2023-12-19 11:54:37 +01:00
Augustin
050f25db2f 🧹 update connectors templates to reflect airbyte-ci introduction (#32211) 
Co-authored-by: alafanechere <alafanechere@users.noreply.github.com>
 2023-11-07 11:04:12 +01:00
Augustin
44baf92d98 connector-generator: remove dockerfile and use base image for python connectors (#31491) 
Co-authored-by: alafanechere <alafanechere@users.noreply.github.com>
 2023-10-18 11:49:02 +02:00
Marius Posta
f8edc18039 airbyte-ci,gradle: replace airbyte-docker with airbyte-ci (#30743) 2023-10-04 08:38:17 -07:00
Ben Church
5c56ac1d84 Airbyte-ci: Remove gradle task connectorAcceptanceTest (#30326) 2023-09-19 15:16:37 -05:00
Alexandre Girard
3db13e0e8d Update python template (#29370) 
* Update python template

* Update source-python-http-api too

* Also update the low-code template
 2023-08-11 10:53:03 -07:00
Augustin
00d9462216 cat/connectors-ci: replace docker runner with dagger runner in CAT (#28000) 2023-08-11 17:58:48 +02:00
Ben Church
e9490e3fb6 Connector Levels: Add new internal metadata fields (#28904) 
* Add airbyte internal

* Add tests

* First pass

* Set destinations to same levels as sources

* Best guess at missing statuses

* Best guess at _ql

* Add separate enum class

* Fix support level name

* Update templates

* Add one more test
 2023-08-01 18:08:33 -05:00
Ben Church
1c871756b3 Allow empty reg in registry entry generation (#28307) 
* Handle empty or missing registries

* Update scaffolds to include cloud reg
 2023-07-14 15:26:04 -07:00
Ben Church
1dabc6208e Metadata: add tags field (#26320) 
* Add optional tags field

* Remove duplicate icons

* Add programming tags to all

* Update docs

* supportUrl -> documentationUrl

* Ensure one language tag is applied

* Add keyvalue check

* rebase and fix tests

* Format

* Add cache buster

* Improve test

* Automated Commit - Formatting Changes

* Update error

* Fix missing tags

* Fix scaffold

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
 2023-05-26 16:13:09 -07:00
Augustin
7310494846 qa-checks: check metadata version matches dockerfile version (#25661) 2023-05-04 16:08:19 -07:00
Augustin
e888d63200 metadata: generate metadata.yaml for new connectors. (#25549) 2023-04-28 16:25:01 +00:00
Catherine Noll
7da6a3bb77 Run CATs with local CDK (#23084) 
Scripts to 

* Run CATs against the local CDK for one connector

* Run CATs against the local CDK for multiple connectors

* Create a connecter image with the local CDK

---------

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2023-02-24 16:13:42 -05:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594) 
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
 2023-02-08 13:01:16 -08:00
Joe Reuter
6a10ae3e05 Rename source acceptance test to connector acceptance test (#21846) 
Rename source acceptance test to connector acceptance test
 2023-02-02 11:38:19 +01:00
Marcos Marx
05fec07428 Connector Generator: remove catalog.json file (#18265) 
* remove catalgo from generator

* regenerate scaffold connectors
 2022-12-08 17:25:32 -03:00
Scott Lee Chua
a821d07728 Low-code CDK: Update autogenerated docs (#19786) 
* Add integration tests command

Add commands to run integration tests using Docker, referencing https://airbytehq.github.io/connector-development/config-based/tutorial/testing

* Add autogenerating documentation URL to spec.yaml template

* Replace docs.airbyte.io with docs.airbyte.com

* Update test command

* run gradlew regenrate scaffold file

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
 2022-12-08 17:15:08 -03:00
Evan Tahler
087018799a Use pytest v6.2.5 (#19848) 
* Use pytest v6.2.5

* bump test definitions
 2022-11-28 17:57:06 -08:00
Brian Lai
3521802545 Update Python Source connectors to use the latest airbyte-cdk minor version (#18261) 
* update all connectors currently only adhering to patch versions to pull latest minor version

* also add greenhouse

* fix scaffolds
 2022-10-20 16:06:02 -04:00
Augustin
4df0a48427 cdk and docs: remove "additionalProperties" (#14881) 2022-07-22 15:34:27 +02:00
Denis Davydov
426e9594cb fix build: run gradlew format (#13265) 
* fix build: run gradlew format

* Revert "fix build: run gradlew format"

This reverts commit e679b4f26f.

* fix connector template to fix the build
 2022-05-27 17:36:43 +03:00
Pedro S. Lopez
6119da60cb Generated python connectors use CDK version with YAML spec support (#12827) 
* bump cdk version for generated python source connectors

* update scaffold connectors
 2022-05-13 20:58:17 -04:00
Daniel Stevenson
22cebe4b5b CDK: DRY acceptance test templates (#12067) 2022-05-02 09:08:50 +02:00
Pedro S. Lopez
0c12ad9136 python generators output spec.yaml files (#12245) 
* generators output spec.yaml files

* source-singer generator also uses spec.yaml

* update scaffold

* update python cdk tutorials to use spec.yaml

* add docs updates

* consistency
 2022-04-26 07:16:04 -07:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763) 
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
 2022-04-11 20:51:37 -07:00
Tim Roes
2855dfcdce Upgrade plop in connector generators (#10578) 
* Upgrade plop

* Remove scaffolded code

* Build fixes

* Remove scaffolded code

* Revert "Remove scaffolded code"

This reverts commit 3911f527f8.

* Revert "Remove scaffolded code"

This reverts commit 549f790e3c.

* Remove .gitignore changes

* Remove .gitignore changes

* Update scaffold generated code

* Replace titleCase with capitalCase (#10654)

* Add capitalCase helper

* Replace titleCase with capitalCase

* Update generated scaffold files

Co-authored-by: LiRen Tu <tuliren.git@outlook.com>
 2022-02-28 08:05:14 +01:00
Maksym Pavlenok
64bd0a6cd8 🐛 Source Intercom: backoff for companies' scrolling (#8395) 
* backoff for companies scroll

* remove a unused companies stream property

* fix tests

* bump version

* update source_specs
 2021-12-03 01:17:54 +02:00
Yevhenii
a39119427a 🎉 Python Connector template: Remove Deprecated Dependency for Python Sources (#7385) 
* Remove Deprecated Dependency for Python API Template
 2021-11-17 11:38:59 +02:00
Marcos Marx
655df2cf86 Fix scaffold format files (#7989) 
* run scaffold format

* format generator files

* correct singer too
 2021-11-15 17:43:09 -03:00
Sherif A. Nada
5f03d32797 fix buffered stream consumer tests (#7834) 2021-11-11 08:25:52 -08:00
Marcos Marx
10434372de Change license in connector generator base files (#6522) 
* change license in generator base files

* run gradlew scaffold

* format eof

* add eof singer base

* remove license from test_* http source

* format test_incremental http source
 2021-09-29 04:45:42 -03:00
Maksym Pavlenok
802a8184cd 🐛 Python Alpine tempales: add build-base packages (#6457) 
Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
 2021-09-28 11:43:51 +03:00
Maksym Pavlenok
f76ca9f215 🐛 Python tempales: add default timezone for Alpine images (#5702) 
Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
 2021-08-27 19:01:36 +03:00
Dmytro
3630cdeded 🎉 New Source: Amazon Ads (initial version) (#5023) 
* 3772: Amazon Ads source, initial version


Co-authored-by: Dmytro Rezchykov <dmitry.rezchykov@zazmic.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-08-26 11:15:38 +03:00
Maksym Pavlenok
70713902cd 🚀 Dockerfile Python templates: migration to Alpine Docker image (#5397) 
* update Dockerfile templates

* update dockerfile for python desctinations

* add bash for all python Dockerfile templates

Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
 2021-08-18 22:42:46 +03:00
LiRen Tu
da98e5784d Clean up json schema in code generators (#5383) 
Follow up on #4907.
 2021-08-12 14:40:38 -07:00
LiRen Tu
2906ec287a CI: Add action to check broken doc links (#5254) 
* Add action to check broken doc links

* Ignore localhost

* Update config

* Fix broken links

* Use quiet mode

* Ignore PR link

* Fix more broken links

* Fix more broken links

* Fix more broken links

* Verify pattern

* Fix more broken links

* Separate full and pr check

* Update pattern

* Test invalid link

* Remove invalid link
 2021-08-07 14:28:02 -07:00
Sherif A. Nada
b86ebf63fb Automatically pull latest SAT & build connector image in SAT runner (#4969) 2021-07-26 11:56:19 -07:00
Eugene Kulak
0f51edbd60 SAT: Replace validate_output_from_all_streams with empty_streams param (#4897) 
* Replace validate_output_from_all_streams with empty_streams param

* update docs

* fix comment

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-07-22 16:26:35 -04:00
Sherif A. Nada
a4bb30430d Python Demo Destination: KVDB (#4786) 2021-07-19 09:44:36 -07:00
Davin Chia
5f7d36efa6 🐛 Add missing dependencies for acceptance tests to run. (#4808) 2021-07-19 12:24:07 +08:00
Jared Rhizor
b4793b2510 add AIRBYTE_ENTRYPOINT for kubernetes support (#3973) 
* add AIRBYTE_ENTRYPOINT for kubernetes support

* bump versions

* bump version in seed

* Update generic template

* keep scaffold sources at 0.1.0

* add missing newline

* handle python base versions correctly

* re-bump mysql and postgres sources

* re-bump snowflake destination

* add skip tests option

* switch to running tests

* reverse conditional to make it safer

* fix publish to include the test running

* fix iterable version

* fix file generation

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-06-09 13:01:45 -07:00