1
0
mirror of synced 2025-12-22 19:38:29 -05:00
Commit Graph

51 Commits

Author SHA1 Message Date
Natik Gadzhi
63ab010eaa docs: Connector Builder contribution flow text-based docs (#44616) 2024-08-27 00:42:47 +00:00
Audrey Maldonado
6c980bd8f9 Re-ordering some material and temporarily removing a document (#40612) 2024-06-28 21:17:48 +00:00
Audrey Maldonado
7cc188ed34 move disambiguation section for connectors, connections, and configur… (#39134) 2024-06-07 12:35:09 -04:00
Audrey Maldonado
a2816910df Update README.md with a link to the YAML schema reference (#38708) 2024-05-28 11:26:00 -03:00
Audrey Maldonado
f3c6ca4ca2 Edits for clarifying process, visual presentation, and removal of ina… (#38651)
Co-authored-by: Natik Gadzhi <natik@respawn.io>
2024-05-28 09:13:50 -04:00
Natik Gadzhi
cb80e6922a [tools] prettier rules for .md + formatting cleanup 2024-05-07 08:19:33 -07:00
Natik Gadzhi
fd451593f6 docs: Change connector development overview to link to the new guide (#36972) 2024-04-10 13:15:23 -07:00
Alexandre Girard
87f051c3a9 Add a walkthrough of building a custom Python connector (#36743)
Co-authored-by: Natik Gadzhi <natik@respawn.io>
2024-04-08 21:08:25 -07:00
Natik Gadzhi
0e9bdf314d connector-templates: move generated code from src so airbyte-ci builds it (#36428)
### What

For newly generated python and low-code connectors, move sources from `src` to `source_%CONN_NAME%` so `airbyte-ci build` can pick them up correctly. Closes #35893.

### What changed?

- Moved the sources from `src`
- Updated package paths in `pyproject.toml` templates

### How to test?

You can make a new connector with `./generate.sh` and verify that it works with Poetry and that it builds with `airbyte-ci build --name source-test`

---
2024-03-25 10:01:30 -07:00
Marius Posta
f90b5d1b62 docs: update and remove some bad content (#36197) 2024-03-15 13:12:37 -07:00
Natik Gadzhi
1d6b93d634 Updated connector development overview doc (#36010) 2024-03-13 18:05:16 -07:00
Cynthia Yin
7d89fc9112 📚 Destinations V2: update typing & deduping doc link (#34505) 2024-01-24 17:50:42 -06:00
Evan Tahler
dd445a254d Remove old normalization guide for building connectors (#32208) 2023-11-06 17:53:44 +00:00
Evan Tahler
cc5d54558f Docs: remove 'build a python destination' (#31394) 2023-10-13 16:31:40 +00:00
Purushottam Khedre
3b9892994d Update broken github connector link in connector-development readme (#31118)
Co-authored-by: Sajarin <sajarindider@gmail.com>
2023-10-09 02:35:25 -04:00
Augustin
5b8200181c connectors-ci: deprecate slash test (#27200) 2023-06-14 18:19:13 +02:00
Augustin
80032f73f9 connectors-ci: deprecate slash publish (#25865) 2023-05-22 10:10:56 +02:00
Ben Church
8557eb96d9 Remove autobump from /publish (#26166) 2023-05-16 18:44:37 -05:00
Marcos Marx
d0fa40c307 Update Troubleshooting documentation (#26048)
* remove unused docs and create troubleshooting main page

* add desc

* update our troubleshooting page

* add premium support mention

* update link to daily oh

* add notes of premium support in other pages

* remove ref in local deployment and sidebar
2023-05-15 16:52:08 -03:00
Ben Church
f2944dd59b Docs: Metadata add docs ahead of merge (#26015)
* Cherry pick docs files from  metadata-service/feat-remove-definitions

* Add link to checklist

* Fix docasaurus error
2023-05-11 16:10:13 -05:00
Joe Reuter
5ec81cc8d1 Connector builder: Restructure documentation (#25353)
* authentication documentation

* some fixes

* fix broken link

* remove session token auth

* review comments

* first version of tutorial

* Revert "first version of tutorial"

This reverts commit bbdeb997f2.

* first version of tutorial

* review comments

* improvements

* partitioning documentation

* incremental sync

* review comments

* improvements

* record processing doc

* improvements

* error handler connector builder ui documentation

* add missing link

* overview page

* mention current limitation

---------

Co-authored-by: brianjlai <brian.lai@airbyte.io>
2023-04-27 12:32:26 +02:00
Cynthia Yin
fd849d4a0c small docs fixes (#25377) 2023-04-21 10:49:57 -07:00
Ella Rohm-Ensing
eecb080446 update link (#25349) 2023-04-19 19:24:00 -07:00
Rodi Reich Zilberman
0bab1756b8 Rename airbyte-config module (#24885)
* rename airbyte-config module

* Automated Commit - Formatting Changes

* sanity

* update import

* update import

* update script

* update script

* update script

* update script

* Automated Change

* Automated Change

* Automated Change

* Automated Change

* update awsdatalake icon

* point slash commands to new path

* sanity

* Automated Commit - Formatting Changes

* sanity

* Automated Change

* Automated Change

* sanity

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
2023-04-06 10:47:30 -07:00
Sherif A. Nada
1feda0817e Rearrange connector contribution docs (#24004) 2023-03-13 17:01:36 -07:00
Evan Tahler
ced5309009 Update README.md (#23505) 2023-02-27 09:28:18 -08:00
Pedro S. Lopez
4991e95c65 📝 add docs for how to add normalization (#21563)
* add docs

* add schema link

* update based on feedback
2023-01-19 15:11:00 +00:00
Sherif A. Nada
f5d8e25519 Add connector builder UI docs (#19588) 2022-11-18 10:56:30 -08:00
Amruta Ranade
e37979b357 Added link to low-code connector framework 2022-10-03 19:28:29 -04:00
Jonathan Pearlin
735f77d9e9 Add changelog update to publishing steps (#14421)
* Add changelog update to publishing steps

* Remove OPTIONAL
2022-07-19 12:53:48 -04:00
Amruta Ranade
c1b9f82239 Docs: Fixed broken links (#14622)
* fixing broken links

* more broken links
2022-07-12 10:12:54 -04:00
Topher Lubaway
9c6c092a22 Revert "Improving docusaurus sidebar generation (#1927) (#14369)" (#14596)
This reverts commit a2c194a11f.
2022-07-11 15:27:14 -05:00
Mykyta Serbynevskiy
a2c194a11f Improving docusaurus sidebar generation (#1927) (#14369)
* Improving docusaurus sidebar generation (#1927)

* Added "Career & open positions" folder to sidebar, adjusted "Project overview" folder

* Deleted "career-and-open-positions" folder from sidebar
2022-07-08 14:18:27 -05:00
Sherif A. Nada
40b43adf73 Remove skiptests option (#14100) 2022-06-24 08:22:23 -07:00
George Claireaux
e7dae06168 Enhance /publish to allow for multiple connectors and parallel execution (#13864)
* start

* revert

* azblob

* bq

* bq denorm

* megapublish baaaabyyyy

* fix needs

* matrix connectors

* auto-bump connector version

* dont failfast and max parallel 5

* multi runno

* minor

* testing matrix agents

* name

* testing multi agents

* tmp fix

* new multi agents

* multi test

* tryy

* let's do this

* magico

* fix

* label test

* couple more connector bumps

* temp

* things

* check this

* lets gooo

* more connectors

* Delete TEMP-testing-command.yml

* auto-bump connector version

* added comment describing bash part

* running single thread

* catch sentry cli

* auto-bump connector version

* destinations

* + snowflake

* saved

* auto-bump connector version

* auto-bump connector version

* java source bumps

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* remove twice-defined methods

* label things

* revert action

* using the new test action

* point at action

* wrong tag on action

* update pool label

* update to use new ec2-github-runner fork

* this needs to be more generic than publisher

* change publish to run on pool

* add comment about runner-pool usage

* updated publish command docs for multi & parallel connector runs

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* unbump failed publish versions

* missed dockerfiles

* remove failed docs

* mssql fix

* overhauled the git comment output

* bumping a test connector that should work

* slight order switcheroo

* output connectors properly in first message

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-21 18:40:33 +01:00
Charles
22b727c0ea Update Airbyte Protocol Docs (#13709) 2022-06-14 15:04:38 -07:00
Evan Tahler
d7a1fe8224 Clarify what the /publish command does with the version (#13236)
I found this confusing in the past - do I need to bump the version in 2 places?
2022-05-27 09:30:27 -07:00
George Claireaux
ded2b65244 clarity on version bump with /publish (#11813) 2022-04-07 19:18:48 +01:00
George Claireaux
42fdebd1de 📖 Added section to docs about the /publish command (#11739)
* using yq + more success/failure comments

* added gradle process + git push

* dummy bump to test publish

* dummy bump to test publish

* dummy bump to test publish

* bump version

* fix connector variable

* bump version

* only git add necessary files

* remove git config

* bump version

* making octavia user

* version bump

* auto-bump connector version

* added docs and auto-bumo flag

* bump version

* separate IMAGE_NAME and IMAGE_VERSION env vars from sentry prep

* version bump

* auto-bump connector version

* added entry to apify changelog for clarity

* added details on /publish command

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-05 17:17:17 -07:00
George Claireaux
f7d3a706b2 Publish connector command now auto bumps version (#11712)
* using yq + more success/failure comments

* added gradle process + git push

* dummy bump to test publish

* dummy bump to test publish

* dummy bump to test publish

* bump version

* fix connector variable

* bump version

* only git add necessary files

* remove git config

* bump version

* making octavia user

* version bump

* auto-bump connector version

* added docs and auto-bumo flag

* bump version

* separate IMAGE_NAME and IMAGE_VERSION env vars from sentry prep

* version bump

* auto-bump connector version

* added entry to apify changelog for clarity

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-05 23:21:55 +01:00
Maksym Pavlenok
d3b0c998b8 🎉 CI secrets: full migration to GSM (#8561)
* add python packages for CI scripts

* add tox config for all subpackages

* draft version

* init venv for scripts

* fix venv

* remove used comments

* fix run test

* change base folder

* update secret format

* update docs

* remove an unused file

* remove github secrets logic fully

* fix base_folder balue

* add functions desc
2021-12-15 15:08:46 +02:00
Menno Hamburg
03e5160569 🎉 New CDK: Dotnet/.NET/C# implementation + free exchange rates source connector (#7945)
* Added cdk source

* Added free exchange rates api as a dotnet example

* Corrected the use of JsonDocument, properly enabled utf-8

* Fixed error catching

* Added cli, documentation and nuget improvements

* Small improvements where needed

* Removed obsolete files

* Update README.md

* Added documentation and reference

* Update README.md

* Corrected dotnet

* Update README.md
2021-12-13 20:24:09 -03:00
Lake Mossman
34565c6767 Add better command for generating seed specs to readme 2021-11-18 10:44:18 -08:00
Maksym Pavlenok
c0cc288f97 🎉 CI secrets: migration to GSM (#7528)
* fix 404 responses for the ticket_comments stream

* add unit test

* add unit test

* add oauth2 access token

* Update airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* switching among auth methods

* update spec file

* update CI secrets logic

* update CI secrets logic

* remove debug data

* add a debug message

* fix json convertation

* fix json convertation

* support one secret by several connectors

* Update tools/bin/ci_credentials.sh

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update tools/bin/ci_credentials.sh

Co-authored-by: LiRen Tu <tuliren@gmail.com>

* update function names

* update docs

* reset failed changes

* update json set value

* update secrets json key

* Update tools/bin/ci_credentials.sh


* add JSON validation

* Update docs/connector-development/README.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* update doc

* Update README.md

* rename the function write_standard_creds

Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: LiRen Tu <tuliren@gmail.com>
2021-11-10 10:49:44 +02:00
Lake Mossman
09fce77107 REDO: Generate seed connector specs on build (#7613)
* add specs module with logic to fetch specs on build

* format + build and add gradle dependency for new script

* check seed file for existing specs + refactor

* add tests + a bit more refactoring

* run gw format

* update yaml config persistence to merge specs into definitions

* add comment

* add dep

* add tests for GcsBucketSpecFetcher

* get rid of static block + format

* DRY up parse call

* add GCS details to comment

* formatting + fix test

* update comment

* do not format seed specs files

* change signature of run to allow cloud to reuse this script

* run gw format

* revert commits that change signature of run

* fix comment typo

Co-authored-by: Davin Chia <davinchia@gmail.com>

* rename enum to be distinct from the enum in cloud

* add missing dependencies between modules

* add readme for seed connector spec generator

* reword

* reference readme in comment

* ignore 'spec' field in newFields logic

* rearrange dependencies so that CONNECTORS_BASE build does not depend on SeedConnectorSpecGenerator

* run format

* add some more helpful info to the GCS fetch failure message

* add more info

* get rid of unnecessary static block

* Fix publishing docs (#7589)

* Fix publishing docs

* Reorder steps and add a comment about rebuilding the platform

* Update README.md

Co-authored-by: Lake Mossman <lake@airbyte.io>

* add dependency and rebuild

* update PR template with seed connector generation steps

* revert formatting changes to PR template

* Update build.gradle

* Remove unnecessary dep

Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
2021-11-04 17:06:16 -07:00
Abhi Vaidyanatha
ae32ecbb27 GitBook: [master] 186 pages and 77 assets modified 2021-10-08 21:17:47 +00:00
Abhi Vaidyanatha
516d276893 Add Faros CDK information to CDK docs (#5686)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-08-27 18:36:37 -07:00
Jenny Brown
6a3b594574 Clarified wording that confused a new connector developer (#5558) 2021-08-25 16:09:49 -05:00
George Claireaux
2fe0043014 Temporary solution for hitting secrets limit (#5085)
* add more-secrets environment to publish and test workflows

* add link to more-secrets environment in relevant docs location
2021-07-29 22:23:46 +01:00
Abhi Vaidyanatha
211383d456 Fix broken links in (#5059)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-28 14:14:23 -07:00