1
0
mirror of synced 2026-01-03 06:02:23 -05:00
Commit Graph

17 Commits

Author SHA1 Message Date
George Claireaux
e7dae06168 Enhance /publish to allow for multiple connectors and parallel execution (#13864)
* start

* revert

* azblob

* bq

* bq denorm

* megapublish baaaabyyyy

* fix needs

* matrix connectors

* auto-bump connector version

* dont failfast and max parallel 5

* multi runno

* minor

* testing matrix agents

* name

* testing multi agents

* tmp fix

* new multi agents

* multi test

* tryy

* let's do this

* magico

* fix

* label test

* couple more connector bumps

* temp

* things

* check this

* lets gooo

* more connectors

* Delete TEMP-testing-command.yml

* auto-bump connector version

* added comment describing bash part

* running single thread

* catch sentry cli

* auto-bump connector version

* destinations

* + snowflake

* saved

* auto-bump connector version

* auto-bump connector version

* java source bumps

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* remove twice-defined methods

* label things

* revert action

* using the new test action

* point at action

* wrong tag on action

* update pool label

* update to use new ec2-github-runner fork

* this needs to be more generic than publisher

* change publish to run on pool

* add comment about runner-pool usage

* updated publish command docs for multi & parallel connector runs

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* unbump failed publish versions

* missed dockerfiles

* remove failed docs

* mssql fix

* overhauled the git comment output

* bumping a test connector that should work

* slight order switcheroo

* output connectors properly in first message

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-21 18:40:33 +01:00
Charles
22b727c0ea Update Airbyte Protocol Docs (#13709) 2022-06-14 15:04:38 -07:00
Evan Tahler
d7a1fe8224 Clarify what the /publish command does with the version (#13236)
I found this confusing in the past - do I need to bump the version in 2 places?
2022-05-27 09:30:27 -07:00
George Claireaux
ded2b65244 clarity on version bump with /publish (#11813) 2022-04-07 19:18:48 +01:00
George Claireaux
42fdebd1de 📖 Added section to docs about the /publish command (#11739)
* using yq + more success/failure comments

* added gradle process + git push

* dummy bump to test publish

* dummy bump to test publish

* dummy bump to test publish

* bump version

* fix connector variable

* bump version

* only git add necessary files

* remove git config

* bump version

* making octavia user

* version bump

* auto-bump connector version

* added docs and auto-bumo flag

* bump version

* separate IMAGE_NAME and IMAGE_VERSION env vars from sentry prep

* version bump

* auto-bump connector version

* added entry to apify changelog for clarity

* added details on /publish command

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-05 17:17:17 -07:00
George Claireaux
f7d3a706b2 Publish connector command now auto bumps version (#11712)
* using yq + more success/failure comments

* added gradle process + git push

* dummy bump to test publish

* dummy bump to test publish

* dummy bump to test publish

* bump version

* fix connector variable

* bump version

* only git add necessary files

* remove git config

* bump version

* making octavia user

* version bump

* auto-bump connector version

* added docs and auto-bumo flag

* bump version

* separate IMAGE_NAME and IMAGE_VERSION env vars from sentry prep

* version bump

* auto-bump connector version

* added entry to apify changelog for clarity

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-05 23:21:55 +01:00
Maksym Pavlenok
d3b0c998b8 🎉 CI secrets: full migration to GSM (#8561)
* add python packages for CI scripts

* add tox config for all subpackages

* draft version

* init venv for scripts

* fix venv

* remove used comments

* fix run test

* change base folder

* update secret format

* update docs

* remove an unused file

* remove github secrets logic fully

* fix base_folder balue

* add functions desc
2021-12-15 15:08:46 +02:00
Menno Hamburg
03e5160569 🎉 New CDK: Dotnet/.NET/C# implementation + free exchange rates source connector (#7945)
* Added cdk source

* Added free exchange rates api as a dotnet example

* Corrected the use of JsonDocument, properly enabled utf-8

* Fixed error catching

* Added cli, documentation and nuget improvements

* Small improvements where needed

* Removed obsolete files

* Update README.md

* Added documentation and reference

* Update README.md

* Corrected dotnet

* Update README.md
2021-12-13 20:24:09 -03:00
Lake Mossman
34565c6767 Add better command for generating seed specs to readme 2021-11-18 10:44:18 -08:00
Maksym Pavlenok
c0cc288f97 🎉 CI secrets: migration to GSM (#7528)
* fix 404 responses for the ticket_comments stream

* add unit test

* add unit test

* add oauth2 access token

* Update airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* switching among auth methods

* update spec file

* update CI secrets logic

* update CI secrets logic

* remove debug data

* add a debug message

* fix json convertation

* fix json convertation

* support one secret by several connectors

* Update tools/bin/ci_credentials.sh

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update tools/bin/ci_credentials.sh

Co-authored-by: LiRen Tu <tuliren@gmail.com>

* update function names

* update docs

* reset failed changes

* update json set value

* update secrets json key

* Update tools/bin/ci_credentials.sh


* add JSON validation

* Update docs/connector-development/README.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* update doc

* Update README.md

* rename the function write_standard_creds

Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: LiRen Tu <tuliren@gmail.com>
2021-11-10 10:49:44 +02:00
Lake Mossman
09fce77107 REDO: Generate seed connector specs on build (#7613)
* add specs module with logic to fetch specs on build

* format + build and add gradle dependency for new script

* check seed file for existing specs + refactor

* add tests + a bit more refactoring

* run gw format

* update yaml config persistence to merge specs into definitions

* add comment

* add dep

* add tests for GcsBucketSpecFetcher

* get rid of static block + format

* DRY up parse call

* add GCS details to comment

* formatting + fix test

* update comment

* do not format seed specs files

* change signature of run to allow cloud to reuse this script

* run gw format

* revert commits that change signature of run

* fix comment typo

Co-authored-by: Davin Chia <davinchia@gmail.com>

* rename enum to be distinct from the enum in cloud

* add missing dependencies between modules

* add readme for seed connector spec generator

* reword

* reference readme in comment

* ignore 'spec' field in newFields logic

* rearrange dependencies so that CONNECTORS_BASE build does not depend on SeedConnectorSpecGenerator

* run format

* add some more helpful info to the GCS fetch failure message

* add more info

* get rid of unnecessary static block

* Fix publishing docs (#7589)

* Fix publishing docs

* Reorder steps and add a comment about rebuilding the platform

* Update README.md

Co-authored-by: Lake Mossman <lake@airbyte.io>

* add dependency and rebuild

* update PR template with seed connector generation steps

* revert formatting changes to PR template

* Update build.gradle

* Remove unnecessary dep

Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
2021-11-04 17:06:16 -07:00
Abhi Vaidyanatha
ae32ecbb27 GitBook: [master] 186 pages and 77 assets modified 2021-10-08 21:17:47 +00:00
Abhi Vaidyanatha
516d276893 Add Faros CDK information to CDK docs (#5686)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-08-27 18:36:37 -07:00
Jenny Brown
6a3b594574 Clarified wording that confused a new connector developer (#5558) 2021-08-25 16:09:49 -05:00
George Claireaux
2fe0043014 Temporary solution for hitting secrets limit (#5085)
* add more-secrets environment to publish and test workflows

* add link to more-secrets environment in relevant docs location
2021-07-29 22:23:46 +01:00
Abhi Vaidyanatha
211383d456 Fix broken links in (#5059)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-28 14:14:23 -07:00
Abhi Vaidyanatha
3765bcf3b8 Refactor docs to improve clarity around connector development. (#5037)
* Refactor docs to improve clarity around connector development.

* Merge architecture and setup of source acceptance tests to reference

* Address reviewsa

* Address review comments

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-28 14:00:08 -07:00