1
0
mirror of synced 2025-12-21 11:01:41 -05:00
Commit Graph

106 Commits

Author SHA1 Message Date
Augustin
c1bfd7624c 🐙 source-google-directory: run up-to-date pipeline [2024-06-22] (#39961) 2024-06-23 02:26:15 +02:00
Natik Gadzhi
1090fd60cf source-google-directory: [autopull] base image + poetry + up_to_date (#38449)
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Danylo Jablonski <150933663+DanyloGL@users.noreply.github.com>
2024-06-18 12:27:19 +02:00
Augustin
31c95da499 [skip ci] Add connectorTestSuitesOptions to metadata (#38188) 2024-05-15 15:04:10 -05:00
Natik Gadzhi
cb80e6922a [tools] prettier rules for .md + formatting cleanup 2024-05-07 08:19:33 -07:00
Ella Rohm-Ensing
e766ba3a9e Python CDK: remove "deprecated" package (#37497) 2024-04-23 13:23:05 -05:00
Augustin
8c862a8013 Fix tags in metadata.yaml files: declare cdk tags (#35670) 2024-03-07 04:32:29 -06:00
Joe Reuter
6660c45adc Publish to pypi batch5 (#34668) 2024-01-31 05:10:46 -06:00
Joe Reuter
4cfaebeacf airbyte-lib: Refactor follow-up (#34649) 2024-01-30 12:11:41 +01:00
Joe Reuter
f29234a2db airbyte-lib: Refactor connectors (#34552) 2024-01-30 03:22:13 -06:00
Augustin
368ba78b64 🧹doc: update connectors README and remove acceptance-test-docker.sh (#32209) 2023-11-06 12:36:07 -06:00
Marius Posta
7ae97175a6 gradle: fix repo wide behaviour (#30607) 2023-09-28 05:01:13 -07:00
Ben Church
5c56ac1d84 Airbyte-ci: Remove gradle task connectorAcceptanceTest (#30326) 2023-09-19 15:16:37 -05:00
Marius Posta
f5c7c1c0b8 chore: get ./gradlew format to pass for the whole repo (same java style) (#29786) 2023-08-24 05:09:42 -05:00
Ben Church
40781313da Update Internal Fields: update ql levels to better resemble previous high strictness (#29450) 2023-08-15 18:03:10 -05:00
Augustin
00d9462216 cat/connectors-ci: replace docker runner with dagger runner in CAT (#28000) 2023-08-11 17:58:48 +02:00
Ben Church
2f7deaee02 [skip ci] Metadata: Remove leading underscore (#29024)
* DNC

* Add test models

* Add model test

* Remove underscore from metadata files

* Regenerate models

* Add test to check for key transformation

* Allow additional fields on metadata

* Delete transform
2023-08-03 10:56:13 -07:00
Ben Church
e9490e3fb6 Connector Levels: Add new internal metadata fields (#28904)
* Add airbyte internal

* Add tests

* First pass

* Set destinations to same levels as sources

* Best guess at missing statuses

* Best guess at _ql

* Add separate enum class

* Fix support level name

* Update templates

* Add one more test
2023-08-01 18:08:33 -05:00
Daryna Ishchenko
dc7255bb6f 🐛 Source Google Directory: update version in cloud to 0.2.0 (#27236)
* update version in cloud

* 🤖 Auto format source-google-directory code [skip ci]

* increased connectors version

---------

Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
2023-06-22 13:30:53 +03:00
Denys Davydov
c10bd5e706 🐛 Source Google Directory: migrate to advanced auth (#26775)
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #26232 source Google Directory: remove authSpecification from spec

* #26775 upd changelog

* #26232 source google directory: update stream schemas

* source google directory: update the Users stream schema

* source google directory: fix schema

* #26232 source google directory: add even more fields

* source google directory: add primary key

* expose pk for CATs
2023-05-31 20:53:34 +03:00
Ben Church
1dabc6208e Metadata: add tags field (#26320)
* Add optional tags field

* Remove duplicate icons

* Add programming tags to all

* Update docs

* supportUrl -> documentationUrl

* Ensure one language tag is applied

* Add keyvalue check

* rebase and fix tests

* Format

* Add cache buster

* Improve test

* Automated Commit - Formatting Changes

* Update error

* Fix missing tags

* Fix scaffold

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-05-26 16:13:09 -07:00
Ben Church
248bbf94c1 HACKDAY: Icon CDN (#26158)
* Move icons to connector folder

* Delete old icons

* Update upload logic

* Add icon url to definitions

* Update registry model

* Populate cdn url

* DNC butcher the pipeline

* Low hanging fruit fixes

* Fix bucket name

* Merge old and new approaches

* Fix metadata upload step

* Format

* Fix test
2023-05-24 17:25:41 -07:00
Augustin
7310494846 qa-checks: check metadata version matches dockerfile version (#25661) 2023-05-04 16:08:19 -07:00
Ben Church
5563179782 Dagster: rename catalog to registry (#25254)
* rename catalog to registry in metadata service

* rename catalog to registry in metadata files

* Run generate models

* Fix missed renames

* Add github personal access token

* Run black

* Automated Change

---------

Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-04-18 22:15:11 +02:00
Augustin
ebc907cdf7 create metadata files for all connectors (#24964) 2023-04-13 07:45:04 +02:00
Catherine Noll
7da6a3bb77 Run CATs with local CDK (#23084)
Scripts to 

* Run CATs against the local CDK for one connector

* Run CATs against the local CDK for multiple connectors

* Create a connecter image with the local CDK

---------

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2023-02-24 16:13:42 -05:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594)
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
2023-02-08 13:01:16 -08:00
Joe Reuter
6a10ae3e05 Rename source acceptance test to connector acceptance test (#21846)
Rename source acceptance test to connector acceptance test
2023-02-02 11:38:19 +01:00
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585)
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
2022-10-11 11:04:23 -04:00
Marcos Marx
dca2256a7c Bump 2022 license version (#13233)
* Bump year in license short to 2022

* remove protocol from cdk
2022-05-26 15:00:42 -03:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763)
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
2022-04-11 20:51:37 -07:00
Iryna Grankova
fefd5759c8 Update fields in source-connectors specifications: file, freshdesk, github, google-directory, google-workspace-admin-reports, iterable (#8524)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-12-11 01:47:29 +02:00
Christophe Duong
86ca36c5c0 Format code (#7978) 2021-11-15 14:51:10 +01:00
vitaliizazmic
1f295f24e6 🎉 Source Google Directory: support oauth
* Source Google Directory #6265 - add oauth support

* Source Google Directory #6265 - update credentials

* Source Google Directory #6265 - fixing according to PR

* Source Google directory #6265 - update docs

* Source Google directory #5190 - update doc

* Source Google Directory #6265 - resolve merge conflict

* Source Google Directory #6265 - SAT for oauth

* Source Google Directory #6265 - bump version and update changelog

* Source Google Directory #6265 - bump version and update changelog (update publish)
2021-11-05 21:40:56 +02:00
vitaliizazmic
d37e00e6a4 🎉 Source Google Directory: migrate to the CDK
* Source Google Directory #7415 - migrate to the CDK

* Source Google Directory #5190 - fix timeout error

* Source Google Directory #7415 - fix according to PR review

* Source Google Directory #7415 - added etag and lastLoginTime to ignored fields for full refresh acceptance test

* Source Google Directory #7415 - fix full refresh acceptance test config

* Source Google Directory #7415 - bump version
2021-11-02 22:17:00 +02:00
Marcos Marx
0585a5d52d Publish PR 6930: google dir fix members (#7218)
* Google Directory: Fix crash if there's a group don't have members

* bump version

Co-authored-by: Xiangxuan Liu <xiangxuan.liu@rightcapital.com>
2021-10-21 17:32:21 -03:00
Xiangxuan Liu
8a54f1be2a Google Directory: Fix crash if there's a group don't have members (#6930) 2021-10-21 17:31:57 -03:00
Marcos Marx
e821919066 run gradlew format (#7223) 2021-10-20 16:43:14 -03:00
Yevhenii
25110c167d 🎉 Google Directory: Add organizations and phones to users schema (#7167)
* Google Directory: Add organizations and phones to users schema

https://developers.google.com/admin-sdk/directory/v1/guides/manage-users#json-response

* bump version, add changelogs to google-directory.md

Co-authored-by: Xiangxuan Liu <9570112+nauxliu@users.noreply.github.com>
2021-10-19 16:59:00 +03:00
Michel Tricot
1773e41e47 Shorten our headers + adds contributors file (#6478) 2021-09-27 10:45:50 -07:00
LiRen Tu
2906ec287a CI: Add action to check broken doc links (#5254)
* Add action to check broken doc links

* Ignore localhost

* Update config

* Fix broken links

* Use quiet mode

* Ignore PR link

* Fix more broken links

* Fix more broken links

* Fix more broken links

* Verify pattern

* Fix more broken links

* Separate full and pr check

* Update pattern

* Test invalid link

* Remove invalid link
2021-08-07 14:28:02 -07:00
Jared Rhizor
b4793b2510 add AIRBYTE_ENTRYPOINT for kubernetes support (#3973)
* add AIRBYTE_ENTRYPOINT for kubernetes support

* bump versions

* bump version in seed

* Update generic template

* keep scaffold sources at 0.1.0

* add missing newline

* handle python base versions correctly

* re-bump mysql and postgres sources

* re-bump snowflake destination

* add skip tests option

* switch to running tests

* reverse conditional to make it safer

* fix publish to include the test running

* fix iterable version

* fix file generation

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-06-09 13:01:45 -07:00
Dmytro
a1035e007b 📚 Update all Python connectors READMEs with minimum required version (#3760) 2021-06-02 12:35:29 +03:00
Charles
0df53170c9 Stop formatting python with spotless (#3388) 2021-05-13 17:46:34 -07:00
Davin Chia
f660b0a946 Add template generation for Santa aka CDK. (#3034)
Template generation for new Source using the Santa CDK - provide basic scaffolding for someone implementing a new source.

General approach is to buff up comments in the original SDK, and add TODOs with secondary comments in the generated stub methods, as well as links to existing examples (e.g. Stripe or ExchangeRate api) users can look at.

Checked in and added tests for the generated modules.
2021-04-25 18:02:33 +08:00
Charles
f445fdb5b2 match styling for spotlessApply and format (#3017)
* as a java developer I want to be able to run spotlessApply without changing styles in python code
2021-04-23 09:21:41 -07:00
Sherif A. Nada
1d51962a2d publish mongodb discovery speedup (#2900) 2021-04-15 11:38:41 -07:00
Nathan Nowack
0e433aa145 🎉 New Source Connector: Smartsheets (#2880) 2021-04-15 09:48:30 -07:00
Sherif A. Nada
2d2d0173d6 Update codegen template READMEs (#2873) 2021-04-13 11:41:52 -07:00
Sherif A. Nada
ed89509cf0 update docs links for Google Workspace/Audit logs (#2789) 2021-04-07 13:49:46 -07:00
Jared Rhizor
0609011e05 fix connector base versioning (#2726)
* fix connector base versioning

* only run integration tests in non-bases directories

* upgrade datamodel-code-generator which coincidentally has already fixed this issue with generating List of Lists

* bump forgotten hubspot version

* fix build
2021-04-02 22:46:06 -07:00