1
0
mirror of synced 2025-12-22 03:21:25 -05:00
Commit Graph

108 Commits

Author SHA1 Message Date
Augustin
a69856bf59 🐙 source-apify-dataset: run up-to-date pipeline [2024-08-10] (#43607) 2024-08-11 02:21:31 +02:00
Augustin
5237b4d9fa 🐙 source-apify-dataset: run up-to-date pipeline [2024-08-03] (#43071) 2024-08-04 02:31:52 +02:00
Catherine Noll
b471a7ccf0 Update metadata.yaml for all sources with sandbox connections (#42577) 2024-07-31 12:38:17 -04:00
Augustin
75cc6c32a5 🐙 source-apify-dataset: run up-to-date pipeline [2024-07-27] (#42627) 2024-07-28 02:29:12 +02:00
Augustin
5aa9b11095 🐙 source-apify-dataset: run up-to-date pipeline [2024-07-20] (#42364) 2024-07-21 02:18:19 +02:00
Augustin
2a21c95e24 🐙 source-apify-dataset: run up-to-date pipeline [2024-07-13] (#41893) 2024-07-14 02:19:24 +02:00
Augustin
2a71a5277d 🐙 source-apify-dataset: run up-to-date pipeline [2024-07-10] (#41344) 2024-07-10 14:30:24 +02:00
Augustin
ae84568224 🐙 source-apify-dataset: run up-to-date pipeline [2024-07-09] (#41189) 2024-07-10 02:21:11 +02:00
Augustin
03400aeb1f 🐙 source-apify-dataset: run up-to-date pipeline [2024-07-06] (#40813) 2024-07-07 02:26:33 +02:00
Natik Gadzhi
0ab6314e08 chore: Removes .dockerignore and requirements.txt in connectors (#40217) 2024-07-05 11:46:02 -07:00
Augustin
a52797f94e 🐙 source-apify-dataset: run up-to-date pipeline [2024-06-25] (#40411) 2024-06-26 02:23:38 +02:00
Augustin
a19d70d4b5 🐙 source-apify-dataset: run up-to-date pipeline [2024-06-22] (#40187) 2024-06-23 02:16:33 +02:00
Augustin
7057fc36b4 Make python connectors using base image 1.2.1 use 1.2.2 (#39129) 2024-06-05 18:30:08 +02:00
Augustin
de9da5b7e4 source-apify-dataset: [autopull] Upgrade base image to v1.2.1 (#39010) 2024-06-05 09:35:46 +02:00
Augustin
31c95da499 [skip ci] Add connectorTestSuitesOptions to metadata (#38188) 2024-05-15 15:04:10 -05:00
Natik Gadzhi
cb80e6922a [tools] prettier rules for .md + formatting cleanup 2024-05-07 08:19:33 -07:00
Brian Leonard
a267225e80 source-apify-dataset: ensure inline schemas, updated cdk, poetry (where possible) (#37115) 2024-05-01 11:31:21 -07:00
Augustin
8c862a8013 Fix tags in metadata.yaml files: declare cdk tags (#35670) 2024-03-07 04:32:29 -06:00
Joe Reuter
dfec691232 Publish to pypi batch13 (#34658) 2024-02-01 05:07:49 -06:00
Joe Reuter
f29234a2db airbyte-lib: Refactor connectors (#34552) 2024-01-30 03:22:13 -06:00
Joe Reuter
03c725b950 Unify lowcode tags (#34284) 2024-01-17 07:58:26 -06:00
Joe Reuter
7fe72581b2 Prepare Google Sheets, Google Drive, Apify dataset and connector templates for airbyte-lib (#33414) 2023-12-19 11:54:37 +01:00
Augustin
368ba78b64 🧹doc: update connectors README and remove acceptance-test-docker.sh (#32209) 2023-11-06 12:36:07 -06:00
Joe Reuter
92af34dfbb Apify Dataset: Remove unnecessary breaking change and improve documentation (#31462) 2023-10-17 18:27:21 +02:00
Vlada Dusek
d8cfa79dfb Source Apify Dataset: add item_collection stream with dynamic schema (#31333)
Co-authored-by: Joe Reuter <joe@airbyte.io>
2023-10-13 15:21:40 +02:00
Vlada Dusek
7bff33fc5d Source Apify Dataset: fix broken stream, manifest refactor (#30428)
Co-authored-by: Joe Reuter <joe@airbyte.io>
Co-authored-by: flash1293 <flash1293@users.noreply.github.com>
2023-10-07 01:05:19 +02:00
Marius Posta
7ae97175a6 gradle: fix repo wide behaviour (#30607) 2023-09-28 05:01:13 -07:00
Ben Church
5d8278900f Github Action: Add format.yml workflow (#30604)
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
Co-authored-by: octavia-approvington <octavia-approvington@users.noreply.github.com>
2023-09-21 18:11:41 -05:00
Ben Church
5c56ac1d84 Airbyte-ci: Remove gradle task connectorAcceptanceTest (#30326) 2023-09-19 15:16:37 -05:00
Joe Reuter
6340436a76 Pin apify version (#30481) 2023-09-15 17:59:44 +02:00
btkcodedev
05b7d01801 Source Apify Dataset: Migrate Python CDK to Low-code CDK (#29859)
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2023-08-30 13:12:00 -03:00
Marius Posta
f5c7c1c0b8 chore: get ./gradlew format to pass for the whole repo (same java style) (#29786) 2023-08-24 05:09:42 -05:00
Ben Church
40781313da Update Internal Fields: update ql levels to better resemble previous high strictness (#29450) 2023-08-15 18:03:10 -05:00
Augustin
00d9462216 cat/connectors-ci: replace docker runner with dagger runner in CAT (#28000) 2023-08-11 17:58:48 +02:00
Ben Church
2f7deaee02 [skip ci] Metadata: Remove leading underscore (#29024)
* DNC

* Add test models

* Add model test

* Remove underscore from metadata files

* Regenerate models

* Add test to check for key transformation

* Allow additional fields on metadata

* Delete transform
2023-08-03 10:56:13 -07:00
Ben Church
e9490e3fb6 Connector Levels: Add new internal metadata fields (#28904)
* Add airbyte internal

* Add tests

* First pass

* Set destinations to same levels as sources

* Best guess at missing statuses

* Best guess at _ql

* Add separate enum class

* Fix support level name

* Update templates

* Add one more test
2023-08-01 18:08:33 -05:00
Joe Reuter
b6531ac120 🐛 Source apify-dataset: Fix broken sync (#28290)
* fix connector

* fix more things

* prepare release

* cleanup

* format

* revert back to wrapper object

* format
2023-07-18 11:28:23 +02:00
Ben Church
1dabc6208e Metadata: add tags field (#26320)
* Add optional tags field

* Remove duplicate icons

* Add programming tags to all

* Update docs

* supportUrl -> documentationUrl

* Ensure one language tag is applied

* Add keyvalue check

* rebase and fix tests

* Format

* Add cache buster

* Improve test

* Automated Commit - Formatting Changes

* Update error

* Fix missing tags

* Fix scaffold

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-05-26 16:13:09 -07:00
Ben Church
248bbf94c1 HACKDAY: Icon CDN (#26158)
* Move icons to connector folder

* Delete old icons

* Update upload logic

* Add icon url to definitions

* Update registry model

* Populate cdn url

* DNC butcher the pipeline

* Low hanging fruit fixes

* Fix bucket name

* Merge old and new approaches

* Fix metadata upload step

* Format

* Fix test
2023-05-24 17:25:41 -07:00
Augustin
7310494846 qa-checks: check metadata version matches dockerfile version (#25661) 2023-05-04 16:08:19 -07:00
Ben Church
5563179782 Dagster: rename catalog to registry (#25254)
* rename catalog to registry in metadata service

* rename catalog to registry in metadata files

* Run generate models

* Fix missed renames

* Add github personal access token

* Run black

* Automated Change

---------

Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-04-18 22:15:11 +02:00
Augustin
ebc907cdf7 create metadata files for all connectors (#24964) 2023-04-13 07:45:04 +02:00
Catherine Noll
7da6a3bb77 Run CATs with local CDK (#23084)
Scripts to 

* Run CATs against the local CDK for one connector

* Run CATs against the local CDK for multiple connectors

* Create a connecter image with the local CDK

---------

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2023-02-24 16:13:42 -05:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594)
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
2023-02-08 13:01:16 -08:00
Joe Reuter
6a10ae3e05 Rename source acceptance test to connector acceptance test (#21846)
Rename source acceptance test to connector acceptance test
2023-02-02 11:38:19 +01:00
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585)
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
2022-10-11 11:04:23 -04:00
Alexandre Girard
3894134d11 Bump year in license short to 2022 (#13191)
* Bump to 2022

* format
2022-05-25 17:56:49 -07:00
George Claireaux
7e1d95c783 🤖 publish command: fix for version bumping multiple connectors in same PR (#12397)
* add a git pull from origin master before commit auto changes

* dummy bump 2 connectors

* pull from current branch not master

* auto-bump connector version

* dummy bump 2 connectors

* revert faker bump

* auto-bump connector version

* better descript in changelog

* dummy bump... again

* commit first, then pull merge, then push

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-27 16:18:22 +01:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763)
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
2022-04-11 20:51:37 -07:00
George Claireaux
f7d3a706b2 Publish connector command now auto bumps version (#11712)
* using yq + more success/failure comments

* added gradle process + git push

* dummy bump to test publish

* dummy bump to test publish

* dummy bump to test publish

* bump version

* fix connector variable

* bump version

* only git add necessary files

* remove git config

* bump version

* making octavia user

* version bump

* auto-bump connector version

* added docs and auto-bumo flag

* bump version

* separate IMAGE_NAME and IMAGE_VERSION env vars from sentry prep

* version bump

* auto-bump connector version

* added entry to apify changelog for clarity

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-05 23:21:55 +01:00