1
0
mirror of synced 2026-01-06 06:04:16 -05:00
Commit Graph

974 Commits

Author SHA1 Message Date
Alexandre Girard
97a353d5c5 Run mypy on airbyte-cdk as part of the build pipeline and fix typing issues in the file-based module (#27790)
* Try running only on modified files

* make a change

* return something with the wrong type

* Revert "return something with the wrong type"

This reverts commit 23b828371e.

* fix typing in file-based

* format

* Mypy

* fix

* leave as Mapping

* Revert "leave as Mapping"

This reverts commit 908f063f70.

* Use Dict

* update

* move dict()

* Revert "move dict()"

This reverts commit fa347a8236.

* Revert "Revert "move dict()""

This reverts commit c9237df2e4.

* Revert "Revert "Revert "move dict()"""

This reverts commit 5ac1616414.

* use Mapping

* point to config file

* comment

* strict = False

* remove --

* Revert "comment"

This reverts commit 6000814a82.

* install types

* install types in same command as mypy runs

* non-interactive

* freeze version

* pydantic plugin

* plugins

* update

* ignore missing import

* Revert "ignore missing import"

This reverts commit 1da7930fb7.

* Install pydantic instead

* fix

* this passes locally

* strict = true

* format

* explicitly import models

* Update

* remove old mypy.ini config

* temporarily disable mypy

* format

* any

* format

* fix tests

* format

* Automated Commit - Formatting Changes

* Revert "temporarily disable mypy"

This reverts commit eb8470fa3f.

* implicit reexport

* update test

* fix mypy

* Automated Commit - Formatting Changes

* fix some errors in tests

* more type fixes

* more fixes

* more

* .

* done with tests

* fix last files

* format

* Update gradle

* change source-stripe

* only run mypy on cdk

* remove strict

* Add more rules

* update

* ignore missing imports

* cast to string

* Allow untyped decorator

* reset to master

* move to the cdk

* derp

* move explicit imports around

* Automated Commit - Formatting Changes

* Revert "move explicit imports around"

This reverts commit 56e306b72f.

* move explicit imports around

* Upgrade mypy version

* point to config file

* Update readme

* Ignore errors in the models module

* Automated Commit - Formatting Changes

* move check to gradle build

* Any

* try checking out master too

* Revert "try checking out master too"

This reverts commit 8a8f3e373c.

* fetch master

* install mypy

* try without origin

* fetch from the script

* checkout master

* ls the branches

* remotes/origin/master

* remove some cruft

* comment

* remove pydantic types

* unpin mypy

* fetch from the script

* Update connectors base too

* modify a non-cdk file to confirm it doesn't get checked by mypy

* run mypy after generateComponentManifestClassFiles

* run from the venv

* pass files as arguments

* update

* fix when running without args

* with subdir

* path

* try without /

* ./

* remove filter

* try resetting

* Revert "try resetting"

This reverts commit 3a54c424de.

* exclude autogen file

* do not use the github action

* works locally

* remove extra fetch

* run on connectors base

* try bad  typing

* Revert "try bad  typing"

This reverts commit 33b512a3e4.

* reset stripe

* Revert "reset stripe"

This reverts commit 28f23fc6dd.

* Revert "Revert "reset stripe""

This reverts commit 5bf5dee371.

* missing return type

* do not ignore the autogen file

* remove extra installs

* run from venv

* Only check files modified on current branch

* Revert "Only check files modified on current branch"

This reverts commit b4b728e654.

* use merge-base

* Revert "use merge-base"

This reverts commit 3136670cbf.

* try with updated mypy

* bump

* run other steps after mypy

* reset task ordering

* run mypy though

* looser config

* tests pass

* fix mypy issues

* type: ignore

* optional

* this is always a bool

* ignore

* fix typing issues

* remove ignore

* remove mapping

* Automated Commit - Formatting Changes

* Revert "remove ignore"

This reverts commit 9ffeeb6cb1.

* update config

---------

Co-authored-by: girarda <girarda@users.noreply.github.com>
Co-authored-by: Joe Bell <joseph.bell@airbyte.io>
2023-07-13 16:55:48 -07:00
Augustin
086ddb750b connectors-ci: revert to Dagger 0.5.4 (#27976) 2023-07-05 20:40:22 +02:00
Augustin
efe21d1659 connectors-ci: better capture stderr (#27833) 2023-07-04 11:13:12 +02:00
Sherif A. Nada
c33ea9926c make publish_option required in publish connectors workflow (#27879) 2023-06-30 13:06:02 -07:00
Conor
f2d5b7449a Revert "connectors-ci: switch back to large-runner" (#27767) 2023-06-28 17:29:45 +02:00
Augustin
58dd54f97f change default bucket (#27749) 2023-06-27 03:27:52 -05:00
Augustin
8a03adcd98 switch back to large-runner (#27748) 2023-06-27 02:47:12 -05:00
Ben Church
543897c2ad Remove Affected connector report (#27622)
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
2023-06-26 09:54:58 -07:00
Augustin
2f02d7fb96 weekly build: fix typo on workflow name (#27707) 2023-06-26 07:56:50 -05:00
Augustin
d3c33a6321 connector-ci: better and lighter logging (#27606) 2023-06-24 09:17:46 +00:00
Ella Rohm-Ensing
12c7da7f11 fix docker auth for metadata tests and orchestrator deploy (#27655)
* test change in metadata library to trigger tests

* add credentials when calling dagger from metadata_service_tests_dagger.yml

* Fix env for metadata_service_deploy_orchestrator_dagger.yml

* revert adding metadata api token input

* reorder inputs by how they are listed in workflow

* add context for orchestrator deploy
2023-06-23 14:57:55 +00:00
Augustin
4638209e0d connectors-ci: update runners to new node pools (#27615) 2023-06-22 19:54:42 +02:00
Ben Church
3bd340a971 Update to use rev-parse (#27598)
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
2023-06-21 20:42:56 -05:00
Ryan Fu
4d76ac5809 Turn off java formatter (#27579) 2023-06-21 21:13:20 +00:00
Augustin
3050c24ab4 fix typo in env var name (#27553) 2023-06-21 10:54:20 -05:00
Augustin
22480ac642 ref: broaden the use of the run-dagger-pipeline action (#27506) 2023-06-21 06:47:33 -05:00
Ben Church
943150a19b Add any glob to filter to ensure all steps are run in CI (#27525)
* Ensure all steps are run

* Update comment

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
2023-06-20 15:14:28 -07:00
Augustin
e9d7c54adc connectors ci: auto format connectors code (#27353) 2023-06-19 17:44:43 +00:00
Augustin
12c1602883 fix: retry action latest version is not stable (#27465) 2023-06-19 05:37:11 -05:00
Augustin
55423c2495 connectors-ci: do not run tests on fork PR (#27402) 2023-06-15 18:14:47 +00:00
Augustin
d5dc309560 gradle.yml: remove Cache Build Artifacts (#27409) 2023-06-15 19:55:00 +02:00
Augustin
ecc310e69d fix trigger on format workflow (#27391) 2023-06-15 03:56:27 -05:00
Augustin
02a592dd4a create new workflow to format connectors (#27389) 2023-06-15 03:41:22 -05:00
Ben Church
68c0d6dbbc Connector CI: update badges test summary (#27215)
* Rename to connector test report

* Get all test outputs

* Get last 10 output for all connectors

* Get runs by connector

* Crudely write badge and summary

* Add render functions

* Update connector report badge and link

* Fix url link

* Fix qa report tests

* update qa-engine tests

* Move regex to resource

* Add cron

* Run format

* New line

* Make badge green if last is successful

* Fix test

* Remove report.sh and build_report.py

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
2023-06-14 11:12:47 -07:00
Augustin
5b8200181c connectors-ci: deprecate slash test (#27200) 2023-06-14 18:19:13 +02:00
Augustin
ad1a992097 connectors-ci: move connectors_weekly_build.yml to workflows dir (#27330) 2023-06-13 14:41:37 -05:00
Augustin
6879cdf5d6 weekly build: add missing env var (#27320) 2023-06-13 11:22:45 -05:00
Augustin
09ec1efcc4 connectors-ci: weekly build for alpha connectors (#27300) 2023-06-13 18:17:14 +02:00
Evan Tahler
521a798fb5 Experiments to fix CI (#27262)
* don't pin virtualenv version

* comments

* relax python version

* keep virtualenv relax

* remove cache

* add note

* Update .github/workflows/gradle.yml

Co-authored-by: Ben Church <ben@airbyte.io>

---------

Co-authored-by: Ben Church <ben@airbyte.io>
2023-06-12 17:59:57 -04:00
Tim Roes
cdfa75e781 Remove old growth related labeling code (#27248) 2023-06-12 20:18:53 +02:00
Augustin
09ae1bafb0 connectors-ci: disable concurrency for Java connectors testing (#27012) 2023-06-09 08:39:23 +00:00
Ben Church
2df72cc63c Connector CI: Update nightlies to write to GCS (#26929)
* Generate report output prefix

* DNC

* Upload to gcs bucket

* Revert dev comments

* Format

* Reenable for cron

* remove reports

* Connector CI reports

* Reremove AWS

* Missed some env vars

* Move bucket to CI

* Bad merge

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
2023-06-06 16:37:19 +00:00
Augustin
1ab95effbb connectors-ci: re-enable concurrency on java connectors (#27052) 2023-06-06 10:06:24 +02:00
Ben Church
723093b095 Metadata: Validate that metadata file exists on new connectors (#26933)
* Ensure we test to see IF there is a metadata file

* Run format

* Add modifed files as path

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
2023-06-05 17:52:07 -07:00
Edward Gao
ae43522659 Destination TiDB: install normalization (#27025)
* install normalization

* changelog

* add to codeowners

* update changelog messages
2023-06-06 00:05:50 +00:00
Augustin
80584edea1 fix typo on workflow (#27005) 2023-06-05 01:54:31 -05:00
Augustin
4e88af04bb connectors-ci: no concurrency on java connectors for nightly builds (#27003) 2023-06-05 01:48:47 -05:00
Augustin
af890515ee from 6 to 10hours timeout (#26999) 2023-06-04 15:30:19 -05:00
Conor
6324b28ba3 Use repository tags to clean up zombine instances instead of size (#26919) 2023-06-02 10:01:53 -05:00
Augustin
ecddb3634d connectors-ci: re-enable scheduling of nightly builds (#26956)
* connectors-ci: re-enable scheduling of nightly builds

* fix timing
2023-06-02 09:38:47 -05:00
Ben Church
68e4d503ef Update gradle command to remove gitignored files (#26924)
* Update gradle command to remove gitignored files

* fix typo

* Automated Commit - Format and Process Resources Changes

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-06-02 02:24:59 +00:00
Augustin
d5650eeabd connectors-ci: use new dagger cli on tests (#26927)
* connectors-ci: use new dagger cli on tests

* Automated Change

---------

Co-authored-by: alafanechere <alafanechere@users.noreply.github.com>
2023-06-01 17:46:45 -05:00
Augustin
a63041603e connectors-ci: dynamic dagger pipeline name (#26886)
* connectors-ci: dynamic dagger pipeline name

* tmp

* add retry logic

* tmp

* Revert "tmp"

This reverts commit 976feeb584.

* Revert "tmp"

This reverts commit 5682a9a064.

* Revert "add retry logic"

This reverts commit 71e074991d.

* fix typo on GHA input
2023-06-01 03:18:15 -05:00
Duy Nguyen
88438bc6f5 Add paths to only trigger on source-postgres changes (#26866)
* Add paths to only trigger on source-postgres changes
Co-authored-by: nguyenaiden <nguyenaiden@users.noreply.github.com>
2023-05-31 15:40:15 -07:00
Duy Nguyen
7089869927 Jacoco report for source-postgres (#26792)
* Add jacoco coverage workflow for source-postgres
* Update min coverage to 64 pct for source-postgres
2023-05-31 13:19:16 -07:00
Ben Church
5e0e317eac Move qa engine to aibyte-ci (#26377)
* Add change notes

* Add qa-engine poetry

* Port qa_engine code

* Add missing fixture

* Update import

* Update git code

* Delete old qa-engine files

* convert tests

* Add path

* Fix mocks

* Fix test_enable_in_cloud

* Fix metadata path test

* Run formater

* Automated Commit - Formatting Changes

* Update readme

* Remove duplicates

* Remove redundant if

* Add ruamel

* Use ruamel

* Remove safe load

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-05-31 05:26:28 +00:00
Augustin
9e96effc89 connectors-ci: connectors base build not required for majority of connectors (#26739) 2023-05-30 22:55:36 +02:00
Augustin
fc0c5fbc34 connectors-ci: renable automated tests on PRs (#26742) 2023-05-30 04:14:23 -05:00
Augustin
312a3615f3 connectors-ci: pass PR id to airbyte-ci (#26504) 2023-05-30 08:15:53 +00:00
Augustin
2c9192e7a0 connectors-ci: improve global status reporting to GitHub for tests (#26575) 2023-05-29 19:53:59 +02:00