1
0
mirror of synced 2025-12-26 14:02:10 -05:00
Commit Graph

38 Commits

Author SHA1 Message Date
Aaron ("AJ") Steers
5629bb9e6b CI: fix missing slash command input args (#37564) 2024-04-25 08:58:55 -07:00
Marius Posta
b9ab3d1a90 migrate everything from java 17 to java 21 (#35103) 2024-02-09 19:26:19 -06:00
Ben Church
1d1ed472d6 Run ensure path (#28792) 2023-07-27 11:29:15 -05:00
Ben Church
fb7258e2bd Move tools/ci_* projects to airbyte-ci, update to use Poetry, bump to python 3.10 (#27957)
* Move ci_connector_ops

* Move ci_credentials

* Move tools/ci_common_utils

* Rename tools to airbyte-ci

* Move to ci

* Convert ci_credentials

* Convert ci_common_utls

* Convert ci_connector_ops

* Get pipelines running

* Move pipelines to own poetry project

* Update readme

* Delete

* Add ci_code_validator

* Use pipx to install gha deps

* Fix'

* Ensure every thing is running

* Automated Commit - Formatting Changes

* Gitignore miss

* Add pipx installer

* Get local pipx dependencies

* Fix paths

* Install pipx

* ceremonial source-faker change

* Add installation step for ci_code_validator

* Add comment

* remove ci_code_validator

* Address code review comments

* add pipx install to acceptance-test-docker.sh

* Run formater

* Revert "ceremonial source-faker change"

This reverts commit 26884cd0db.

* gitignore lecacy pipeline report path

* update poetry.lock

* skip upload if logs do not exist

---------

Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
2023-07-26 15:49:59 +00:00
Ben Church
68c0d6dbbc Connector CI: update badges test summary (#27215)
* Rename to connector test report

* Get all test outputs

* Get last 10 output for all connectors

* Get runs by connector

* Crudely write badge and summary

* Add render functions

* Update connector report badge and link

* Fix url link

* Fix qa report tests

* update qa-engine tests

* Move regex to resource

* Add cron

* Run format

* New line

* Make badge green if last is successful

* Fix test

* Remove report.sh and build_report.py

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
2023-06-14 11:12:47 -07:00
Conor
7953c45a09 delete platform code and workflows (#23214)
* delete platform code and workflows

* delete more platform workflows

* sunset kustomize (#23187)

* sunset kustomize

* remove build steps that use kustomize

* remove k8s kustomize gke-kube-acceptance-tests

* delete docker-compose files

* remove platform from check_images_exist.sh

* Working replacement script for docker

* rename start script and add download files to gitignore

* add args and download only option

* integration test downloads docker compose files

* remove helm charts

* remove more platform files

* add flags.yml

* use version from gradle.properties instead of .env

* Working replacement script for docker (#23227)

* Working replacement script for docker

* Working debugging line

* rename start script and add download files to gitignore

* add args and download only option

* Made flags cleaner and more cleanup

i've never used this flag pattern, its much
easier than my usual way

* warn about a sharp edge

* Don't run when we don't know a command

* better help spacing

* add flags.yml

---------

Co-authored-by: cpdeethree <conor@airbyte.io>
Co-authored-by: Conor <cpdeethree@users.noreply.github.com>

* remove more-secrets env

---------

Co-authored-by: Charles <charles@airbyte.io>
Co-authored-by: Topher Lubaway <asimplechris@gmail.com>
2023-02-19 09:42:08 -06:00
Mykyta Serbynevskiy
d4bfcf3372 introduce new pattern of GH PAT usage for #2641 (#22177) 2023-02-01 20:58:12 +02:00
Augustin
295fcecde0 ci_credentials: disable tooling test run by tox (#21580)
* disable tox

* rename steps

* revert changes on experimental workflow

* do not install tox
2023-01-19 03:42:56 -06:00
Augustin
f57fcdf87c test/publish commands: always run update secrets (#20183) 2022-12-07 17:13:20 +01:00
Augustin
9cb5714cd1 ci-credentials: update GSM secrets with updated configuration values (#20076) 2022-12-07 11:53:22 +01:00
Peter Hu
0c73b70ac7 find_non_rate_limited_PAT (#19736)
* find_non_rate_limited_PAT

* checkout repo to find pat
2022-11-22 16:33:38 -08:00
Volodymyr Pochtar
7454a42055 ci: bump marketplace actions due to deprecations (#19126) 2022-11-10 16:01:35 +02:00
Serhii Chvaliuk
a0f01ad20b ami-0d648081937c75a73 -> ami-06cf12549e3d9c522 (#18667)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-10-30 21:41:52 +02:00
Conor
21ddc93731 chore: cleanup actions and workflows (#16139)
* chore: cleanup actions

* Update .github/workflows/shared-issues.yml

Co-authored-by: Topher Lubaway <asimplechris@gmail.com>

* Update .github/workflows/shared-issues.yml

Co-authored-by: Topher Lubaway <asimplechris@gmail.com>

* Update .github/workflows/shared-pulls.yml

Co-authored-by: Topher Lubaway <asimplechris@gmail.com>

* Update .github/workflows/test-performance-command.yml

Co-authored-by: Topher Lubaway <asimplechris@gmail.com>

Co-authored-by: Topher Lubaway <asimplechris@gmail.com>
2022-09-01 08:46:50 -05:00
Edward Gao
6b897c46e5 Fix normalization integration test workflow (#14454) 2022-07-06 09:14:54 -07:00
Edward Gao
bb74373c8b Run some DATs as part of base-normalization tests (#14312) 2022-07-01 14:55:23 -07:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763)
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
2022-04-11 20:51:37 -07:00
Topher Lubaway
90100e013c Increase 2 minute timer to 5 minutes for ec2 (#11519)
same intended commit as before, i didn't tag correctly
we are missing some builds due to the timer and can afford
some wasted actions
2022-03-30 10:35:14 -05:00
Topher Lubaway
a562c2963b More ec2 pipelines using dynamic PATs (#11525)
parker noticed we missed a few
reviewer: please help me make sure the function exists
for `find_valid_pat` in each place it's called
2022-03-29 17:30:21 -05:00
Topher Lubaway
232d9914e6 Revert "Increase runner timeout to 5 minutes from 2 (#11485)" (#11504)
This reverts commit 8cfee1066a.
2022-03-29 10:44:04 -05:00
Topher Lubaway
8cfee1066a Increase runner timeout to 5 minutes from 2 (#11485)
seeing failing builds from 2 and we have higher API
limits now
2022-03-29 10:15:56 -05:00
Topher Lubaway
2693107d60 Adds valid PAT testing to existing ec2 pipes (#11486)
* Adds valid PAT testing to existing ec2 pipes

adds the same workflow seen in gradle.yml to other pipelines

* Reorder PAT selection

rename SELF_RUNNER_GITHUB_ACCESS_TOKEN_1 ->
DAVINCHIA_PAT

remove LABEL_BOT_USERNAME
2022-03-29 10:03:59 -05:00
Topher Lubaway
906887c327 Use shell hacks to find valid PAT (#11439)
* Use shell hacks to find valid PAT

if rate limit exceeded try another key

* Use tim's version of the ec2 runner

this is a weird solution with weird limitations.  should address the short term pain (march 2022)
2022-03-28 15:17:07 -05:00
Topher Lubaway
e3f36eca96 Increase the AWS runner timer via tool update (#11471) 2022-03-28 09:00:56 -05:00
Topher Lubaway
3a8df2fb23 Thottle and retry AWS runner creation in CI (#11435)
* Adds tim's ocotocat rate limiter

* Change all the actions to our homebrew ec2 fork

* Compiles the dist in npm

* Update action to v15.0.1

* Update to v15.0.2

* Update to v15.0.3

* Update to v15.0.4

Co-authored-by: Tim Roes <tim@airbyte.io>
2022-03-25 17:09:26 +01:00
Tim Roes
3df0619b84 Allow slash commands on PRs from forks (#9742)
This enables all slash commands to also work on PRs created from forks. This will not run CI on PRs coming from a fork. There are unfortunately some limitations around injecting secrets in pull_request actions, which I described in detail in this comment.

I've done most of the testing for this in https://github.com/timroes/github-actions-test to test out that the context are set the way I'd expect them.

There's one risk: if any of the actual build scripts called by one of the slash commands would use this repository hard-coded they might fail. I've tried to search through the whole code base and don't believe this is the case.
2022-03-08 18:15:18 +08:00
LiRen Tu
2bad2cf3cf Use new github action runner ami with 100 gb disk (#9450)
* Use new ami with 100 gb for action runner

* Revert slash command ami

Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com>
2022-01-15 20:38:56 -08:00
andriikorotkov
0c99c5d7a5 🎉 Setting CPU and Memory limits for performance tests via github (#8807)
* updated ci performance test sh script

* updated ci performance test sh script

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* run tests

* update ci creds

* update performance test yaml file

* update performance test yaml file

* update performance test yaml file

* update performance test yaml file

* update performance test sh file

* update performance test sh file

* update performance test sh file

* update performance test sh file

* update java to 17 version in yaml file for performance tests

* update java to 17 version in yaml file for performance tests

* update java to 17 version in yaml file for performance tests

* update java to 17 version in yaml file for performance tests

* update java to 17 version in yaml file for performance tests

* clean test logs

* remove test logs

* remove test logs

* remove test logs

* return performance test logs

* add readme by performance tests
2021-12-17 12:47:45 +02:00
Maksym Pavlenok
d3b0c998b8 🎉 CI secrets: full migration to GSM (#8561)
* add python packages for CI scripts

* add tox config for all subpackages

* draft version

* init venv for scripts

* fix venv

* remove used comments

* fix run test

* change base folder

* update secret format

* update docs

* remove an unused file

* remove github secrets logic fully

* fix base_folder balue

* add functions desc
2021-12-15 15:08:46 +02:00
andriikorotkov
0fd129e5e5 🎉 Updated ci performance test sh script (#8780)
* updated ci performance test sh script

* updated ci performance test sh script

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg
2021-12-15 11:12:20 +02:00
andriikorotkov
57c4590beb 🎉 Updated ci performance test sh script (#8774)
* updated ci performance test sh script

* updated ci performance test sh script

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg
2021-12-14 13:39:44 +02:00
andriikorotkov
ef7684b711 🎉 Updated ci performance test sh script (#8772)
* updated ci performance test sh script

* updated ci performance test sh script

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg
2021-12-14 13:09:45 +02:00
andriikorotkov
f37f68c106 🎉 Updated ci performance test sh script (#8771)
* updated ci performance test sh script

* updated ci performance test sh script

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg
2021-12-14 12:35:09 +02:00
andriikorotkov
4f2add004f 🎉 Updated ci performance test sh script (#8769)
* updated ci performance test sh script

* updated ci performance test sh script

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg
2021-12-14 11:57:46 +02:00
andriikorotkov
94e4364fc4 🎉 Updated ci performance test sh script (#8751)
* updated ci performance test sh script

* updated ci performance test sh script

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg
2021-12-13 18:13:31 +02:00
andriikorotkov
b1c067a6f6 🎉 Updated ci performance test sh script (#8741)
* updated ci performance test sh script

* updated ci performance test sh script

* set all params to one arg

* set all params to one arg

* set all params to one arg

* set all params to one arg
2021-12-13 15:40:52 +02:00
andriikorotkov
e41b7d24e7 🎉 Updated ci performance test sh script (#8740)
* updated ci performance test sh script

* updated ci performance test sh script

* set all params to one arg

* set all params to one arg
2021-12-13 14:21:22 +02:00
Eugene
2e743d999c Added new slash command for triggering performance tests (#7920)
* Added new slash command as a prerequisite for https://github.com/airbytehq/airbyte/pull/7820
2021-11-12 21:15:57 +02:00