1
0
mirror of synced 2026-01-06 06:04:16 -05:00
Commit Graph

42 Commits

Author SHA1 Message Date
Topher Lubaway
b689d5a63f Revert "Update action.yml (#19416)" (#19450)
This reverts commit 78fb528a9a.
2022-11-15 15:09:38 -06:00
Mykyta Serbynevskiy
78fb528a9a Update action.yml (#19416)
Changing instance type of AWS self-hosted runner from from c5.2xlarge to c6a.4xlarge.

It will result in:

CPU: 8 -> 16
Sustained clock speed (GHz): 3.4 -> 3.6
RAM: 16 -> 32
cost: 0.34 USD/hr -> 0.612 USD/hr
2022-11-15 18:02:18 +02:00
Volodymyr Pochtar
7454a42055 ci: bump marketplace actions due to deprecations (#19126) 2022-11-10 16:01:35 +02:00
Alexandre Girard
8f00f2d896 install python on build branch (#19254) 2022-11-09 13:45:54 -08:00
Peter Hu
013ecf7063 use longer short shas (#19190)
it seems github APIs want us to use longer shas in some cases where a
7 character short sha is still entirely valid

So just set the minimum short sha length to 10
2022-11-08 16:09:52 -08:00
Volodymyr Pochtar
afde9c6c94 ci: return 7 symbols short git hash on branch build (#18934) 2022-11-03 21:50:19 +02:00
Volodymyr Pochtar
f488081366 ci: bring old set-output to build branch and artifact tags (#18923) 2022-11-03 19:27:52 +02:00
Volodymyr Pochtar
fa37148e7d ci: replace set-output with new github actions syntax (#18691) 2022-10-31 19:05:00 +02:00
Serhii Chvaliuk
bf06decf73 AMI change: ami-0f23be2f917510c26 -> ami-005924fb76f7477ce (#18689)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-10-31 18:21:39 +02:00
Malik Diarra
f3f895d6a9 Use docker-compose-cloud.buildx file (#15994) 2022-08-26 15:29:41 -07:00
George Claireaux
e7dae06168 Enhance /publish to allow for multiple connectors and parallel execution (#13864)
* start

* revert

* azblob

* bq

* bq denorm

* megapublish baaaabyyyy

* fix needs

* matrix connectors

* auto-bump connector version

* dont failfast and max parallel 5

* multi runno

* minor

* testing matrix agents

* name

* testing multi agents

* tmp fix

* new multi agents

* multi test

* tryy

* let's do this

* magico

* fix

* label test

* couple more connector bumps

* temp

* things

* check this

* lets gooo

* more connectors

* Delete TEMP-testing-command.yml

* auto-bump connector version

* added comment describing bash part

* running single thread

* catch sentry cli

* auto-bump connector version

* destinations

* + snowflake

* saved

* auto-bump connector version

* auto-bump connector version

* java source bumps

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* remove twice-defined methods

* label things

* revert action

* using the new test action

* point at action

* wrong tag on action

* update pool label

* update to use new ec2-github-runner fork

* this needs to be more generic than publisher

* change publish to run on pool

* add comment about runner-pool usage

* updated publish command docs for multi & parallel connector runs

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* unbump failed publish versions

* missed dockerfiles

* remove failed docs

* mssql fix

* overhauled the git comment output

* bumping a test connector that should work

* slight order switcheroo

* output connectors properly in first message

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-21 18:40:33 +01:00
George Claireaux
b9f79ccdf0 remove test action (#13944) 2022-06-20 17:45:10 +01:00
George Claireaux
546d633337 newaction (#13942) 2022-06-20 17:07:20 +01:00
Tim Roes
fe7e1c2272 Upgrade node to latest LTS (#13683)
* Upgrade node to latest LTS

* Upgrade to newer setup-node action
2022-06-10 17:07:57 +02:00
Sherif Nada
b763bdf4ab Revert "add github workflows for publishing ERDs"
This reverts commit c7a23c680b.
2022-06-08 03:32:56 -07:00
Sherif Nada
c7a23c680b add github workflows for publishing ERDs 2022-06-07 22:42:13 -07:00
Parker Mossman
aaf34cfc20 update OSS action to take in docker username input (#13352) 2022-05-31 13:41:40 -07:00
Evan Tahler
d1ff8dd8b4 [Github Actions] Rename docker secrets and parameterize docker user (#13297)
* Rename docker secrets and parameterize docker user

* Apply suggestions from code review

Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>

Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>
2022-05-27 18:48:26 -07:00
a-honcharenko
570770c10b A honcharenko/multiarch build 801 (#12570)
* added multiarch image publishing/modified dockerfiles

* added new ami

* changed version to test

* rollback version

* check version test

* env vars temp fix

* apt-utils error fix

* disabled failed test

* remove excluded tests

* Excluded :airbyte-db:lib:test

* Excluded :airbyte-db:lib:test

* static jdk version for test

* ok test

* ok test

* ok test

* test vars

* quemu issue fix

* Returned version vars

* Scripts update

* Version vars change

* Comment fix

* Added comments, minor changes and comments in dockerfiles

* Uncomment line to push images
2022-05-11 20:07:13 +03:00
Peter Hu
0d3088e456 notify cloud repo on push to master (#12229)
* notify cloud repo on push to master

* split out build-branch action

so we can use it in cloud workflow
2022-04-20 16:53:20 -07:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763)
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
2022-04-11 20:51:37 -07:00
Parker Mossman
884a94ed29 Un-Revert OSS branch build for Cloud workflow (#11808)
* Revert "Revert "Build OSS branch for deploying to Cloud env (#11474)""

This reverts commit 55e3c1e051.

* add action to get dev branch tag to OSS project instead of doing it in cloud

* remove dev branch version action, going to do this in cloud afterall
2022-04-08 15:17:04 -07:00
lmossman
55e3c1e051 Revert "Build OSS branch for deploying to Cloud env (#11474)"
This reverts commit 189efe7b42.
2022-04-05 15:44:31 -07:00
Parker Mossman
189efe7b42 Build OSS branch for deploying to Cloud env (#11474)
* add VERSION buildArg to Dockerfiles, default to current airbyte version but overwritable

* use VERSION env var consistently as Dockerfile buildArg, jar version, and tag

pass version and image_tag into docker build task function

* add github action for building and pushing an OSS branch for Cloud to consume

* allow AirbyteVersion to validate versions containing 'oss-branch' prefix

* change oss-branch prefix to dev for branch-based versions

* better action name

* add docker-compose-cloud.build.yaml to define minimum set of cloud images that are pushed by oss branch action

* update local dev docs to describe optional usage of VERSION env var

* make branch_version_tag input optional, if not provided, generates dev-<commit_hash>

* fix typo

* fix missed merge conflict

* update docker docs

* update integrationRunner isDev check
2022-04-05 15:06:17 -07:00
Topher Lubaway
90100e013c Increase 2 minute timer to 5 minutes for ec2 (#11519)
same intended commit as before, i didn't tag correctly
we are missing some builds due to the timer and can afford
some wasted actions
2022-03-30 10:35:14 -05:00
Topher Lubaway
232d9914e6 Revert "Increase runner timeout to 5 minutes from 2 (#11485)" (#11504)
This reverts commit 8cfee1066a.
2022-03-29 10:44:04 -05:00
Topher Lubaway
8cfee1066a Increase runner timeout to 5 minutes from 2 (#11485)
seeing failing builds from 2 and we have higher API
limits now
2022-03-29 10:15:56 -05:00
Topher Lubaway
906887c327 Use shell hacks to find valid PAT (#11439)
* Use shell hacks to find valid PAT

if rate limit exceeded try another key

* Use tim's version of the ec2 runner

this is a weird solution with weird limitations.  should address the short term pain (march 2022)
2022-03-28 15:17:07 -05:00
Topher Lubaway
e3f36eca96 Increase the AWS runner timer via tool update (#11471) 2022-03-28 09:00:56 -05:00
Topher Lubaway
3a8df2fb23 Thottle and retry AWS runner creation in CI (#11435)
* Adds tim's ocotocat rate limiter

* Change all the actions to our homebrew ec2 fork

* Compiles the dist in npm

* Update action to v15.0.1

* Update to v15.0.2

* Update to v15.0.3

* Update to v15.0.4

Co-authored-by: Tim Roes <tim@airbyte.io>
2022-03-25 17:09:26 +01:00
Davin Chia
b47e058744 Use action with better rate limits for start aws action to see if this helps. (#11415) 2022-03-25 19:02:43 +08:00
Tim Roes
eced94a52c Use new AMI including cypress dependencies (#11191) 2022-03-16 12:06:09 +01:00
Greg Solovyev
baae71390b Fix Sonar for non-python connectors (#10928)
This is a quick fix that moves cat ${REPORT_FOLDER}/*
inside if block for python-based connectors.
2022-03-07 16:41:19 -08:00
Davin Chia
ba3265a76b Add caching to all jobs in the main build. (#10801)
Add build dependency caching to all jobs in the main build.

This speeds things up by 5 mins over the previously uncached time.
2022-03-03 21:55:32 +08:00
Maksym Pavlenok
61b0e9e196 🎉 Single py checker (#10246) 2022-02-21 14:01:42 +02:00
Yevhenii
961f2f6caf 🎉 Codecov: Implement Codecove into Sonar (#9666)
* Implement Codecov into Sonar config
2022-01-23 20:08:25 +02:00
h-chlor
1832c69c26 Add SonarQube to CI (#8362) 2022-01-19 18:25:43 +02:00
LiRen Tu
2bad2cf3cf Use new github action runner ami with 100 gb disk (#9450)
* Use new ami with 100 gb for action runner

* Revert slash command ami

Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com>
2022-01-15 20:38:56 -08:00
Davin Chia
f552539619 Upgrade to patched version. (#9506) 2022-01-14 17:30:45 +08:00
Davin Chia
22db152ab6 Use patched version. (#9466)
Due to actions/runner#1605, all our builds are failing.

This changes pins us to machulav/ec2-github-runner#88 until a more stable solution is available.

Either the upstream action merges this in and we bump our version, or Github reverts their breaking change.
2022-01-13 14:28:51 +08:00
Davin Chia
2093b19da6 Upgrade start runner to v2.3.0. (#8703) 2021-12-11 17:47:41 +08:00
Davin Chia
214b158bde Add labels to EC2 build machines. Refactor actions to reduce duplicate start ec2 runner actions. (#7119)
As the title.

I also took the chance to compose the actions, instead of injecting it everywhere.
2021-10-19 18:02:18 +08:00