1
0
mirror of synced 2025-12-21 11:01:41 -05:00
Commit Graph

851 Commits

Author SHA1 Message Date
Evan Tahler
38302e82a2 Add some docker version info to the publish script (#13247) 2022-05-26 17:46:11 -07:00
Evan Tahler
392f445187 Publish connector Docker images for multiple platform architectures (#13004)
* Publish connector images for multiple platform architectures

* testing

* notes

* show path in log

* Build arch 1-by-1 and use manifest file

* build & test comment

* delete tmp docker tags

* share docker auth code

* remove manifests when done & fix trailing / for tag delete
2022-05-26 11:31:37 -07:00
Sherif A. Nada
981dc655ee include SAT and normalization in build reports (#13216)
* include SAT and normalization in build reports

* uncomment
2022-05-25 19:47:29 -07:00
Evan Tahler
be8d8574f2 Remove M1-specific docs - Airbyte works on M1 Macs now (#11856) 2022-05-24 16:46:26 -07:00
Anne
ab070a7724 Apply pmd to airbyte-config (#13003)
* Apply pmd to airbyte-config, exclude rules for now
2022-05-24 10:58:51 -07:00
noahkawasaki-airbyte
7a79216d01 Make publish command work with private repos by adding JWT into curl headers (#13141)
* Make publish command work with private repos by adding in JWT token to headers for curl

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-24 10:55:46 -07:00
Brian Leonard
4d3e0418a4 Java integration test update (#13113)
* Get more java error messages

* Bolder result

* Debug file contents

* Just give an output message
2022-05-23 15:36:53 -07:00
Brian Leonard
e82e6fc0a2 Show integration test results on the Action Summary (#13072)
* Show summary on action results for failed test

* Write title and steps to log

* Log test

* Remove debugging

* Support python and java output

* Extra line break
2022-05-23 11:23:18 -07:00
Davin Chia
bdbd68f9bb Remove old test scripts. (#13014)
No longer needed since we removed the old scheduler.
2022-05-19 22:51:57 +08:00
Topher Lubaway
2c690f2144 More helpful error message for detected changes (#12956)
* More helpful error message for detected changes

* Wrote the check backwards

* more clear errors

* Newline cinema
2022-05-18 09:01:18 -05:00
Lake Mossman
a82f41edcf Acceptance test scripts - kill containers using volumes on shutdown (#12887)
* kill containers using volumes on shutdown in acceptance test scripts

* apply change to octavia integration tests as well

* escape inner command to make it be executed at the right time
2022-05-16 17:09:46 -07:00
Sherif A. Nada
7ec7ba33e0 continue sending connector build reports (#12879) 2022-05-16 10:41:37 -07:00
a-honcharenko
852528533d Fix naming in script (#12786)
* Artifact naming fix

* Indent properly
2022-05-11 22:20:27 +03:00
a-honcharenko
7b43e39219 Added init exception to script (#12783) 2022-05-12 02:17:54 +08:00
a-honcharenko
570770c10b A honcharenko/multiarch build 801 (#12570)
* added multiarch image publishing/modified dockerfiles

* added new ami

* changed version to test

* rollback version

* check version test

* env vars temp fix

* apt-utils error fix

* disabled failed test

* remove excluded tests

* Excluded :airbyte-db:lib:test

* Excluded :airbyte-db:lib:test

* static jdk version for test

* ok test

* ok test

* ok test

* test vars

* quemu issue fix

* Returned version vars

* Scripts update

* Version vars change

* Comment fix

* Added comments, minor changes and comments in dockerfiles

* Uncomment line to push images
2022-05-11 20:07:13 +03:00
Sherif A. Nada
fa22d32e14 Filter daily build report to only GA & Beta connectors (#12684) 2022-05-06 16:41:38 -07:00
Topher Lubaway
2119f4d3a2 docs deploy command works in more cases (#12622)
if the user has more than one page of branches
they will be prompted in less
this fixes that
2022-05-05 15:18:39 -05:00
Topher Lubaway
fced083be5 Comments and aids to help future devs build OSS (#12571) 2022-05-05 07:26:31 -05:00
Topher Lubaway
cc24c5dda3 Adds error message for missing homebrew (#12547)
tested by Brian locally
2022-05-04 09:05:28 -05:00
noahkawasaki-airbyte
e8406a2cd6 Initial PR /publish workflow into pre-merge /build-connector and post-merge /publish-connector workflows (#12405) 2022-04-29 07:03:27 -07:00
Topher Lubaway
1413aca478 Toph deploy docs ssh repo check (#12323)
* Check for ssh github repo

because assumptions mostly.
more clear fail messages for an unsupported workflow
also removes the pipeline which now is redundant with local workflow

* WIP1

* working test

* More clear git test

* less weird characters for grep

* remove verbose for excho
2022-04-27 10:26:35 -05:00
Topher Lubaway
f065d361b1 Toph docs pipeline fixes bugs n gitbook (#12114)
* WIP

* WIP2

* WIP3

* Remove comment/better comments

* Remote deployment updates

* Adds GH user for yarn deploy

* WIP 6

* WIP7

* WIP8

* WIP 10

* Working easy from home

not so much in the cloud
2022-04-19 14:49:04 -05:00
Topher Lubaway
e615b2faaa Addreses another docs build error (#12084)
* Addreses another docs build error

* Remove two typos

* Adds a test pattern

* Another stab

* More test

* Removing pipeline until more testing

docusaurus deploy isn't into automation
2022-04-15 15:37:51 -05:00
Topher Lubaway
d6b6e4b065 Addressing docusaurus deploy (#12083)
error is unclear and only happens in cloud
2022-04-15 14:43:13 -05:00
Topher Lubaway
ff7b2eecf5 Specifies a git user to address build error (#12082)
error:
[INFO] Deploy command invoked...
Error:  Error: Please set the GIT_USER environment variable,
or explicitly specify USE_SSH instead!
2022-04-15 14:16:32 -05:00
Topher Lubaway
8f6acc72be Cloud's version of which doesn't support -s (#12080)
so we removed -s and sent the output to the dark place
2022-04-15 14:02:16 -05:00
Topher Lubaway
6dd1daf6a0 Toph docs deployment workflow (#11991)
* Adds deploy docusaurus command

(won't work yet. commit for testing)

not force pushing

* Adds GH token workflow for GH actions

* Small comment changes

* Adds branch name to push

* adds gh switch

* Commit for clear dir for testing

* Addresses comments by adding comments
2022-04-15 13:32:55 -05:00
Parker Mossman
197806102e make sure VERSION is up to date after bumpversion (#11945) 2022-04-12 15:36:43 -07:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763)
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
2022-04-11 20:51:37 -07:00
Alexandre Girard
266f05cecd Add script to setup venv for a connector (#11699)
* Add script to setup venv for a connector

* new line

* echo

* Add missing requirements file

* Add script to add venvs to intellij

* script to install venv

* Update

* This should be easier to use

* Better default

* Add unit tests

* add install venv option flag

* remove references to my own own home directory

* Call tools/bin/setup_connector_venv.sh

* Skip module if it already exists in the jdk table

* update

* Set right test name

* better interface

* update

* rename flag

* move to a function

* fix variable name

* derp

* Use real path

* Update setup instructions

* update instructions

* Instructions to both install the venv and update intellij

* update link

* Add command to install venv for all connectors

* <>

* newline
2022-04-08 17:18:14 -07:00
Augustin
582baf3b68 gradle: create a separate sub build for all connectors (#11833) 2022-04-08 10:35:37 -07:00
Topher Lubaway
c8337ea3ba Add docs sidebar (#11817)
* Brings pairity for doc checks from cloud

* WIP Sidebar config

It's sketchy JSON let's commit before we break i

* MVP sidebars

summary.md is no longer functional
it's higherarchy was translated into sidebar.js
as a JSON format that docusaurus likes

* Removes another unused readme
2022-04-07 14:16:35 -07:00
Augustin
5e676feb12 Publish command: call docker hub api to check the tag was registered (#11654) 2022-04-04 16:14:03 -07:00
Jared Rhizor
a809514ea7 improve naming of cloud integration tests (#11713) 2022-04-04 16:01:22 -07:00
Topher Lubaway
d82c2a0505 Same logic as before with more clear user facing (#11700)
Old code was hard to understand errors from.

this should communicate problems more clearly
2022-04-04 13:16:16 -07:00
Greg Solovyev
afc1976a80 Skip empty line output from gradle (#11667)
Currently, every time this script attempts to launch a github action for an empty connector name
  see log output here: https://github.com/airbytehq/airbyte/runs/5786705935?check_suite_focus=true
  This change fixes this minor bug.
2022-04-04 09:47:47 -07:00
Jared Rhizor
493f0ea9f6 use Kubernetes watch api for retrieving exit codes (#11083)
* use kubernetes api for retrieving exit codes

* undelete test

* clean up more status check interval

* fmt

* wip

* clean up

* smarter filtering

* reordering

* exception handling

* better logging for test + speed up acceptance tests temp

* re-enable running on branch

* fix race condition in test

* add log

* trigger build

* trigger build

* re-run tests with everything enabled

* run tests

* run tests

* clean up

* respond to comments

* fix formatting

* fix whitespace

* remove comment

* 10 -> 5

* log exit code error message
2022-03-31 04:01:55 -07:00
Augustin
2d8f66f9e1 🐛 octavia-cli: specific release job for octavia (#11517) 2022-03-30 20:47:56 +02:00
Augustin
b42639a0e7 rollback build and publish of octavia (#11518) 2022-03-29 15:45:04 -07:00
Augustin
9d46ef83ba 🐙 octavia-cli: bump and publish with same version as Airbyte core (#11464) 2022-03-29 19:22:04 +02:00
Augustin
55ae3f856a 🐛 octavia-cli: propagate open api spec update (#11441) 2022-03-29 09:31:00 +02:00
Topher Lubaway
906887c327 Use shell hacks to find valid PAT (#11439)
* Use shell hacks to find valid PAT

if rate limit exceeded try another key

* Use tim's version of the ec2 runner

this is a weird solution with weird limitations.  should address the short term pain (march 2022)
2022-03-28 15:17:07 -05:00
Jared Rhizor
0191af9bf3 build acceptance test connector images for m1 (#11374)
* build acceptance test connector images for m1

* executable

* rename file

* add comment
2022-03-24 06:53:27 -07:00
Charles
c1c8675366 Add readmes to all modules (#8893) 2022-03-13 14:45:36 -07:00
girarda
fd13ab1970 Fix CONNECTORS_BASE build for M1 (#10925)
* Pass in arm64

* default value

* fix
2022-03-07 17:30:41 -08:00
Tim Roes
d73af697e4 Use cypress dashboard and stabilize e2e tests (#10807)
* Record e2e tests to cypress dashboard

* Make env variable accessible in script

* Improve e2e_test script

* Properly wait for server to be ready

* Isolate test suites better

* More test isolation

* Revert baseUrl for development
2022-03-03 18:29:07 +01:00
Octavia Squidington III
74a05e4604 Bump Airbyte version from 0.35.43-alpha to 0.35.44-alpha (#10789)
* Bump Airbyte version from 0.35.43-alpha to 0.35.44-alpha

* Commit.

* Add exception block.

* Why would having try catch work?

* Add logging to figure out.

* Undo all debugging changes.

* Better comments.

Co-authored-by: davinchia <davinchia@users.noreply.github.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>
2022-03-02 22:47:26 +08:00
Maksym Pavlenok
61b0e9e196 🎉 Single py checker (#10246) 2022-02-21 14:01:42 +02:00
girarda
7f6dd9b6fc Fix build by updating broken tests and freezing MarkupSafe version (#10468)
* freeze version

* fix test

* Fix another test
2022-02-18 18:18:16 -08:00
Jared Rhizor
a66d8be03a continue workflows on restarts (#10294)
* fix normalization output processing in container orchestrator

* add full scheduler v2 acceptance tests

* speed up tests

* fixes

* clean up

* wip handle worker restarts

* only downtime during sync test not passing

* commit temp

* mostly cleaned up

* add attempt count check

* remove todo

* switch all pending checks to running checks

* use ++

* Update airbyte-container-orchestrator/src/main/java/io/airbyte/container_orchestrator/ContainerOrchestratorApp.java

Co-authored-by: Charles <giardina.charles@gmail.com>

* Update airbyte-workers/src/main/java/io/airbyte/workers/temporal/sync/LauncherWorker.java

Co-authored-by: Charles <giardina.charles@gmail.com>

* add more context

* remove unused arg

* test on CI that no_retry is insufficient

* revert back to orchestrator retry

* test for retry logic

* remove fialing test and switch back activity config to just no retry

Co-authored-by: Charles <giardina.charles@gmail.com>
2022-02-17 15:14:51 -08:00