jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-19 10:00:34 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,862 Commits 5,105 Branches 518 Tags
master
Commit Graph

851 Commits

Author SHA1 Message Date
Jose Pefaur
d2a4813a9b ci: make cdk connector compatibility test create gh issue (#67611) 
## What
We have a workflow that once a day checks if the latest CDK version
would break any connector. If there are test failures, it sends a slack
alert. In addition to sending a slack alert, we would like to get an
oncall ticket in our github project. We should only do this for
destinations (connectors owned by move)

## How
Added a few extra steps to the workflow that calls a script that creates
gh issues. It checks if an issue already exists for the connector. If it
does it will update the description with information about the new
failure. If it doesn't it will create a new ticket

## How did I test this?
Tested in this PR https://github.com/airbytehq/airbyte/pull/67617.
Confirmed that tickets were created if they didn't exist and that they
were updated if they did exist.

## Can this PR be safely reverted and rolled back?
<!--
* If unsure, leave it blank.
-->
- [x] YES 💚
- [ ] NO ❌
 2025-10-16 12:37:37 -05:00
Jose Pefaur
0ddf24b639 ci: run cdk compatibility test only on certified connectors (#67604) 
## What
The existing `CDK Connector Compatibility Test` workflow currently tests
all connectors using the bulk CDK. However, we only want to test
certified connectors. This PR modifies the workflow to run tests
exclusively on certified connectors using the CDK.

## How
Instead of getting the list of connectors to test from the diff
generated by running a CDK upgrade task, we will get the list of
certified connectors by reusing the script used in the auto bump
workflow to get this information.

## How did I test this?
Tested it in [this](https://github.com/airbytehq/airbyte/pull/67605) PR.
More specifically, here's the PR check run
https://github.com/airbytehq/airbyte/actions/runs/18410594109/job/52461620803?pr=67605

Note: it looks like the connectors we are testing didn't change. I'd
guess it is possible that in the future we will have some none certified
connectors using the cdk so the change in this PR still makes sense.

## Can this PR be safely reverted and rolled back?
<!--
* If unsure, leave it blank.
-->
- [x] YES 💚
- [ ] NO ❌
 2025-10-16 09:19:14 -05:00
Edward Gao
6dbd718fed Bulk CDK: add GHA cron to bump CDK version in certified connectors (#66826) 
## What
Part of
https://github.com/airbytehq/airbyte-internal-issues/issues/14308.
Example run
https://github.com/airbytehq/airbyte/actions/runs/18284850032, example
PR https://github.com/airbytehq/airbyte/pull/67093/. See also
https://github.com/airbytehq/airbyte-enterprise/pull/257.

The cron runs on the first Monday of every month. Past discussions
wanted every 3 weeks, but that's a PITA to do in GHA.

I added one more commit after that run:
679081a47d.
Just changing the trigger to be on a cron, and marking the PR as ready
for review.

There's no automation on actually merging the PRs, but they'll tag the
appropriate team(s) for review. In principle, reviewers can just click
the merge button on green CI though.

There's some weird behavior where the PR sometimes fails to submit b/c
of push protection rules
([example](https://github.com/airbytehq/airbyte/actions/runs/18235269794/job/51927690655#step:12:119)).
@wennergr is looking into this.

## How
Use the various gradle tasks we've added to upgrade the CDK version.

I wanted to use the [bump-version
command](https://github.com/airbytehq/airbyte/actions/workflows/bump-version-command.yml)
to do the metadata+changelog, but it seems to be
[broken](https://airbytehq-team.slack.com/archives/C02U9R3AF37/p1759499392004199).
Got claude to write a hacky equivalent.

(a lot of the bash stuff is also from claude, but heavily edited for
readability+comments+style. I've done enough testing to be confident in
it though.)

## Can this PR be safely reverted and rolled back?
<!--
* If unsure, leave it blank.
-->
- [x] YES 💚
- [ ] NO ❌
 2025-10-07 15:15:36 -07:00
Davin Chia
3851e5053d chore: removed unused tooling scripts. (#62825) 
I cannot find usages for these scripts in the repo, suggesting they are unused and can be removed.
 2025-07-07 09:21:04 -07:00
Benoit Moriceau
95b4f29b96 feat: improved bulk cdk bump flow (#58648) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>
 2025-05-02 11:28:11 -07:00
devin-ai-integration[bot]
7b485beb44 chore: bump schema-generator version to 0.1.1 and update poetry.lock (#54878) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: natik@airbyte.io <natik@airbyte.io>
 2025-03-01 18:29:31 +00:00
Natik Gadzhi
331c4a3721 chore: delete unused internal tools (#53169) 2025-02-04 14:52:37 -08:00
Aaron ("AJ") Steers
83ecbe0fc3 CI: apply pre-commit format fix from #49806 (#49852) 2024-12-18 14:05:43 -08:00
Natik Gadzhi
439fbe0739 chore(live-tests): comments for record_obfuscator.py (#46661) 2024-10-09 09:44:43 +02:00
Catherine Noll
32c45a9182 Create record obfuscator and use it in live tests (#43318) 2024-08-09 12:37:10 -04:00
Natik Gadzhi
4a4a45675a chore(schema-generator): bump schema generator certifi & other deps (#41022) 2024-07-08 08:01:44 -07:00
Natik Gadzhi
cb80e6922a [tools] prettier rules for .md + formatting cleanup 2024-05-07 08:19:33 -07:00
Natik Gadzhi
9bc1f72dd9 Remove unused tools: spec-linter (#37453) 2024-04-23 17:54:47 -07:00
Natik Gadzhi
6269b7f158 [tools] Remove tox.ini (#37454) 2024-04-22 21:32:44 -07:00
Natik Gadzhi
15a32723f6 [schema_generator] Use Poetry for schema_generator (#37455) 2024-04-22 21:32:11 -07:00
Ella Rohm-Ensing
51d1353593 Move source-declarative-manifest to a standard source, published in step with python cdk (#36501) 2024-04-05 11:42:10 -05:00
Marius Posta
0755321569 gradle: remove unused pmd rules (#36195) 2024-03-15 12:11:01 -07:00
Marius Posta
f47db9051b delete bad or useless README files (#36196) 2024-03-15 12:02:23 -07:00
Alexandre Girard
349faeda48 🐛 Install arm64 emulator when building source_declarative_manifest (#35839) 2024-03-05 16:19:04 -08:00
Alexandre Girard
a871899540 Revert "Do not build java base image when publishing python cdk" 
This reverts commit 58bde70e0f.
 2024-03-05 13:35:11 -08:00
Alexandre Girard
58bde70e0f Do not build java base image when publishing python cdk 2024-03-05 13:32:45 -08:00
Alex Birdsall
602d3ce063 Fix syntax error in tools/bin/manage.sh, used to publish airbyte cdk (#35466) 2024-02-21 11:32:34 -08:00
Marius Posta
088b9b7762 .github: fix python CDK publish (#35391) 2024-02-19 07:23:22 -08:00
Marcos Marx
27b0490229 Remove octavia-cli (#33950) 2024-02-16 18:20:43 -03:00
Marius Posta
dc088bc3a9 gradle: split off python cdk (#35306) 2024-02-16 04:39:14 -08:00
Aaron ("AJ") Steers
d083d15665 Docs fix: ConfiguredAirbyteCatalog refs (#33483) 2023-12-14 08:39:29 -08:00
Augustin
098a285b2f airbyte-ci format: exclude .gitignored files from format (#33249) 
Co-authored-by: Marius Posta <marius@airbyte.io>
 2023-12-13 10:33:37 +01:00
Augustin
0b33caecda Revert "[skip ci] formatting: add missing license headers (#33250)" (#33289) 2023-12-11 11:38:37 +01:00
Augustin
60c1cc01ad [skip ci] formatting: add missing license headers (#33250) 2023-12-11 10:15:18 +01:00
Sajarin
b5053c2512 🎉 New Source: Klaus API connector (#30993) 
Co-authored-by: Deke Li <deke.li@sendinblue.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
 2023-12-04 19:18:34 -03:00
Ben Church
13b631ca91 CI: Add approve-and-merge back as admin only. (#33028) 2023-12-01 16:04:34 -08:00
Ben Church
9ab20ebdaf Revert "Admin approve and merge" 
This reverts commit 262f93dde5.
 2023-12-01 14:58:14 -08:00
Ben Church
262f93dde5 Admin approve and merge 2023-12-01 14:47:36 -08:00
Ella Rohm-Ensing
ac3eb28de2 airbyte-ci: add format commands (#31831) 
Co-authored-by: Ben Church <ben@airbyte.io>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Marius Posta <marius@airbyte.io>
Co-authored-by: alafanechere <alafanechere@users.noreply.github.com>
 2023-11-14 02:17:48 -06:00
Marius Posta
7cd8020ac8 java CDK: hoist top-level gradle projects into CDK (#31960) 
Co-authored-by: postamar <postamar@users.noreply.github.com>
 2023-10-30 12:03:06 -07:00
Xiaohan Song
99b435ff71 Support incremental sync in performance harness tool (#31588) 2023-10-20 10:29:52 -07:00
Xiaohan Song
a946a3840b Integrate source performance harness with datadog (#31410) 
Co-authored-by: xiaohansong <xiaohansong@users.noreply.github.com>
 2023-10-17 09:57:24 -07:00
Marius Posta
53735b6d37 gradle: remove SUB_BUILD env var and references (#31027) 2023-10-04 14:58:22 -07:00
Sajarin
0fe0b911a1 [Non-forked] ✨ Source Younium: Adding Booking and Account streams (#31064) 
Co-authored-by: Hans Peter Hagblom <hans.peter.hagblom@eletive.com>
 2023-10-04 17:51:09 -04:00
Marius Posta
f8edc18039 airbyte-ci,gradle: replace airbyte-docker with airbyte-ci (#30743) 2023-10-04 08:38:17 -07:00
Marius Posta
7ae97175a6 gradle: fix repo wide behaviour (#30607) 2023-09-28 05:01:13 -07:00
Aaron ("AJ") Steers
da32fc86bf 🚀 Launch Java CDK v0.1.0 (PR #30383) 
- Launch Java CDK `v0.1.0`. 🚀
- Remove `airbyte-docker` gradle plugin from java connectors.
- Add `airbyte-java-connector` gradle plugin.
- Remove `airbyte-commons` from global build.gradle.
- Add `/publish-java-cdk` slash command.
- Initial launch of `/airbyte-ci` slash command to test connectors.
 2023-09-27 15:46:02 -07:00
Evan Tahler
de188beff6 Remove destination-bigquery-denormalized (#30369) 
Co-authored-by: evantahler <evantahler@users.noreply.github.com>
 2023-09-25 12:53:38 -05:00
Sajarin
5a2a3b81b7 tools: update schema_generator to use latest version of cdk (#30673) 2023-09-25 10:28:31 -04:00
Evan Tahler
3c113a24b0 Move docs.airbyte.com hosting to Vercel (#30591) 2023-09-19 16:52:10 -05:00
Ben Church
5c56ac1d84 Airbyte-ci: Remove gradle task connectorAcceptanceTest (#30326) 2023-09-19 15:16:37 -05:00
Marius Posta
ef2849e35e gradle: fix airbyteDocker task inputs (#30187) 2023-09-07 03:46:31 -07:00
Marius Posta
be1e1adabd gradle: cleanup (#30060) 2023-09-05 14:05:40 -05:00
Marius Posta
d5fdb1b982 .github,tools: clean up dead files (#30011) 2023-08-31 13:34:41 -07:00
Marius Posta
3e680675a4 github workflows: repo-wide auto-format (#29798) 
Co-authored-by: postamar <postamar@users.noreply.github.com>
 2023-08-25 10:20:41 -07:00