1
0
mirror of synced 2025-12-22 03:21:25 -05:00
Commit Graph

60 Commits

Author SHA1 Message Date
Natik Gadzhi
4982562054 docs: document contributing via Builder (#44536) 2024-08-22 14:51:05 -07:00
Bryce Groff
96d1f19f72 docs: refactor the developing locally page around abctl (#42496)
Co-authored-by: Evan Tahler <evan@airbyte.io>
2024-08-01 07:55:13 -07:00
Marcos Marx
62ee7026e6 Docs: update contributing guides (#27812)
* changes

* update how to pr and readme

* update contributeing guides

* update code of conduct text

* update submit new connector

* submit new connector doc

* make one line for submit new conn find request

* improve words
2023-06-30 15:00:15 -06:00
Rodi Reich Zilberman
0bab1756b8 Rename airbyte-config module (#24885)
* rename airbyte-config module

* Automated Commit - Formatting Changes

* sanity

* update import

* update import

* update script

* update script

* update script

* update script

* Automated Change

* Automated Change

* Automated Change

* Automated Change

* update awsdatalake icon

* point slash commands to new path

* sanity

* Automated Commit - Formatting Changes

* sanity

* Automated Change

* Automated Change

* sanity

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
2023-04-06 10:47:30 -07:00
Amruta Ranade
8098bb45ba Updated the local development guide (#23400) 2023-02-23 14:14:59 -05:00
Conor
610c9c01df Update developing-locally.md (#22947)
* Update developing-locally.md

* Update developing-locally.md

* update local deployment docs
2023-02-21 13:13:28 -06:00
Joe Reuter
aace7ae577 🪟🔧 Connector builder: Do not rely on local cdk anymore (#22391)
* do not rely on local cdk anymore

* cache versioned schemas

* fix used version

* review comments
2023-02-07 16:53:55 +00:00
Tim Roes
86fbcdc52f 🪟 🔧 Switch to pnpm for package managing (#22053)
* WIP

* WIP

* Adjust lockfile

* Fix issues

* Adjust github flows

* More fixes

* Adjust documentation

* Remove unnecessary tsconfig entry

* Cleanup .eslintrc

* Update dependencies

* Fix more issues

* Correct TS

* Make pnpm install run always via gradle

* Install @storybook/react explicitaly

* Update snapshots due to new libraries

* Add node-fetch types

* Add `url` package

* Freeze react router version (due to useBlocker)

* Fix typing issue

* Update airbyte-webapp/src/components/ui/BarChart/BarChart.tsx

Co-authored-by: Lake Mossman <lake@airbyte.io>

* Rewrite README

* Update e2e test library

* Remove retry

* Latest pnpm version

* Revert removal of retry logic

---------

Co-authored-by: Lake Mossman <lake@airbyte.io>
2023-02-06 16:29:21 +01:00
Jimmy Ma
c6b8279386 Add dependencies to docker-compose (#19321)
* Revert "Revert "Add dependencies to docker-compose (#19257)" (#19306)"

This reverts commit b16f28fd4d.

* Bump docker-compose version

* test

* Revert "test"

This reverts commit c3b28c9ec3.

* Bump docker compose version to 3.8

* test

* test

* Use a newer docker for platform build

* test

* Use docker compose v2

* Update docker for frontend e2e

* Use docker compose v2 for e2e tests

* Update tools to use docker compose v2

* Update more docker compose references to v2

* Update docker compose usage in docs

* Update deploying airbyte docs

* Fix permission

* Case insensistive check

* Add extra mention for docker compose in upgrading airbyte

* Improve upgrade docker note
2023-01-12 10:41:00 -08:00
Ella Rohm-Ensing
426fe04d55 add authentication info to local development guide (#18509)
* add authentication info to local development guide

* rephrase
2022-10-27 11:11:41 -04:00
Joey Marshment-Howell
39c551279e Update docs for local webapp development with basic auth (#18288)
* add note to disable basic auth when developing the UI locally
2022-10-21 13:45:02 +01:00
Alex
4f03c09d48 Add documentation for running the dev image of a connector (#16176)
* add documentation for running the dev image of a connector

* Apply suggestions from code review

Co-authored-by: Augustin <augustin.lafanechere@gmail.com>

* remove unwanted directory from PR

Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
2022-09-01 13:25:36 -05:00
Evan Tahler
110fe5d7fe LOG_CONNECTOR_MESSAGES enables logging of all messages seen by the worker in a sync (#15895)
* `LOG_CONNECTOR_MESSAGES` to enable logging of all messages seen by the worker in a sync

* also log initial state

* linting

* getState 2x for to match input file

* use `Jsons.serialize` utility

* add note to `developing-locally`

* fix bad merge
2022-08-24 10:29:48 -07:00
Yiyang Li
133ae07b35 Docs: improve doc for contributing locally (#14661)
* improve doc for contributing locally

- the format command for an individual connector is airbytePythonFormat
- provide examples for the format commands

* add `./gradlew format` for java connectors
2022-07-15 21:36:05 -03:00
Topher Lubaway
9c6c092a22 Revert "Improving docusaurus sidebar generation (#1927) (#14369)" (#14596)
This reverts commit a2c194a11f.
2022-07-11 15:27:14 -05:00
Mykyta Serbynevskiy
a2c194a11f Improving docusaurus sidebar generation (#1927) (#14369)
* Improving docusaurus sidebar generation (#1927)

* Added "Career & open positions" folder to sidebar, adjusted "Project overview" folder

* Deleted "career-and-open-positions" folder from sidebar
2022-07-08 14:18:27 -05:00
Evan Tahler
be8d8574f2 Remove M1-specific docs - Airbyte works on M1 Macs now (#11856) 2022-05-24 16:46:26 -07:00
Evan Tahler
3e4263a5bd More ways to reset your local environment (#12826) 2022-05-18 08:46:04 -07:00
Jared Rhizor
14dfc1d417 fix parentheses and info/warning/danger formatting in docs (#11981)
* fix parens

* fix formatting for hints

* remove hint from non-docusaurus or gitbook doc
2022-04-13 12:08:34 -07:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763)
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
2022-04-11 20:51:37 -07:00
Parker Mossman
884a94ed29 Un-Revert OSS branch build for Cloud workflow (#11808)
* Revert "Revert "Build OSS branch for deploying to Cloud env (#11474)""

This reverts commit 55e3c1e051.

* add action to get dev branch tag to OSS project instead of doing it in cloud

* remove dev branch version action, going to do this in cloud afterall
2022-04-08 15:17:04 -07:00
lmossman
55e3c1e051 Revert "Build OSS branch for deploying to Cloud env (#11474)"
This reverts commit 189efe7b42.
2022-04-05 15:44:31 -07:00
Parker Mossman
189efe7b42 Build OSS branch for deploying to Cloud env (#11474)
* add VERSION buildArg to Dockerfiles, default to current airbyte version but overwritable

* use VERSION env var consistently as Dockerfile buildArg, jar version, and tag

pass version and image_tag into docker build task function

* add github action for building and pushing an OSS branch for Cloud to consume

* allow AirbyteVersion to validate versions containing 'oss-branch' prefix

* change oss-branch prefix to dev for branch-based versions

* better action name

* add docker-compose-cloud.build.yaml to define minimum set of cloud images that are pushed by oss branch action

* update local dev docs to describe optional usage of VERSION env var

* make branch_version_tag input optional, if not provided, generates dev-<commit_hash>

* fix typo

* fix missed merge conflict

* update docker docs

* update integrationRunner isDev check
2022-04-05 15:06:17 -07:00
Jared Rhizor
526c5a0c64 automatically figure out ARM-related variables (#11450)
* automatically figure out ARM-related variables

* clean up

* explicitly use amd64-prefixed versions of alpine and postgres
2022-03-31 13:58:46 -07:00
Greg Solovyev
1c41517e34 Fix instructions for running acceptance tests (#10846)
1) Remove requirement for local Postgres, because it is no longer needed
2) Fix bash code snippet for running acceptance tests (remove the line that starts Airbyte)
3) Document how to use USE_EXTERNAL_DEPLOYMENT flag
2022-03-04 15:41:30 -08:00
Greg Solovyev
6c92fec7ea Add a note about running only in dev mode on M1 (#10772)
Macs with M1 chip can run Airbyte only in dev mode right now, so to make it clear, I added a note about it and moved the hint about M1 chips to the top of the section.
2022-03-01 11:16:17 -08:00
Abhi Vaidyanatha
15cd088e8a Clarify Connector Spec Caching. (#7664) 2022-01-14 00:29:52 -08:00
Parker Mossman
1fe641a8d3 use airbyte/temporal-auto-setup which has multi-arch support (#8888) 2021-12-20 15:17:03 -08:00
Jared Rhizor
25674fc306 upgrade to Gradle 7.3.1 / Java 17 (#7964)
* upgrade gradle

* upgrade to Java 17 (and fix a few of the node versioning misses)

* oops

* try to run a different format version

* fix spotless by upgrading / reformatting some files

* fix ci settings

* upgrade mockito to avoid other errors

* undo bad format

* fix "incorrect" sql comments

* fmt

* add debug flag

* remove

* bump

* bump jooq to a version that has a java 17 dist

* fix

* remove logs

* oops

* revert jooq upgrade

* fix

* set up java for connector test

* fix yaml

* generate std source tests

* fail zombie job attempts and add failure reason (#8709)

* fail zombie job attempts and add failure reason

* remove failure reason

* bump gcp dependencies to pick up grpc update (#8713)

* Bump Airbyte version from 0.33.9-alpha to 0.33.10-alpha (#8714)

Co-authored-by: jrhizor <jrhizor@users.noreply.github.com>

* Change CDK "Caching" header to "nested streams & caching"

* Update fields in source-connectors specifications: file, freshdesk, github, google-directory, google-workspace-admin-reports, iterable (#8524)

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* move S3Config into destination-s3; update dependencies accordingly (#8562)

Co-authored-by: Lake Mossman <lake@airbyte.io>
Co-authored-by: jrhizor <jrhizor@users.noreply.github.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: Iryna Grankova <87977540+igrankova@users.noreply.github.com>
Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
2021-12-10 16:57:54 -08:00
Harsha Teja Kanna
97594c7b20 Added temporal build script (#8325) 2021-11-30 13:24:31 -08:00
Jared Rhizor
ce3426c0e4 upgrade to node 16 (#7724)
* upgrade to node 16

* does upgrading fontawesome prevent errors
2021-11-08 12:55:25 -08:00
Harsha Teja Kanna
45f6559c79 Fixed Mac M1 build (#7687) 2021-11-05 16:14:36 -07:00
LiRen Tu
779c39c088 Copy job attempt state to configs database (#7219)
* Add migration to create latest state table

* Log migration name

* Expose db variables to airbyte-db

* Implement migration

* Fix migration test

* temp

* Rebase on master

* Save state in temporal (#7253)

* Copy state to airbyte_configs table

* Add standard sync state

* Move state methods to config repository

* Add unit tests

* Fix unit tests

* Register standard sync state in migration

* Add comment

* Use config model instead of json node

* Add comments

* Remove unnecessary method

* Fix migration query

* Remove unused config database

* Move persist statement and log the call

* Update dev doc

* Add unit tests for sync workflow

Co-authored-by: Charles <giardina.charles@gmail.com>
2021-10-25 17:08:08 -07:00
LiRen Tu
e87604eccb Update doc about m1 (#7233) 2021-10-20 23:59:02 -07:00
LiRen Tu
e3be3d2ad0 Update M1 doc and fix doc links (#7205)
* Add note about jdk 14 vs 17 for m1

* Fix doc links
2021-10-19 22:31:16 -07:00
Jared Rhizor
c2eabc182d m1 pr cleanups (#7197) 2021-10-19 14:53:47 -07:00
Harsha Teja Kanna
3e7f95c25a 🎉 Support build on MacOS M1 (Apple Silicon) (#7104)
- See this doc for details: https://github.com/airbytehq/airbyte/blob/master/docs/contributing-to-airbyte/developing-locally.md
- Unit test does not work yet.
2021-10-19 11:20:21 -07:00
Charles
d154f8b278 update docs to avoid long run times for formatting (#7064) 2021-10-15 13:04:42 -07:00
Abhi Vaidyanatha
ae32ecbb27 GitBook: [master] 186 pages and 77 assets modified 2021-10-08 21:17:47 +00:00
LiRen Tu
833dce16f8 📝 Add link to java 14 in documentation (#6252) 2021-09-19 12:22:50 -07:00
Charles
76adc25820 Split Platform and Connector Builds (#4514) 2021-07-15 10:49:15 -07:00
Charles
335f5edef8 copy seed configurations from config/init to server (#4417)
Co-authored-by: subodh <subodh1810@gmail.com>
2021-06-30 10:38:19 -07:00
Subodh Kant Chaturvedi
887752822c 🎉 introduce automatic migration at the startup of server for docker environment (#3980)
* introduce automatic migration at the startup of server

* handle versions with non-zero patch

* it works!!!

* add dummy data

* cleanup orphan configs

* add more assertions

* format + add comments

* move migration acceptance test to acceptance test directory

* add automatic migration test to the build

* address review comments

* missed out on these

* format

* add more assertions

* format

* fix test

* format

* use default port for temporal

* move seed to server + introduce atomice replacement for config

* make tests better

* remove unwanted changes

* move atomic replacement logic behind persistence + pass path to latest seeds

* format

* update seeds

* review comments

* update seeds

* merge latest seeds with configs

* fix bug around latest seed

* update seed

* update seed

* seeds should be populated by separate container

* address review comment + change latest definition url

* update seeds

* format

* update seed references

* update seed

* update seed

* update seed

* update seed references

* update seed references + add Migration Acceptance Test

* update seed container in kube + disable automatic migration for kube + update docs

* update docs

* address review comments from Michel

* update doc

* temporary commmit to see if build becomes green

* delete seeds from airbyte config + undo temp commit
2021-06-29 23:50:00 +05:30
Marcos Marx
a18b0e80d0 GitBook: [master] 161 pages and 75 assets modified 2021-05-16 22:51:02 +00:00
Abhi Vaidyanatha
0c3cca2f24 Document usage of ./gradlew format and workaround for files without import statements (#3269)
* Make Spotless license check look for variables.

* Test and confirm.

* Document real workaround.

* Address review comments.

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-05-06 18:33:01 -07:00
Charles
8bfb1170e4 in docs, ./gradlew build => CORE_ONLY=1 ... (#2904) 2021-04-15 10:45:08 -07:00
Charles
cac6808af8 Remove outdated docs about running scheduler and server with gradle run (#2676) 2021-04-08 14:39:39 -07:00
John Lafleur
55256d293d GitBook: [master] 55 pages and 36 assets modified 2021-04-05 00:43:26 +00:00
Sherif A. Nada
05698de749 Fix typo in developing-locally.md 2021-03-09 16:39:54 -08:00
Anthony Hillairet
b3fbc02746 Add troubleshooting in local development (#2344) 2021-03-09 16:39:23 -08:00