* init
* bad copy/paste
* move to top level
* Revert "move to top level"
This reverts commit aca3534d38.
* attempt to wire up connector builder frontend to server
* copy from octaviacli
* fix connection to builder server
* update
* delete
* Update
* delete python-version
* Revert "delete python-version"
This reverts commit f9258a7755.
* setup python
* install python
* rename
* kube stuff
* Install python
* missing kube file
* rename
* Update files
* Update bumpversion
* install python
* try with different entrypoint
* rename container
* point to docker-compose.yaml file
* derp
* copy acceptance_test.sh
* copy from acceptance tests
* delete cruft
* update
* remove application env
* reset
* reset to master
* update
* skip comprehensive incremental tests
* Revert "skip comprehensive incremental tests"
This reverts commit 9cee657596.
* reset to master
* remove cruft
* delete superfluous steps
* update port to 8003
* reset to master
* Update publish docker
* move openapi spec to airbyte-connector-builder
* point to openapi spec
* dont expose the connector builder to localhost
* reset FE components to master
* Don't deploy the connector-builder
* Revert "Don't deploy the connector-builder"
This reverts commit 3d157494cf.
* Revert "Revert "Don't deploy the connector-builder""
This reverts commit beac3d48f0.
* comment out more things related to connector builder server
* more attempts at removing the connector builder
* comment out more things
* Apply suggestions from code review
Co-authored-by: Lake Mossman <lake@airbyte.io>
* Update airbyte-webapp/src/config/configProviders.ts
Co-authored-by: Lake Mossman <lake@airbyte.io>
* update
* rename
* indent
* Revert "move openapi spec to airbyte-connector-builder"
This reverts commit 57dda04723.
* Revert "rename"
This reverts commit b2d802b8fa.
* Revert "Revert "rename""
This reverts commit 91db24fd4a.
* point to wrong file in case it fixes the build
* point to right openapi file
* Revert "Revert "move openapi spec to airbyte-connector-builder""
This reverts commit e46a837454.
* point to moved file
* fix path
* Update from master
* newline
* Add failing test
* Revert "Add failing test"
This reverts commit ed9fea09b5.
* comment
* update commented requires
* Add a comment
* 2022
* rename to connector-builder-server
* typo
Co-authored-by: lmossman <lake@airbyte.io>
* WIP Convert airbyte-workers to Micronaut framework
* Rebase cleanup
* Fix broken tests
* Simplify code
* Support control vs data plane configuration
* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests
* Formatting
* Pairing on Worker Micronaut (#16364)
* add RouteToSyncTaskQueue activity
* use new route activity in connection manager workflow
* format
* call router service for task queue
* Revert temporal proxy changes
* Formatting
* Fix default value
* register new route activity in test
* fix SyncWorkflowTest now that it isn't doing any routing
* Update dependencies
* More dependency updates
* Update dependencies
* Improve conditional bean check
* Match existing Optional functionality
* Add notEquals check
* Add missing env var to Helm chart
* Fix typo
* Mark LogConfigs as Singleton
* Env vars for log/state storage type
* Remove use of Optional in bean declarations
* Fix typo in config property name
* Support Temporal Cloud namespace
* Change to @Value
* Use correct value for conditional check
* Upgrade Micronaut
* Fix merge conflict
* Formatting
* Add missing env var
* Use sync task queue environment variable
* Handle sync task queue as set
* format and force http
* Handle case where sync task queue is empty
* Add correct path to config property
* Remove unused import
* Remove unused parameter
* Formatting
* Use pattern for condition process factory beans
* Cleanup
* PR feedback
* Revert hack for testing
Co-authored-by: pmossman <parker@airbyte.io>
- sort docker env vars.
- remove all non-docker related env vars.
- add what is missing.
For the .env file:
- sort the file to match the Configs.java lay out for better reading.
- get rid of env vars that are not used in docker
- get rid of env vars that have defaults, with the exception of var that are for scaling e.g. submitter_num_threads, worker related vars to prevent the env file from getting too large
- add a header to clarify when/where to add env vars to the file
For the docker compose file:
- sort the env vars alphabetically
- get rid of env vars that aren't used in that application
- add missing env vars into the worker application
* Integrate Sentry for performance and errors tracking.
* Add sentry sensitive data scrubbing.
* updated cdk version and changelog
* Integrate Sentry for performance and errors tracking
Add `SENTRY_DSN` environment variable
* Integrate Sentry for performance and errors tracking
Add `sentry_sdk` to install requirements
* format cdk
* enable Sentry for google-search-console
* updated connector version
* update spec and source yamls
Co-authored-by: auganbay <auganenu@gmail.com>
Co-authored-by: Sergei Solonitcyn <sergei.solonitcyn@zazmic.com>
* Rename GcsStorageBucket to GcsLogBucket.
* Update all references to GCP_STORAGE_BUCKET to GCS_LOG_BUCKET.
* Undo this for configuration files for older Airbyte versions.
* Clean up Job env vars. (#8462)
* Rename MAX_SYNC_JOB_ATTEMPTS to SYNC_JOB_MAX_ATTEMPTS.
* Rename MAX_SYNC_TIMEOUT_DAYS to SYNC_JOB_MAX_TIMEOUT_DAYS.
* Rename WORKER_POD_TOLERATIONS to JOB_POD_TOLERATIONS.
* Rename WORKER_POD_NODE_SELECTORS to JOB_POD_NODE_SELECTORS.
* Rename JOB_IMAGE_PULL_POLICY to JOB_POD_MAIN_CONTAINER_IMAGE_PULL_POLICY.
* Rename JOBS_IMAGE_PULL_SECRET to JOB_POD_MAIN_CONTAINER_IMAGE_PULL_SECRET.
* Rename JOB_SOCAT_IMAGE to JOB_POD_SOCAT_IMAGE.
* Rename JOB_BUSYBOX_IMAGE to JOB_POD_BUSYBOX_IMAGE.
* Rename JOB_CURL_IMAGE to JOB_POD_CURL_IMAGE.
* Rename KUBE_NAMESPACE to JOB_POD_KUBE_NAMESPACE.
* Rename RESOURCE_CPU_REQUEST to JOB_POD_MAIN_CONTAINER_CPU_REQUEST.
* Rename RESOURCE_CPU_LIMIT to JOB_POD_MAIN_CONTAINER_CPU_LIMIT.
* Rename RESOURCE_MEMORY_REQUEST to JOB_POD_MAIN_CONTAINER_MEMORY_REQUEST.
* Rename RESOURCE_MEMORY_LIMIT to JOB_POD_MAIN_CONTAINER_MEMORY_LIMIT.
* Remove worker suffix from created pods to reduce confusion with actual worker pods.
* Use sync instead of worker to name job pods.
See https://airbytehq.slack.com/archives/C019WEENQRM/p1629383779144300 for more details.
TLDR:
Retrying within an attempt is confusing for a UX perspective (we already have attempts), does not provides more value from a retries perspective (we already have attempts) and inefficient from a resource perspective (hogs up api limits and compute/memory resources).
Follow up ticket: #5571
* Making single port 8000 default and update NetworkErrorBoundary.tsx response code 502
* add .env files
* Updating instructions in docs/deploying-airbyte/*
* fix mounting
* svc accounts and other fixes
* remove redundant sync closure call
* switch back to using dev tags and format
* do not overwrite in initContainers
* rename to scheduler-service-account
* add documentation
* fix lack of cp -n on busybox
* use kubectl run, still needs affinity
* add affinity
* downgrade kubectl on scheduler to match docker-desktop latest
* fix overrides
* THE MISISNG PIECE
* add attach permissions
* fmt
* update kube documentation
* put docs on the docs site instead
* initial attempt at generating local kube setup from docker compose
* update current state
* mounts not working
* working mounts, failing cors
* working UI
* add remaining todos
* update todos
* A
* use kustomize to select image versions
* kube process builder factory
* fix misalignment
* don't allow any retries for requested jobs
* fix log waiting and path handling
* update todos
* local volume handling
* propagate return code correctly
* update todos
* update docs
* fmt
* add to docs
* fix conflicting config file bug
* fmt
* delete unused file
* remove comment
* add job id and attempt as inputs
* rename to WorkerEnvironment
* fix example custom overlay
* less trigger-happy docs
* rename mounts
* show local csv as not working in kube in the docs
* use config maps for everything
* fix paths
* fix build
* fix stripe integration test usage
* fix papercups on kube
