1
0
mirror of synced 2026-01-04 09:04:47 -05:00
Commit Graph

7 Commits

Author SHA1 Message Date
Octavia Squidington III
74a05e4604 Bump Airbyte version from 0.35.43-alpha to 0.35.44-alpha (#10789)
* Bump Airbyte version from 0.35.43-alpha to 0.35.44-alpha

* Commit.

* Add exception block.

* Why would having try catch work?

* Add logging to figure out.

* Undo all debugging changes.

* Better comments.

Co-authored-by: davinchia <davinchia@users.noreply.github.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>
2022-03-02 22:47:26 +08:00
Charles
058bb2a424 State Storage for new Scheduler (#8613) 2021-12-10 09:52:26 -08:00
Davin Chia
341f505a94 Rename env vars for better readability. (#8447)
* Rename GcsStorageBucket to GcsLogBucket.

* Update all references to GCP_STORAGE_BUCKET to GCS_LOG_BUCKET.

* Undo this for configuration files for older Airbyte versions.

* Clean up Job env vars. (#8462)

* Rename MAX_SYNC_JOB_ATTEMPTS to SYNC_JOB_MAX_ATTEMPTS.

* Rename MAX_SYNC_TIMEOUT_DAYS to SYNC_JOB_MAX_TIMEOUT_DAYS.

* Rename WORKER_POD_TOLERATIONS to JOB_POD_TOLERATIONS.

* Rename WORKER_POD_NODE_SELECTORS to JOB_POD_NODE_SELECTORS.

* Rename JOB_IMAGE_PULL_POLICY to JOB_POD_MAIN_CONTAINER_IMAGE_PULL_POLICY.

* Rename JOBS_IMAGE_PULL_SECRET to JOB_POD_MAIN_CONTAINER_IMAGE_PULL_SECRET.

* Rename JOB_SOCAT_IMAGE to JOB_POD_SOCAT_IMAGE.

* Rename JOB_BUSYBOX_IMAGE to JOB_POD_BUSYBOX_IMAGE.

* Rename JOB_CURL_IMAGE to JOB_POD_CURL_IMAGE.

* Rename KUBE_NAMESPACE to JOB_POD_KUBE_NAMESPACE.

* Rename RESOURCE_CPU_REQUEST to JOB_POD_MAIN_CONTAINER_CPU_REQUEST.

* Rename RESOURCE_CPU_LIMIT to JOB_POD_MAIN_CONTAINER_CPU_LIMIT.

* Rename RESOURCE_MEMORY_REQUEST to JOB_POD_MAIN_CONTAINER_MEMORY_REQUEST.

* Rename RESOURCE_MEMORY_LIMIT to JOB_POD_MAIN_CONTAINER_MEMORY_LIMIT.

* Remove worker suffix from created pods to reduce confusion with actual worker pods.

* Use sync instead of worker to name job pods.
2021-12-03 23:28:48 +08:00
Davin Chia
6565a23af5 Test Kube Log4j2 Logging Configuration (#4972)
- Use JUnit 5 tags for a cleaner build set up + make sure tests are not run unnecessarily.
- The various env set ups are replicated in the cloud_storage_logging_test.sh script. Since there isn't an easy way to dynamically inject env vars from within a JVM, this was the cleanest way of doing this.
2021-07-27 13:47:51 +08:00
Charles
76adc25820 Split Platform and Connector Builds (#4514) 2021-07-15 10:49:15 -07:00
Davin Chia
e2074a4dc1 Logging to GCS. (#4501)
Add the ability to log to GCS.
2021-07-07 21:06:25 +08:00
Davin Chia
e60f5369e0 🎉 Kube Logs support stored and reading from S3. (#4053)
Use Log4j2 appender to support routing logs to S3.

Create LogClient to support reading from S3.

Some clean up of the Log4j2 xml variables.

Several dependency changes to be more explicit when configuring jackson.
2021-06-19 12:15:15 +08:00