* test exposing secrets in configrepo
* fix local persistence sql
* working propagation, just without check/discover replacements and without feature flagging
* switch if statement
* set up secret persistence for google secrets manager
* add ttl-based secret persistence for check/discover usage in the future
* set up check/discover to pass around necessary parts
* Revert "set up check/discover to pass around necessary parts"
This reverts commit 489d2d5f5d.
* working updates + check/discover operations
* fix additional configs created on deletion
* clean up docker compose file
* finish up configrepo
* make api path optional
* clean up schedulerapp and local testing persistence
* make optional in the worker app
* add rest of feature flagging
* fmt
* remove completed todo
* fix refactoring typo
* fix another refactoring typo
* fix compilation error in test case
* fix tests
* final cleanups
* fix conditional
* address a couple of things
* add hydrator interface
* add replaceAllConfigs
* specfetcher handling
* fix constructor
* fix test
* fix typo
* fix merge build error
* remove extra config
* fix integration test
* fix final piece
This solves the problem where you may be running airbyte on an endpoint
that isn't http://localhost:8001
Additionally it improves the error experience when this script can not
reach the API.
* updated mysql tests
* updated mysql tests
* added mysql ssh tunnel tests by key
* fixed remarks
* fixed remarks
* updated DatabricksStreamCopier
* switch to custom file for ssh config in normalization
* updated MySQL SSH tests
* bump version
* get local port properly
* updated assertSameValue for MySQL ssh tunnel
* updated image version and documentation
* updated code style
* updated CI credentials
* updated normalization documentation
Co-authored-by: George Claireaux <george@claireaux.co.uk>
* ssh-test
* add authentification via ssh tunnel with bastion docker host and postgres testcontainer
* created SshBastion class in base-java module
* implement Postgres source basic ssh tunneling connection for integration tests
* implement Postgres source ssh tunneling connection and refactoring SshBastion
* generate keys inside a bastion container
* remove throwing Exception from startTestContainers method
* fix checkstyle
* add documentation and changelog for Posthres source and destination
* update documentation for ssh readme.md | update version fo Postgres source and destination to 0.3.12
* update version of Postgres source and destination to 0.3.12
* removed static variables, removed version bump, rename class to SshBastionContainer, removed ci credentials for ssh Postgres Source and Destination
Co-authored-by: vmaltsev <vitalii.maltsev@globallogic.com>
* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)
* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)
updated image version
* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)
correction of remarks
* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)
bumb image version
* Added identification of accessible streams for API keys with limited permissions
* added credentials for API key with limited permissions (for user stream only)
* update docs and image version
* updated error message
* fixed formatting
* Create secrets manager API against Google Secrets store, plus feature flag on use.
* Run integration test at the right time, with config, for secrets persistence.
* Provide credentials for google secrets store access to tests
* Separate out secrets persistence test from kube for stability
Implement new destination connector for databricks delta lake.
Resolves#2075.
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* docker-compose split of scheduler and worker
* fix heartbeat location bug + add support for kubernetes
* use two workers in integration tests
* capture logs in AirbyteTestContainer
* add waiting
* rename to make it easier to review
* rename module
* fix remaining conflicts
* allow configuring max workers of each type and document usage
* fix build
* remove comment
* add worker resource requiremetns
* try to fix for connector build
* fix regression in biuld
* add env comments for SUBMITTER_NUM_THREADS
* Update airbyte-workers/src/main/java/io/airbyte/workers/WorkerApp.java
Co-authored-by: Davin Chia <davinchia@gmail.com>
* Update airbyte-workers/src/main/java/io/airbyte/workers/temporal/TemporalPool.java
Co-authored-by: Davin Chia <davinchia@gmail.com>
* merge temporalpool into workerapp
* output docker system info
* move check to before
* remove unnecessary parts of the patch
* could this be the problem? i thought i added this
* show disk usage
* add print statements
* add pruning
* fix prune option
* use force
Co-authored-by: Davin Chia <davinchia@gmail.com>
* introduce a github action for open source release
* add more-secrets environment
* PR body should contain the changelog
* update script
* add new lines
