* update version in cloud
* 🤖 Auto format source-google-directory code [skip ci]
* increased connectors version
---------
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
* Move icons to connector folder
* Delete old icons
* Update upload logic
* Add icon url to definitions
* Update registry model
* Populate cdn url
* DNC butcher the pipeline
* Low hanging fruit fixes
* Fix bucket name
* Merge old and new approaches
* Fix metadata upload step
* Format
* Fix test
* rename catalog to registry in metadata service
* rename catalog to registry in metadata files
* Run generate models
* Fix missed renames
* Add github personal access token
* Run black
* Automated Change
---------
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
Scripts to
* Run CATs against the local CDK for one connector
* Run CATs against the local CDK for multiple connectors
* Create a connecter image with the local CDK
---------
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Dockerfile to 3.9
* Python version
* More python updates
* 3.9 on GitHub actions and lint updates
* Test out 3.9.11 on GitHub actions
* install python with an action
* formatting: newline
* Also has python code
* only check first level for changed modules
Previous example (source-google-search-console/credentials)
* Test failure: there is no logger.trace
* Source Google Directory #6265 - add oauth support
* Source Google Directory #6265 - update credentials
* Source Google Directory #6265 - fixing according to PR
* Source Google directory #6265 - update docs
* Source Google directory #5190 - update doc
* Source Google Directory #6265 - resolve merge conflict
* Source Google Directory #6265 - SAT for oauth
* Source Google Directory #6265 - bump version and update changelog
* Source Google Directory #6265 - bump version and update changelog (update publish)
* Source Google Directory #7415 - migrate to the CDK
* Source Google Directory #5190 - fix timeout error
* Source Google Directory #7415 - fix according to PR review
* Source Google Directory #7415 - added etag and lastLoginTime to ignored fields for full refresh acceptance test
* Source Google Directory #7415 - fix full refresh acceptance test config
* Source Google Directory #7415 - bump version
* add AIRBYTE_ENTRYPOINT for kubernetes support
* bump versions
* bump version in seed
* Update generic template
* keep scaffold sources at 0.1.0
* add missing newline
* handle python base versions correctly
* re-bump mysql and postgres sources
* re-bump snowflake destination
* add skip tests option
* switch to running tests
* reverse conditional to make it safer
* fix publish to include the test running
* fix iterable version
* fix file generation
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Template generation for new Source using the Santa CDK - provide basic scaffolding for someone implementing a new source.
General approach is to buff up comments in the original SDK, and add TODOs with secondary comments in the generated stub methods, as well as links to existing examples (e.g. Stripe or ExchangeRate api) users can look at.
Checked in and added tests for the generated modules.
* fix connector base versioning
* only run integration tests in non-bases directories
* upgrade datamodel-code-generator which coincidentally has already fixed this issue with generating List of Lists
* bump forgotten hubspot version
* fix build
Instead of inserts, we write the data to S3 and issue a COPY command copying from S3 into redshift.
Use a single file as its sufficiently performant and we do not want to introduce file destination related operations yet.
Use an open source library for uploads as AWS does not natively support streaming loads.
My intention with this PR is to first implement the meat of the write-and-copy strategy. This is mainly centered around the RedshiftCopier class. I plan hook up the RedshiftCopier to the actual Destination class, and implement all the plumbing, in a follow up PR.
Co-authored-by: Davin Chia <davinchia@Davins-MacBook-Pro.local>
* Google directory source #2110 - creating new source
* Google Directory #2110 - implementing new source
* Google directory #2110 - handling rate limit
* Google Directory #2110 - handling errors and rate limits
* Google Directory #2110 - reformat
* Google Directory #2110 - adding CI credentials
* Google Directory #2110 - adding to the source definition registry
* Google Directory #2110 - adding to the source definition registry(fix)
* Google Directory #2110 - injecting the config into the build environment
* Update google-directory.md
* Update google-directory.md
* Google directory #2110 - rename max_results to results_per_page and increase it to 100, fixing setup.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
