* Added webflow code
* Updated readme
* Updated README
* Added webflow to source_definitions.yaml
* Enhanced documentation for the Webflow source connector
* Improved webflow source connector instructions
* Moved Site ID to before API token in Spec.yaml (for presentation in the UI)
* Addressed comments in PR.
* Changes to address requests in PR review
* Removed version from config
* Minor udpate to spec.yaml for clarity
* Updated to pass the accept-version as a constant rather than parameter
* Updated check_connection to hit the collections API that requires both site id and the authentication token.
* Fixed the test_check_connection to use the new check_connection function
* Added a streams test for generate_streams
* Re-named "autentication" object to "auth" to be more consistent with the way it is created by the CDK
* Added in an explict line to instantiante an "auth" object from WebflowTokenAuthenticator, to make it easier to describe in the blog
* Fixed a typo in a comment
* Renamed some classes to be more intuitive
* Renamed class to be more intuitive
* Minor change to an internal method name
* Made _get_collection_name_to_id_dict staticmethod
* Fixed a unit-test error that only appeared when running " python -m pytest -s unit_tests".
This was caused by Mocked settings from test_source.py leaking into test_streams.py
* format: add double quotes and remove unused import
* readme: remove semantic version naming of connector in build commands
* Updated spec.yaml
* auto-bump connector version
* format files
* add changelog
* update dockerfile
* auto-bump connector version
Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
* WIP Per-stream state support for Postgres source
* Fix failing test
* Improve code coverage
* Make global the default state manager
* Add legacy adapter state manager
* Formatting
* Include legacy state for backwards compatibility
* Add global state manager
* Implement Global/CDC state handling
* Fix test issues
* Fix issue with updated method signature
* Handle empty state case in global state manager
* Adjust to protocol changes
* Fix failing acceptance tests
* Fix failing test
* Fix unmodifiable list issue
* Fix unmodifiable exception
* PR feedback
* Abstract global state manager selection
* Handle conversion between different state types
* Handle invalid conversion
* Rename parameter
* Refactor state manager creation
* Fix failing tests
* Fix failing integration tests
* Add CDC test
* Fix failing integration test
* Revert change
* Fix failing integration test
* Use per-stream for postgres tests
* Formatting
* Correct stream descriptor validation
* Correct permalink
* PR feedback
* start
* revert
* azblob
* bq
* bq denorm
* megapublish baaaabyyyy
* fix needs
* matrix connectors
* auto-bump connector version
* dont failfast and max parallel 5
* multi runno
* minor
* testing matrix agents
* name
* testing multi agents
* tmp fix
* new multi agents
* multi test
* tryy
* let's do this
* magico
* fix
* label test
* couple more connector bumps
* temp
* things
* check this
* lets gooo
* more connectors
* Delete TEMP-testing-command.yml
* auto-bump connector version
* added comment describing bash part
* running single thread
* catch sentry cli
* auto-bump connector version
* destinations
* + snowflake
* saved
* auto-bump connector version
* auto-bump connector version
* java source bumps
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* remove twice-defined methods
* label things
* revert action
* using the new test action
* point at action
* wrong tag on action
* update pool label
* update to use new ec2-github-runner fork
* this needs to be more generic than publisher
* change publish to run on pool
* add comment about runner-pool usage
* updated publish command docs for multi & parallel connector runs
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* unbump failed publish versions
* missed dockerfiles
* remove failed docs
* mssql fix
* overhauled the git comment output
* bumping a test connector that should work
* slight order switcheroo
* output connectors properly in first message
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* #9322 source Marketo: process fail during creation of an export job
* #9322 source marketo: upd changelog
* #9322 source marketo: fix unit test
* #9322 source marketo: fix SATs
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Removed part_size from connectors that use StreamTransferManager
* fixed S3DestinationConfigTest
* fixed S3JsonlFormatConfigTest
* upadate changelog and bump version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* upadate changelog and bump version for Redshift and Snowflake destinations
* auto-bump connector version
* fix GCS staging test
* fix GCS staging test
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* start and end date for finacial stream should not be more than 180 days apart
* improve unit tests
* make changes to start date for finance stream
* update tests
* lint changes
* update version to 0.2.22 for source-amazon-seller-partner
* 🎉 New Source: Firebolt (#1)
* fix: Boolean type cast
* test: Improve format testing and doc
* refactor: Move some db functionality
* docs: Adding types doc link in utils
* feat: Use future-proof Auth in SDK
* fix: integration tests are failing
* chore: update seed file
Co-authored-by: Petro Tiurin <93913847+ptiurin@users.noreply.github.com>
Co-authored-by: ptiurin <petro.tiurin@firebolt.io>
* Add deployment mode to env shared with jobs
* Add adaptive runners
* Migrate postgres source to use adaptive runner
* Add an array of specs in docker image spec definition
* Add copyright
* Parse docker image spec with specs list
* Update spec yaml files
* Pass in DEPLOYMENT_MODE to docker compose file
* Revert "Parse docker image spec with specs list"
This reverts commit 8fe41dd3b7.
* Revert changes in docker image spec
* Read cloud specific spec files based on deployment mode
* Revert "Update spec yaml files"
This reverts commit 059f326432.
* Publish cloud spec file if necessary
* Fix upload script
* Move test files
* Update docker compose file
* Format code
* Add comment about spec filename
* Add unit tests
* Remove redundant jdbc acceptance test
When running `PostgresStrictEncryptJdbcSourceAcceptanceTest`, the `discover` method tests always fail because there are unexpected columns in the catalog:
- `wakeup_at`
- `last_visited_at`
- `last_comment_at`
These columns only exist in `PostgresJdbcSourceAcceptanceTest`. And this failure cannot be reproduced locally.
The hypothesis is that when the JDBC unit tests are run on CI, they are run in parallel, and the same testcontainer is used for both tests. That's why the strict encrypt test can discover columns from the oridinary unit test.
Given that the JDBC strict encrypt test is basically redundant, it is removed.
* Fix infinite loop when fetching Amplitude data
* Update changelog and version
* Address review
* fix: unit tests were failing
* chore: bump version in source definitions
* chore: update seed file
Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
* Postgres Source: fixed truncated precision if the value of the millisecond or second is 0
* check CI with 1.15.3 testcontainer
* check CI with 1.15.3 testcontainer
* returned latest version of testcontainer
* fixed checkstyle
* fixed checkstyle
* returned latest testcontainer version
* updated CHANGELOG
* bump version
* auto-bump connector version
* Postgres source strict encrypt bump version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Postgres Source: fixed truncated precision if the value of the millisecond or second is 0
* check CI with 1.15.3 testcontainer
* check CI with 1.15.3 testcontainer
* returned latest version of testcontainer
* fixed checkstyle
* fixed checkstyle
* returned latest testcontainer version
* updated CHANGELOG
* bump version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* S3 destination: Updating processing data types for Avro/Parquet formats
* S3 destination: handle comparing data types
* S3 destination: clean code
* S3 destination: clean code
* S3 destination: handle case with unexpected json schema type
* S3 destination: clean code
* S3 destination: Extract the same logic for Avro/Parquet formats to separate parent class
* S3 destination: clean code
* S3 destination: clean code
* GCS destination: Update data types processing for Avro/Parquet formats
* GCS destination: clean redundant code
* S3 destination: handle case with numbers inside array
* S3 destination: clean code
* S3 destination: add unit test
* S3 destination: update unit test cases with number types.
* S3 destination: update unit tests.
* S3 destination: bump version for s3 and gcs
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
