* Added webflow code
* Updated readme
* Updated README
* Added webflow to source_definitions.yaml
* Enhanced documentation for the Webflow source connector
* Improved webflow source connector instructions
* Moved Site ID to before API token in Spec.yaml (for presentation in the UI)
* Addressed comments in PR.
* Changes to address requests in PR review
* Removed version from config
* Minor udpate to spec.yaml for clarity
* Updated to pass the accept-version as a constant rather than parameter
* Updated check_connection to hit the collections API that requires both site id and the authentication token.
* Fixed the test_check_connection to use the new check_connection function
* Added a streams test for generate_streams
* Re-named "autentication" object to "auth" to be more consistent with the way it is created by the CDK
* Added in an explict line to instantiante an "auth" object from WebflowTokenAuthenticator, to make it easier to describe in the blog
* Fixed a typo in a comment
* Renamed some classes to be more intuitive
* Renamed class to be more intuitive
* Minor change to an internal method name
* Made _get_collection_name_to_id_dict staticmethod
* Fixed a unit-test error that only appeared when running " python -m pytest -s unit_tests".
This was caused by Mocked settings from test_source.py leaking into test_streams.py
* format: add double quotes and remove unused import
* readme: remove semantic version naming of connector in build commands
* Updated spec.yaml
* auto-bump connector version
* format files
* add changelog
* update dockerfile
* auto-bump connector version
Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
* switch to temporal cloud client for now
* format
* use client cert/key env secret instead of path to secret
* add TODO comments
* format
* add logging to debug timeout issue
* add more logging
* change workflow task timeout
* PR feedback: consolidate as much as possible, add missing javadoc
* fix acceptance test, needs to specify localhost
* add internal-use only comments
* format
* refactor to clean up TemporalClient and prepare it for future dependency injection framework
* remove extraneous log statements
* PR feedback
* fix test
* return isInitialized true in test
* start
* revert
* azblob
* bq
* bq denorm
* megapublish baaaabyyyy
* fix needs
* matrix connectors
* auto-bump connector version
* dont failfast and max parallel 5
* multi runno
* minor
* testing matrix agents
* name
* testing multi agents
* tmp fix
* new multi agents
* multi test
* tryy
* let's do this
* magico
* fix
* label test
* couple more connector bumps
* temp
* things
* check this
* lets gooo
* more connectors
* Delete TEMP-testing-command.yml
* auto-bump connector version
* added comment describing bash part
* running single thread
* catch sentry cli
* auto-bump connector version
* destinations
* + snowflake
* saved
* auto-bump connector version
* auto-bump connector version
* java source bumps
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* remove twice-defined methods
* label things
* revert action
* using the new test action
* point at action
* wrong tag on action
* update pool label
* update to use new ec2-github-runner fork
* this needs to be more generic than publisher
* change publish to run on pool
* add comment about runner-pool usage
* updated publish command docs for multi & parallel connector runs
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* unbump failed publish versions
* missed dockerfiles
* remove failed docs
* mssql fix
* overhauled the git comment output
* bumping a test connector that should work
* slight order switcheroo
* output connectors properly in first message
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* #9322 source Marketo: process fail during creation of an export job
* #9322 source marketo: upd changelog
* #9322 source marketo: fix unit test
* #9322 source marketo: fix SATs
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Removed part_size from connectors that use StreamTransferManager
* fixed S3DestinationConfigTest
* fixed S3JsonlFormatConfigTest
* upadate changelog and bump version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* auto-bump connector version
* upadate changelog and bump version for Redshift and Snowflake destinations
* auto-bump connector version
* fix GCS staging test
* fix GCS staging test
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* start and end date for finacial stream should not be more than 180 days apart
* improve unit tests
* make changes to start date for finance stream
* update tests
* lint changes
* update version to 0.2.22 for source-amazon-seller-partner
* Common code to deserialize a state message in the new format
* PR comments and type changed to typed
* Format
* Add StateType and StateWrapper objects to the model
* Use state wrapper instead of Either
* Switch to optional
* PR comments
* Support array legacy state
* format
Co-authored-by: Jimmy Ma <jimmy@airbyte.io>
* 🎉 New Source: Firebolt (#1)
* fix: Boolean type cast
* test: Improve format testing and doc
* refactor: Move some db functionality
* docs: Adding types doc link in utils
* feat: Use future-proof Auth in SDK
* fix: integration tests are failing
* chore: update seed file
Co-authored-by: Petro Tiurin <93913847+ptiurin@users.noreply.github.com>
Co-authored-by: ptiurin <petro.tiurin@firebolt.io>
* add current connection state to JobResetConnectionConfig
* pass state from reset connection config to job sync config in generate input activity
* format
* use SHORT_ACTIVITY_OPTIONS on check connection activity so that it has retries
* retry workflow after delay instead of quarantining
* allow activity env vars to be configured in docker and kube
* add env var for workflow restart delay and refactor slightly
* update tests to handle new restart behavior
* update test name
* add empty env var values to .env files
* fail attempt before job in cleanJobState to prevent state machine failure
* change default value of max activity attempt retries from 10 to 5
* Add deployment mode to env shared with jobs
* Add adaptive runners
* Migrate postgres source to use adaptive runner
* Add an array of specs in docker image spec definition
* Add copyright
* Parse docker image spec with specs list
* Update spec yaml files
* Pass in DEPLOYMENT_MODE to docker compose file
* Revert "Parse docker image spec with specs list"
This reverts commit 8fe41dd3b7.
* Revert changes in docker image spec
* Read cloud specific spec files based on deployment mode
* Revert "Update spec yaml files"
This reverts commit 059f326432.
* Publish cloud spec file if necessary
* Fix upload script
* Move test files
* Update docker compose file
* Format code
* Add comment about spec filename
* Add unit tests
* Remove redundant jdbc acceptance test
When running `PostgresStrictEncryptJdbcSourceAcceptanceTest`, the `discover` method tests always fail because there are unexpected columns in the catalog:
- `wakeup_at`
- `last_visited_at`
- `last_comment_at`
These columns only exist in `PostgresJdbcSourceAcceptanceTest`. And this failure cannot be reproduced locally.
The hypothesis is that when the JDBC unit tests are run on CI, they are run in parallel, and the same testcontainer is used for both tests. That's why the strict encrypt test can discover columns from the oridinary unit test.
Given that the JDBC strict encrypt test is basically redundant, it is removed.
* Fix infinite loop when fetching Amplitude data
* Update changelog and version
* Address review
* fix: unit tests were failing
* chore: bump version in source definitions
* chore: update seed file
Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
* Postgres Source: fixed truncated precision if the value of the millisecond or second is 0
* check CI with 1.15.3 testcontainer
* check CI with 1.15.3 testcontainer
* returned latest version of testcontainer
* fixed checkstyle
* fixed checkstyle
* returned latest testcontainer version
* updated CHANGELOG
* bump version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* S3 destination: Updating processing data types for Avro/Parquet formats
* S3 destination: handle comparing data types
* S3 destination: clean code
* S3 destination: clean code
* S3 destination: handle case with unexpected json schema type
* S3 destination: clean code
* S3 destination: Extract the same logic for Avro/Parquet formats to separate parent class
* S3 destination: clean code
* S3 destination: clean code
* GCS destination: Update data types processing for Avro/Parquet formats
* GCS destination: clean redundant code
* S3 destination: handle case with numbers inside array
* S3 destination: clean code
* S3 destination: add unit test
* S3 destination: update unit test cases with number types.
* S3 destination: update unit tests.
* S3 destination: bump version for s3 and gcs
* auto-bump connector version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Fall back to parsing w/ or w/o TZ if parsing a date or a time string fails
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* add ResetSourceConfiguration to JobResetConnectionConfig, and retrieve in GenerateInputActivity
* add newline
* add comment explaining null check
* format
* more formatting
* Bump `source-google-ads` to build for both AMD and ARM
* pin protobuf==3.14
* update readme
* #263 oncall: bump google ads version 15.1.1, protobuf 3.20.0
* auto-bump connector version
Co-authored-by: Denys Davydov <davydov.den18@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
