* Mixpanel initiation
* copied schemas and specs file from singer connector
* authentication and a few streams
* Added Funnels + FunnelsList
* Added example of funnel response
* added incremental Funnels stream with tests
* added Annotations, CohortMembers, Engage, Cohorts, Funnels
* added Revenue
* fixed formatting
* fixed variable names
* fixed cohort_members and updated export streams
* moved start_date and date checks into SourceMixpanel class
* added error handling
* added unit test, update docs and ci creds
* fix url base for export stream
* added full and incremental read for export stream
* updated acceptance tests, added limit correction based on number of streams, export cursor is stored in datatime string
* Funnel stream - added complex state which contains state for each funnel
* added attribution windows support and project timezone config
* fixed formatting
* added default timezone
* added dynamic schema generation for Engage and Export streams
* fixed formatting
* fixed ability to pass start_date in datetime format as well
* fixed ability to pass start_date in datetime format as well
* added additional_properties field for dynamic schemas. updates regex for start_date matching to support old config file
* fixed formatting
* export stream - convert all values to default type - string
* added schema ref
* added new properties for funnel stream
* fixed formatting in funnel schema
* added build related files
* update changelog
* fixed and added comments, renamed rate_limit variable
* fixed formatting
* changed normalization for reserved mixpanel attributes like $browser
* alphabetise spec fields
* added description about API limit handling
* updated comment
* Implement db config persistence
* Fix database readiness check
* Reduce logging noise
* Setup config database in config persistence factory
* Update documentation
* Load seed from yaml files
* Refactor config persistence factory
* Add one more test to mimic migration
* Remove unnecessary changes
* Run code formatter
* Update placeholder env values
* Set default config database parameters in docker compose
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Default setupDatabase to false
* Rename variable
* Set default config db parameters for server
* Remove config db parameters from the env file
* Remove unnecessary environment statements
* Hide config persistence factory (#4772)
* Remove CONFIG_DATABASE_HOST
* Use builder in the test
* Simplify config persistence builder
* Clarify config db connection readiness
* Format code
* Add logging
* Fix typo
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Add a config_id only index
* Reuse record insertion code
* Add id field name to config schema
* Support data loading from legacy config schemas
* Log missing logs in migration test
* Move airbyte configs table to separate directory
* Update exception message
* Dump specific tables from the job database
* Remove postgres specific uuid extension
* Comment out future branch
* Default configs db variables to empty
When defaulting them to the jobs db variables, it somehow does not work.
* Log inserted config records
* Log all db write operations
* Add back config db variables in env file to mute warnings
* Log connection exception to debug flaky e2e test
* Leave config db variables empty
`.env` file does not support variable expansion.
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
Co-authored-by: Charles <giardina.charles@gmail.com>
* Adding Google Cloud Storage as destination
* Removed few comments and amended the version
* Added documentation in docs/integrations/destinations/gcs.md
* Amended gcs.md with the right pull id
* Implemented all the fixes requested by tuliren as per https://github.com/airbytehq/airbyte/pull/4329
* Renaming all the files
* Branch alligned to S3 0.1.7 (with Avro and Jsonl). Removed redundant file by making S3 a dependency for GCS
* Removed some additional duplicates between GCS and S3
* Revert changes in the root files
* Revert jdbc files
* Fix package names
* Refactor gcs config
* Format code
* Fix gcs connection
* Format code
* Add acceptance tests
* Fix parquet acceptance test
* Add ci credentials
* Register the connector and update documentations
* Fix typo
* Format code
* Add unit test
* Add comments
* Update readme
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: Marco Fontana <marco.fontana@sohohouse.com>
Co-authored-by: marcofontana.ing@gmail.com <marcofontana.ing@gmail.com>
Co-authored-by: Marco Fontana <MaxwellJK@users.noreply.github.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Add note about removing orphaned Airbyte configs
* Remove excess baggage
* Add a resetting section to make this more clear.
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
#4669 Stripe Source: Fix subscriptions stream to return all kinds of subscriptions (including expired and canceled)
Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
* this should fix the check if no records in annotations stream
* update schemas for new SAT requirements && apply user hint upgrade on wrong api key
* save schema upd
* upd insights schema
* upd insights schema2
* upd insights schema3
* upd insights schema4
* upd insights schema5 (null is joking)
* upd insights schema6 (null is joking)
* upd insights schema7
* upd insights schema8
* upd insights schema8
* bump version && docs
* Few updates for GitHub source
Set correct `cursor_field` for `IssueEvents` stream.
Add rate limit handling.
Add handling for 403 error.
Add handling for 502 error.
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* first few classes for mssql cdc
* wip
* mssql cdc working against unit tests
* increment version
* add cdc acceptance test
* tweaks
* add file
* working on comprehensive tests
* change isolation from snapshot to read_committed_snapshot
* finalised type tests
* Revert "change isolation from snapshot to read_committed_snapshot"
This reverts commit 20c6768071.
* small docstring fix
* remove unused imports
* stress test fixes
* minor formatting improvements
* mssql cdc docs
* finish off cdc docs
* format fix
* update connector version
* add to changelog
* fix for sql server agent offline failing cdc enable on tables
* final structure
* few more updates
* undo unwanted changes
* add abstract test + more refinement
* remove CDC metadata to debezium
* use new cdc abstraction for mysql
* undo wanted change
* use cdc abstraction for postgres
* add files
* pull in latest changes
* ready
* rename class + add missing property
* use renamed class + move constants to MySqlSource
* use renamed class + move constants to PostgresSource
* move debezium to bases + upgrade debezium version + review comments
* downgrade version + minor fixes
* bring in latest changes from cdc abstraction
* reset to minutes
* bring in the latest changes
* format
* fix build
* address review comments
* bring in latest changes
* bring in latest changes
* use common abstraction for CDC via debezium for sql server
* remove debezium from build
* finalise PR
* should return Optional
* pull in latest changes
* pull in latest changes
* address review comments
* use common abstraction for CDC via debezium for mysql (#4604)
* use new cdc abstraction for mysql
* undo wanted change
* pull in latest changes
* use renamed class + move constants to MySqlSource
* bring in latest changes from cdc abstraction
* format
* bring in latest changes
* pull in latest changes
* use common abstraction for CDC via debezium for postgres (#4607)
* use cdc abstraction for postgres
* add files
* ready
* use renamed class + move constants to PostgresSource
* bring in the latest changes
* bring in latest changes
* pull in latest changes
* lower version for tests to run on CI
* format
* Update docs/integrations/sources/mssql.md
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* addressing review comments
* fix for testGetTargetPosition
* format changes
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
