* add kinesis destination connector
* format code
* add javadoc and general documentation
* fix buffer bug and refactor code
* rename vars to common airbyte usage
* run format
* run format + seed file
* add eof
Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
* add source notion
* update PR number in change log
* bug fix and code improvement as code review suggestions
* code improvement as review advices
* new connector notion
* format
* correct creds file
* run seed
* bump connector version
* format
Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
* Add Pinterest implementation
* Updated to review
* Added rate limits and PR number to doc
* Fixed unit tests and data slicing bug
* Update airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Update airbyte-integrations/connectors/source-pinterest/build.gradle
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Update airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Updated to review
* Updated window in days
* Fixed unit test
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Adding Pulsar destination
* Add enable_chunking property
* Add dependency for DNS resolve for Mac
* Updating build.gradle
* Adding definition
* Refactor and fix tests
* Adding new config properties
* Reformat code
* Close resources when checking connection
* Updating doc
* Adding more test to build the producer map
* Rename brokers property
* Avoid emitting null states to the output collector
* fix config file
* run seed source/destination
Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
* feat: adds destination-elasticsearch
* feat: adds destination-elasticsearch es server container
* refactor: header configuration
* update: only call createIndex when preparing the writes
* update: reuse container
* fix: make index names valid and use namespace
* refactor: use bulk process and buffered consumer
* refactor: fix bulk process and buffered consumer
* chore: update documentation
* update: remove ssl reference
* fix: bulk indexing
adds test logging config to inspect http wire
begins work for overrwriting existing records
* docs: update for authentication
* refactor: simplify config
* refactor: cleanup indices, implement auth
* update: cleanup equals/toString in Elasticsearch ConnectionConfiguration
* chore: use conventions and remove unused code
* update: close underlying rest connection
* update: enable `supportsNormalization`
* refactor: better encapsulate index naming
* update: allow upserting
* update: use oneOf for auth method
* refactor: use encapsulated auth object
* chore: pretty
* update: simplify auth header creation
* chore: remove unused class
* update: use boolean as field type
* adds: elasticsearch example server
* fix: api secret test
* feat(67): add support for 'spec' using Python HTTP API source template and stripe as an example
* chore(67): add sample state and config
* feat(67): add check functionality for paystack source by fetching first customer
* feat(67): add support for discover and read customer stream
* feat(67): add paystack source connector to UI
* feat(67): update source definitions to use 0.1.0
* Hacktoberfest 67 paystack source (#1)
* feat(67): add support for 'spec' using Python HTTP API source template and stripe as an example
* chore(67): add sample state and config
* feat(67): add check functionality for paystack source by fetching first customer
* feat(67): add support for discover and read customer stream
* feat(67): add paystack source connector to UI
* feat(67): update source definitions to use 0.1.0
Co-authored-by: Foluso Ogunlana <foluso_ogunlana@stearsng.com>
* feat(67): update stream state cursor field to be integer and to match API record field name
* chore(67): add unit tests for source and streams
* chore(67): store formatted date time in state to match type of catalog
* chore(67): add configuration for acceptance integration tests
* docs(67): update docs and summary with paystack
* chore(67): add essential schemas to be catalogued for new streams
* feat(67): add support for critical streams - transactions subscriptions transfers refunds settlements
* docs(67): update image and bootstrap
* chore(67): update builds.md to include paystack badge
* docs(67): add changelog and source definition JSON file
* docs(67): add paystack to integrations readme
* chore(67): update check_connection to airbyte standard
* refactor to simplify streams and remove constants file
* fix(67): correct "null, null" values in schemas
* chore(67): update file formatting with gradle format
Co-authored-by: Foluso <5675998+foogunlana@users.noreply.github.com>
* Initial commit, working source with static Creds
* Typo in example queue url
* Adds auto delete of messages after read
* Adds visibility timeout
* remove insecure comments from AWS IAM Key spec
* explicitly set supported sync modes
* explicit sync mode should be lower case
* Adds unit tests for check, discover, read
* remove incremental acceptance test block
* remove incremental from conf catalog sample
* remove test requirement moto from main req
* align int catalog sample with sample_files
* fixing catalog configs
* acceptance testing config
* adds expected records txt
* automated formatting changes
* remove expected records block from acpt test
* Adds Docs page
* Ammends formatting on readme
* Adds doc link to summary
* Improve error handling & debug logging
* Adds bootstrap.md
* Add a todo suggestion for batch output
* Adds SQS to integrations readme list
* lower case properties
* removed unused line
* uses enum for aws region
* updates sample configs to use lowercase
* required props to lower case
* add missed property to lowercase
* gradle formatting
* Fixing issues from acceptance tests
* annotate secrets in spec.json with airbyte_secret
* Adds explicit warnings about data less when using Delete Message option
* add source onesignal
* update PR number in change log
* change source define cursor and sync mode
* get correct max cursor time across stream slices
* code improvement as code review advices
* format code
* remove unused code
* remove TODOs
Co-authored-by: Maksym Pavlenok <antixar@gmail.com>
* create a new connector folder
* add base classes
* add schemas
* add ads/ad_groups streams
* update tests and docs
* add a bootstrap file
* update the base domain name for API
* update the base domain name for API
* update Dockerfile libs
* remove unused comments
* Update airbyte-integrations/connectors/source-tiktok-marketing/bootstrap.md
Co-authored-by: George Claireaux <george@claireaux.co.uk>
* Update airbyte-integrations/connectors/source-tiktok-marketing/bootstrap.md
Co-authored-by: George Claireaux <george@claireaux.co.uk>
* fix UI error with input parameters
* fix problem with updated state
* Update airbyte-integrations/connectors/source-tiktok-marketing/source_tiktok_marketing/spec.py
Co-authored-by: Davin Chia <davinchia@gmail.com>
* add an unit test
* update README.md
* bump version
Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Davin Chia <davinchia@gmail.com>
Implement new destination connector for databricks delta lake.
Resolves#2075.
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* New abstraction for NoSql database sources
* New MongoDbSource: partial impl
* Added MongoDataType
* Improved MongoDatabase and fixed read method
* code review changes;
* merge clean up;
* Renamed NoSqlDatabase to AbstractDatabase
* formatter changes;
* code review changes: changed mongodb-new to mongodb-v2; left only new connector info in all docs
* code review changes: changed mongodb-new to mongodb-v2; left only new connector info in all docs
* updated spec.json and toDatabaseConfig() method
* updated doc accordingly to spec.json changes
Co-authored-by: Iryna Kruk <iryna.o.kruk@globallogic.com>
* new mongo db destination
* fix remarks
* updated documentation and added loggers
* updated documentation
* added hashCode field to mongodb document and fix minor remarks
* fix code style
* updated mongodb data hash from integer to UUID string
* Add Google Analytics v4 implementation
* Add docs and connector index
* Fix a broken link to Airbyte CDK
* Fix a broken link to source acceptance tests docs
* Add condition for a full refresh or incremental stream
* Add unit tests
* Fix formatting to flake8
* Updated to review
* Added logger for custom reports validation
* Add comments to code
* Updated to format
