1
0
mirror of synced 2026-01-05 21:02:13 -05:00
Commit Graph

3601 Commits

Author SHA1 Message Date
Charles
6ff2fc7429 Expose version as AirbyteVersion instead of String in Configs iface (#7369) 2021-10-26 15:56:19 -07:00
Yevhenii
85851b7e0f 🐛 Hotfix Hubspot: add the length of separator symbol to check (#7393)
* hotfix for split_properties function
2021-10-26 23:27:04 +03:00
terencecho
1b1b52d6c9 🎉 New Source: Strava (#7151)
* Initial commit for new source: Strava

* Update readme and invalid_config

* Update PR number in docs

* Update docs/integrations/README

* Annotae spec with airbyte_secret

* Change starting_after input to start_date and add try/catch connection check

* Add connector to connector index

* Address PR commends and separate streams

* Set start_date as a required field

* Remove nexted objects from schema

* Remove schema TODO.md

* Fix schema

* Edit activites schema to allow for null values

* Fix formatting

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Fix formatting

* Update source_definitions.yaml

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2021-10-26 17:06:12 -03:00
Eugene
46a249e5c9 🎉Source Clickhouse: added option to connect via SSH tunnel (aka Bastion server) (#7327)
Source-Clickhouse: added support for connection via ssh tunnel
2021-10-26 21:39:18 +03:00
Abhi Vaidyanatha
1233e33444 Fix star link on GitHub. (#7396) 2021-10-26 11:19:16 -07:00
Abhi Vaidyanatha
498f325c9a Add star count to front page of docs. (#7395) 2021-10-26 11:17:48 -07:00
Artem Astapenko
ac754ce450 Enable fullstory only for cloud (#7298) 2021-10-26 21:10:20 +03:00
Artem Astapenko
6fc9418fcf Jamakase/fix password invalid text (#6939) 2021-10-26 21:09:57 +03:00
Jenny Brown
22f8c391c9 Clarified wording on Save Actions. (#7358) 2021-10-26 13:05:54 -05:00
Jenny Brown
c2adf9be21 Set a timeout on kube pod process integration tests to prevent a hang (#7087)
* Improved logging for troubleshooting.

* Limiting the number of tests for faster iteration testing.

* Re-enable tests

* Removed unnecessary logging from troubleshooting

* Formatting

* whitespace

* Eliminated duplicate timeout.

* Whitespace/formatting

* Show logs when a sync job fails during testing.

* Formatting

* Use alphanumeric random string, not bytes, to test paths
2021-10-26 13:05:08 -05:00
Marcos Marx
3327bf8107 Publish PR 7062: new source lemlist (#7392)
* add source lemlist

* update change log

* apply default formating

* add comment, delete unnecessary files

* remove unnecessary things

Co-authored-by: Igli <iglikoxha@outlook.com>
2021-10-26 14:58:03 -03:00
Igli
19f2541a03 🎉 New Source: Lemlist (#7062)
* add source lemlist

* update change log

* apply default formating

* add comment, delete unnecessary files

* remove unnecessary things

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2021-10-26 14:16:49 -03:00
Artem Astapenko
91b5abc80a Fix page style. Remove bottom beige rectangle (#7364) 2021-10-26 23:51:44 +07:00
Marcos Marx
9157b3d9b2 Publish PR 6937: new source amazon sqs (#7389)
* Initial commit, working source with static Creds

* Typo in example queue url

* Adds auto delete of messages after read

* Adds visibility timeout

* remove insecure comments from AWS IAM Key spec

* explicitly set supported sync modes

* explicit sync mode should be lower case

* Adds unit tests for check, discover, read

* remove incremental acceptance test block

* remove incremental from conf catalog sample

* remove test requirement moto from main req

* align int catalog sample with sample_files

* fixing catalog configs

* acceptance testing config

* adds expected records txt

* automated formatting changes

* remove expected records block from acpt test

* Adds Docs page

* Ammends formatting on readme

* Adds doc link to summary

* Improve error handling & debug logging

* Adds bootstrap.md

* Add a todo suggestion for batch output

* Adds SQS to integrations readme list

* lower case properties

* removed unused line

* uses enum for aws region

* updates sample configs to use lowercase

* required props to lower case

* add missed property to lowercase

* gradle formatting

* Fixing issues from acceptance tests

* annotate secrets in spec.json with airbyte_secret

* Adds explicit warnings about data less when using Delete Message option

* create config files

* remove eof line

* correct catalog with airbyte creds

* format

Co-authored-by: Alasdair Brown <me@alasdairb.com>
2021-10-26 13:39:51 -03:00
Artem Astapenko
002b62d42c Fix arrow in setup connection flow (#7363) 2021-10-26 19:26:37 +03:00
Artem Astapenko
ed4afd9316 Add tooltip (#7365) 2021-10-26 19:26:32 +03:00
Alasdair Brown
58b569da18 🎉 Source Amazon SQS: New connector (#6937)
* Initial commit, working source with static Creds

* Typo in example queue url

* Adds auto delete of messages after read

* Adds visibility timeout

* remove insecure comments from AWS IAM Key spec

* explicitly set supported sync modes

* explicit sync mode should be lower case

* Adds unit tests for check, discover, read

* remove incremental acceptance test block

* remove incremental from conf catalog sample

* remove test requirement moto from main req

* align int catalog sample with sample_files

* fixing catalog configs

* acceptance testing config

* adds expected records txt

* automated formatting changes

* remove expected records block from acpt test

* Adds Docs page

* Ammends formatting on readme

* Adds doc link to summary

* Improve error handling & debug logging

* Adds bootstrap.md

* Add a todo suggestion for batch output

* Adds SQS to integrations readme list

* lower case properties

* removed unused line

* uses enum for aws region

* updates sample configs to use lowercase

* required props to lower case

* add missed property to lowercase

* gradle formatting

* Fixing issues from acceptance tests

* annotate secrets in spec.json with airbyte_secret

* Adds explicit warnings about data less when using Delete Message option
2021-10-26 13:23:10 -03:00
Sherif A. Nada
e92fbd9ea2 prettify connector definition seeds (#7375) 2021-10-26 07:50:18 -07:00
Marcos Marx
2c8151fef2 Add creds for strava, lemlist, amazon sqs and freshservice (#7388)
* add creds for strava, lemlist, amazon sqs and freshservice

* correct source name in ci_creds
2021-10-26 11:42:36 -03:00
Yevhenii
928aa6280c 🐛 Source Hubspot: Fix issue with getting 414 HTTP error for streams (#6954)
* Source Hubspot: Fix issue with getting 414 HTTP error for streams

* update code and schemas

* bump version
2021-10-26 15:37:07 +03:00
John Lafleur
722f82d17a Creating license-faq (#7153)
* Creating license-faq

* Updating links in Licenses and more details on examples

* Create examples.md

* Update SUMMARY.md

* Update README.md

* Fix license FAQ asset.

* Update license-faq.md

* Update license-faq.md

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@gmail.com>
2021-10-26 22:25:02 +11:00
Charles
35e905041d prefer AirbyteVersion over String (#7310) 2021-10-25 18:06:04 -07:00
Benoit Moriceau
06c3fcbed2 Fix the check connection activity (#7366)
Fix acceptance tests
2021-10-25 17:37:33 -07:00
Charles
5c20ab3b01 remove py from fe e2e test build (#7361) 2021-10-25 17:12:59 -07:00
LiRen Tu
779c39c088 Copy job attempt state to configs database (#7219)
* Add migration to create latest state table

* Log migration name

* Expose db variables to airbyte-db

* Implement migration

* Fix migration test

* temp

* Rebase on master

* Save state in temporal (#7253)

* Copy state to airbyte_configs table

* Add standard sync state

* Move state methods to config repository

* Add unit tests

* Fix unit tests

* Register standard sync state in migration

* Add comment

* Use config model instead of json node

* Add comments

* Remove unnecessary method

* Fix migration query

* Remove unused config database

* Move persist statement and log the call

* Update dev doc

* Add unit tests for sync workflow

Co-authored-by: Charles <giardina.charles@gmail.com>
2021-10-25 17:08:08 -07:00
Lake Mossman
5d2b5dcfcb fetch specs from definitions directly (#7293)
* try fetching specs from definitions first

* refactor specFetcher and update tests

* run gradle format

* format again

* fix comment formatting

* fix test

* merge comment lines into single line

* move duplicate job metadata mocking logic to shared static method

* add todo

* formatting

* use local var and clone

* run gw format

* add todo

* skip spec fetcher in docker image validator and update todos

* run gw format
2021-10-25 16:33:15 -07:00
Artem Astapenko
6ead3342fd Fix codestyle for workspace (#7304) 2021-10-26 06:12:55 +07:00
Benoit Moriceau
98d87b116e refactor spec workflow (#7354) 2021-10-25 13:45:01 -07:00
Benoit Moriceau
c689e17e80 refactor the catalog discovery workflow (#7350)
* refactor the catalog discovery workflow

* Format
2021-10-25 13:29:10 -07:00
Benoit Moriceau
209a4aeb2d Split the check connection workflow into multiple files. (#7292)
* Split the check connection workflow into multiple files.

* Format
2021-10-25 13:14:06 -07:00
Marcos Marx
5a52d20979 Source Recharge: change schema for discount and order (#7295) 2021-10-25 14:10:55 -03:00
irynakruk
dbfd6cae88 Source MongoDB v2: Fixed nested document parsing (#7160)
* Fixed nested document parsing: added parsing of nested documents and arrays, added data types test
2021-10-25 09:16:34 -04:00
Christophe Duong
27df558a89 🎉 BigQuery destinations with partitionned/clustered keys (#7240)
* [ #5959 ][ #2579 ] Add support of partitioned tables by _airbyte_emitted_at field (#7141)

Co-authored-by: Andrés Bravo <andresbravog@gmail.com>
2021-10-25 12:41:04 +02:00
Eugene
9a0b8dac6c 🎉Source Clickhouse: added ssl support and "strict-encrypt" connector (#7127)
* [Ticket 6435] Source-ClickHouse: added ssl support

* [Ticket 6716] Source-ClickHouse: added secure only connector
2021-10-25 13:33:25 +03:00
Jared Rhizor
7fcbc5c027 Bump Airbyte version from 0.30.22-alpha to 0.30.23-alpha (#7320)
Co-authored-by: sherifnada <sherifnada@users.noreply.github.com>
v0.30.23-alpha
2021-10-24 23:28:26 -07:00
Sherif A. Nada
15afaf0533 revert incorrectly enabled oauth feature (#7319) 2021-10-24 23:09:11 -07:00
Charles
6deae28dac remove core label, replace with platform (#7305) 2021-10-23 15:49:09 -07:00
Charles
84b69d953e remove python from platform build (#7302) 2021-10-23 15:36:34 -07:00
Benoit Moriceau
e95a8050c8 Bmoric/fix test bad catch (#7290)
Fix a test which was not testing what it was claiming to test.
2021-10-22 19:11:19 -07:00
Benoit Moriceau
a616c29d84 [5889] Bmoric/mdc manipulation tools [1/2] (#7266)
This is preparing the tagging of the application that generate a log which is requested in #5889

It is adding helper to add a color to the log and safely manipulate the MDC.

It is also modifying the log4j pattern.

The use of those helper will will be implemented in another PR.
2021-10-22 15:07:36 -07:00
Lake Mossman
077b90e45f add optional spec field to source and destination definitions (#7254) 2021-10-22 14:49:12 -07:00
irynakruk
1ae5cf42c3 🐛 Source Snowflake: Fixed parsing of extreme values for FLOAT and NUMBER data types (#7257)
* Fixed parsing of extreme values for FLOAT and NUMBER data types
2021-10-22 15:48:55 -04:00
Brian Bugh
a9e01db8b3 Fix incorrect code for Snowflake GCS (#7287)
This was confusing, as `io_airbyte_test_staging` looks like a literal string and is not mentioned in the explanation above. Slack support context: https://app.slack.com/client/T01AB4DDR2N/C01MFR03D5W/thread/C01MFR03D5W-1634852461.058400
2021-10-22 16:22:29 -03:00
Yevhenii
5eb2af6c08 CDK: Autogenerate reference docs (#4759)
* CDK Autogenerated reference docs: base version

* Update docs config.
Add .readthedocs.yaml file.
Update build html files.

* Update .gitignore.
Remove sphinx build files.

* Add newline at the end of .gitignore

* Update setup.py requirements.
Update .readthedocs.yaml config.

* Update rst files.
Add Makefile rst config.

* Update CDK docstring format.
Change rst layout.
Update sphinx config.

* Add Sphinx docs.
Update index.rst.
Update abstract_source.py docstrings.

* Override master_doc and package templates.
Add docs schema enerator script.
Update sphinx docs.

* Add `Publishing to Read the Docs` section to sphinx-docs.md".
Replace sphinx-docs.md to `airbyte-cdk` module.

* Update sphinx-docs.md section name

* Bump airbyte-cdk version.
Update CHANGELOG.md.

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
Co-authored-by: Vadym Hevlich <vege1wgw@gmail.com>
2021-10-22 20:47:48 +03:00
vitaliizazmic
0c7f06018b 🐛 Airbyte CDK: transforming Python log levels to Airbyte protocol log levels
* Airbyte CDK native logger #1279 - transforming Python log levels to Airbyte protocol log levels

* Airbyte CDK native logger #1279 - add test for level types transforming

* CDK standard logger #1279 - test for transform critical level to fatal
2021-10-22 17:14:01 +03:00
Baz
9a435ec15b 🎉 Source Zuora: support of Unlimited option for Data Query (#7093) 2021-10-22 17:01:00 +03:00
Arthur Galuza
cb6408af8c 🎉 Source Facebook Pages: bump FB API version (#7128)
* Upd FB API version

* Upd changelog

* Exclude fields with extra permissions required
2021-10-22 14:35:50 +03:00
VitaliiMaltsev
0c932749d4 🎉 Redshift Source and Destination set SSL as default option (#7234)
* Redshift Source and Restination set SSL as default option

* add changelog

* remove SSL test| add more documentation

* bump new version

* bump new version

Co-authored-by: vmaltsev <vitalii.maltsev@globallogic.com>
2021-10-22 12:28:54 +03:00
Marcos Marx
ad6223d817 Publish PR 6962: New Source Salesloft (#7274)
* Generated connector source code

* Added basic users stream

* Added people stream

* Added cadences and cadence_memberships streams

* Updated unit tests

* Fixed docker build

* Updated connector to use OAuth

* address some checklist items

* review comments

* small fix

* add salesloft into airbyte-config

* correct docs

Co-authored-by: lgomezm <luis@calixa.io>
Co-authored-by: prasrvenkat <pras@calixa.io>
2021-10-22 02:31:37 -03:00
Prasanna Ram Venkatachalam
115af277e2 🎉 New Source: Salesloft (#6962)
* Generated connector source code

* Added basic users stream

* Added people stream

* Added cadences and cadence_memberships streams

* Updated unit tests

* Fixed docker build

* Updated connector to use OAuth

* address some checklist items

* review comments

Co-authored-by: lgomezm <luis@calixa.io>
2021-10-22 02:00:16 -03:00