Sherif A. Nada
743e6c2d6c
[Low-code CDK] Add ability to remove fields ( #14402 )
2022-07-11 19:04:57 -07:00
Brian Lai
374e265fcb
[Low Code CDK] configurable oauth request payload ( #13993 )
...
* configurable oauth request payload
* support interpolation for dictionaries that are not new subcomponents
* rewrite a declarative oauth authenticator that performs interpolation at runtime
* formatting
* whatever i don't know why factory gets flagged w/ the newline change
* we java now
* remove duplicate oauth
* add some comments
* parse time properly from string interpolation
* move declarative oauth to its own package in declarative module
* add changelog info
2022-07-08 16:49:16 -04:00
Alexandre Girard
01fc7a48c7
move jinja macros to their own file ( #14515 )
...
* move jinja macros to their own file
* format
* export as dict
* format
2022-07-08 07:56:06 -07:00
Alexandre Girard
1cf89c63c3
add reference docs for declarative source ( #14501 )
...
* generate pdoc
* set python version to 3.8
* mimic doc
* set the os
* Revert "generate pdoc"
This reverts commit 82bdeabe8f
2022-07-08 07:26:27 -07:00
Brian Lai
35ca7c8d10
[low code cdk] Add MinMaxDate component and allow for it to be specified for DatetimeStreamSlicer ( #14288 )
...
* implement MinMaxDate and incorporate it into the DatetimeStreamSlicer
* rename to datetime and restrict start/end to minmaxdatetime object
* get rid of eval_date which isn't needed anymore and allow for datetimestreamslicer to set datetime format
* add MinMaxDatetime to class types registry
2022-07-07 01:05:13 -04:00
Alexandre Girard
6cced4c7a7
fix type hints ( #14352 )
2022-07-05 20:04:25 -07:00
Alexandre Girard
37acb9289f
[low-code cdk] Enable configurable state checkpointing ( #14317 )
...
* checkout files from test branch
* read_incremental works
* reset to master
* remove dead code
* comment
* fix
* Add test
* comments
* utc
* format
* small fix
* Add test with rfc3339
* remove unused param
* fix test
* configurable state checkpointing
* update test
2022-07-05 19:20:23 -07:00
jordan-glitch
4395fbb4d2
cdk: Attach namespace to stream in catalog ( #13923 )
2022-07-04 18:04:31 +02:00
Alexandre Girard
055763ec73
fix format ( #14354 )
2022-07-01 09:54:48 -07:00
Albin Skott
be2d1a8652
🐛 SingerSource: Fix incompatibilities and typing issues ( #14148 )
...
* Use logging.Logger in SingerSource
* Fix SingerSource ConfigContainer
This fixes typing issues with `ConfigContainer` and makes it compatible
with `split_config`. Fixes #8710 .
* Fix SingerSource state and catalog typer issues
* Rename SingerSource method args to match parent classes
* Remove old comment about excluding Singer
Co-authored-by: Alexandre Girard <alexandre@airbyte.io >
2022-07-01 09:03:18 -07:00
Alexandre Girard
e23789bbd6
Alex/declarative stream incremental fix ( #14268 )
...
* checkout files from test branch
* read_incremental works
* reset to master
* remove dead code
* comment
* fix
* Add test
* comments
* utc
* format
* small fix
* Add test with rfc3339
* remove unused param
* fix test
2022-07-01 08:39:19 -07:00
Alexandre Girard
85449c975c
[low-code connectors] default types and default values ( #14004 )
...
* default types and default values
* cleanup
* fixes so read works
* remove prints and trycatch
* comment
* remove unused param
* split file
* extract method
* extract methods
* comment
* optional
* fix test
* cleanup
* delete interpolated request header provider
* simplify next page url paginator interface
* comment
* format
2022-06-27 20:43:03 -07:00
Sherif A. Nada
7acbcfbfb7
Fix CDK obfuscation of nested secrets ( #14035 )
2022-06-23 09:23:27 -07:00
George Claireaux
4d279f8238
Remove legacy sentry code from cdk ( #14016 )
...
* rip sentry out of cdk
* remove sentry dsn from gsc
2022-06-23 12:14:09 +01:00
Brian Lai
a61224887e
support semi incremental by adding extractor record filter ( #13520 )
...
* support semi incremental by adding extractor record filter
* refactor extractor into a record_selector that supports extraction and filtering of response records
2022-06-23 00:09:44 -04:00
Lake Mossman
c6d83b3239
Fix per stream state protocol backward compatibility ( #14032 )
...
* rename state type field to fix backwards compatibility issue
* replace usages of stateType with type
2022-06-22 17:42:02 -07:00
Alexandre Girard
94abef3acc
[low-code connectors] BasicHttpAuthenticator ( #13733 )
...
* implement basichttpauthenticator
* add optional refresh access token authenticator
* remove prints
* type hints
* Fix and unit test
* missing test
* Add class to __init__ file
* Add comment
2022-06-22 07:35:32 -07:00
Brian Lai
be01b476ce
Add new InterpolatedRequestOptionsProvider that encapsulates all variations of request arguments ( #13472 )
...
* write out new request options provider and refactor components and parts of the YAML config
* fix formatting
* pr feedback to consolidate body_data_provider to simplify the code
* pr feedback get rid of extraneous optional
2022-06-21 16:01:05 -04:00
Alexandre Girard
34e2dc36c6
[low-code connectors] Cartesian product stream slicer ( #13740 )
...
* list slicer
* Add comment
* product stream slicer
* comment
* rename
* format
* Update comment
* split on 2 lines for readability
* Revert "rename"
This reverts commit e801f2d1f8
2022-06-16 08:48:32 -07:00
Alexandre Girard
2c8fef8b7d
[low-code connectors] Support for list-based stream slicers ( #13668 )
...
* list slicer
* Add comment
* test
* ast eval
* refactor
* fix
* remove print
* rename
* return a generator
* smaller fix and reset to master
* union type hint
* assert
* comment
* format
2022-06-14 17:12:53 -07:00
Charles
22b727c0ea
Update Airbyte Protocol Docs ( #13709 )
2022-06-14 15:04:38 -07:00
Alexandre Girard
15fe51b788
[low-code connectors] add a few unit tests ( #13666 )
...
* add a couple of unit tests
* refactor tests
2022-06-13 16:51:36 -07:00
Benoit Moriceau
704dd8b534
Update schema ( #13573 )
...
* Update schema
* generate python
* Stream as an object
* PR comments
* generate python
* rm unused required
* Description the state with no type
* Fix connector build
* Format
* format
Co-authored-by: cgardens <charles@airbyte.io >
2022-06-10 16:35:59 -07:00
Pedro S. Lopez
4f8f979045
Fix Connectors Base Build: CDK tests fail after dependency update ( #13652 )
2022-06-09 16:05:36 -04:00
Alexandre Girard
57dd7a9d4b
[low-code connectors] use jello instead of jq ( #13570 )
...
* use jello instead of jq
* reset
* rename
* remove print
2022-06-07 14:10:26 -07:00
Benoit Moriceau
5a7b1aaacd
Bmoric/add namespace to protocol ( #13356 )
...
* Add namespace
* Matching python update
* Pr comments
* regenerate and format python cdk
* format
Co-authored-by: Pedro Lopez <pedroslopez@me.com >
2022-06-06 15:41:56 -07:00
Davin Chia
dd6d263dd4
Update generate script to use new protocol models directory. ( #13479 )
...
Missed this change from https://github.com/airbytehq/airbyte/pull/13476 .
2022-06-06 01:31:38 +08:00
Alexandre Girard
1225f80bd8
Source hubspot: filter out streams we don't have access to ( #13455 )
...
* filter out streams we don't have access to
* cleanup
* fix associations
* fix for api key auth
* this doesn't need to be a property method
* Bump
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com >
2022-06-03 12:06:51 -07:00
Alexandre Girard
7fbe6dab5c
[Low code CDK] add support for iterating over parent stream slices ( #13287 )
...
* checkout from alex/cac
* checkout from alex/cac
* checkout from alex/cac
* Add missing tests
* Add missing files
* Add missing tests
* add missing file
* missing file
* missing file
* rename
* doc
* doc
* remove broken test
* rename
* jinja dependency
* Add comment
* comment
* comment
* pyjq dependency
* rename file
* delete unused file
* Revert "delete unused file"
This reverts commit 758e9393674a79223944758e9393674a79223944#12852 )
* checkout from alex/cac
* Add missing tests
* Add missing files
* missing file
* rename
* jinja dependency
* Add comment
* comment
* comment
* Revert "delete unused file"
This reverts commit 758e9393674a79223944c8fe154ffc
2022-06-02 08:53:04 -07:00
Brian Lai
ccbd7c6508
conditional paginator + interpolated boolean components ( #13389 )
...
* conditional paginator + interpolated boolean components
* fix bug where empty arrays and dicts evaluated to true
* add more falsey values
* adjust falsey values and tweak based on review comments
2022-06-01 20:24:32 -04:00
Alexandre Girard
aa92518721
Low-code connectors: configurable source from yaml ( #13038 )
...
* checkout from alex/cac
* checkout from alex/cac
* checkout from alex/cac
* checkout from alex/cac
* Add missing tests
* Add missing files
* Add missing tests
* add missing file
* missing file
* missing file
* sengrid low code connector
* rename
* doc
* doc
* remove broken test
* rename
* jinja dependency
* Add comment
* comment
* comment
* pyjq dependency
* update import
* rename file
* delete unused file
* Revert "delete unused file"
This reverts commit 758e9393674a79223944758e9393674a79223944#12852 )
* checkout from alex/cac
* Add missing tests
* Add missing files
* missing file
* rename
* jinja dependency
* Add comment
* comment
* comment
* Revert "delete unused file"
This reverts commit 758e9393674a79223944snadalive@gmail.com >
* more precise exception
* rename class
* add comment
* Try to install packages to build jq
* isort
* only automake
* Revert "only automake"
This reverts commit c8fe154ffcab5a7afd087c9fb8eb33snadalive@gmail.com >
* fix tests and format
* format
* update
* better error message
* Add jq dependency
* Use request header provider
* rename
* rename field
* remove get_context method
* rename
* add a comment
* format
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
2022-06-01 07:31:52 -07:00
Alexandre Girard
d9fa24ffff
Low code connectors: implement components for sendgrid ( #12853 )
...
* checkout from alex/cac
* checkout from alex/cac
* checkout from alex/cac
* Add missing tests
* Add missing files
* Add missing tests
* add missing file
* missing file
* missing file
* rename
* doc
* doc
* remove broken test
* rename
* jinja dependency
* Add comment
* comment
* comment
* pyjq dependency
* rename file
* delete unused file
* Revert "delete unused file"
This reverts commit 758e9393674a79223944758e9393674a79223944#12852 )
* checkout from alex/cac
* Add missing tests
* Add missing files
* missing file
* rename
* jinja dependency
* Add comment
* comment
* comment
* Revert "delete unused file"
This reverts commit 758e9393674a79223944c8fe154ffc
2022-06-01 07:02:35 -07:00
Denis Davydov
acd1533cec
🎉 Python CDK: handle requests.exceptions.ChunkedEncodingError for broken connections ( #13260 )
2022-05-27 22:05:39 +03:00
Alexandre Girard
731c8bf178
Add airbyte protocol file ( #13234 )
2022-05-26 11:22:46 -07:00
Marcos Marx
dca2256a7c
Bump 2022 license version ( #13233 )
...
* Bump year in license short to 2022
* remove protocol from cdk
2022-05-26 15:00:42 -03:00
Brian Lai
c4ef7db5eb
[Low Code CDK] add support for string interpolation for request headers ( #13215 )
...
* add support for string interpolation for request headers + small refactor
* fix formatting
* move parameter provider to folder for consistency
2022-05-25 22:15:19 -04:00
Alexandre Girard
3894134d11
Bump year in license short to 2022 ( #13191 )
...
* Bump to 2022
* format
2022-05-25 17:56:49 -07:00
Alexandre Girard
150ab593f8
Low code connectors: core structure ( #12850 )
...
* checkout from alex/cac
* doc
* doc
* remove broken test
* rename
* rename file
* delete unused file
* rename
* abstract property
* isort
* update state
* Update comment
* remove incremental mixin
* delete comment
* update comments
* update comments
* remove no_state
* rename package
* pass parameters through kwargs
* update interface to pass source in interface
* update interface to pass source in interface
* rename to stream_slicer
* Low code connectors: string interpolation with jinja (#12852 )
* checkout from alex/cac
* Add missing tests
* Add missing files
* missing file
* rename
* jinja dependency
* Add comment
* comment
* comment
* Revert "delete unused file"
This reverts commit 758e9393674a79223944
2022-05-25 17:27:54 -07:00
Pedro S. Lopez
41dc82f056
CDK: provide better user-friendly error messages ( #12944 )
...
* initial implementation
* add parse_response_error_message tests
* move error handler to existing try/catch in AbstractSource
* formatting
* var rename
* use isinstance for type checking
* add docstrings
* add more abstract_source and httpstream tests
* fix wrong httperror usage
* more test cases
* bump version, update changelog
2022-05-19 17:28:24 -04:00
Serhii Chvaliuk
9a960c4ca6
CDK: TypeTransformer.default_convert catch TypeError ( #12948 )
...
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com >
2022-05-19 07:32:48 +03:00
Pedro S. Lopez
7ea27d0ecd
CDK: publish version 0.1.57 ( #12938 )
2022-05-17 16:33:06 -04:00
Pedro S. Lopez
b8703ea6a1
update CDK airbyte protocol models to fix master build ( #12829 )
2022-05-12 22:38:30 -04:00
Pedro S. Lopez
73c7fad7fc
CDK: emit AirbyteTraceMessage with exception trace information ( #12593 )
2022-05-06 17:16:57 -04:00
Sherif A. Nada
49810f8c13
sweep airbyte-integrations/bases/airbyte-protocol ( #12598 )
2022-05-05 13:51:16 -07:00
Sherif A. Nada
cc46255190
cull base-python and base-python-test ( #12596 )
2022-05-05 13:48:15 -07:00
Pedro S. Lopez
3c1eab3e27
generate AirbyeTraceMessage.type enum with descriptive class name ( #12581 )
...
* generate AirbyeTraceMessage `type` enum with descriptive class name
* add comment on `title` usage
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
* apply changes to bases/airbyte-protocol
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
2022-05-04 18:48:34 -04:00
LiRen Tu
a294f40383
Use jsongenerator 1.0.1 ( #12554 )
...
* Use jsongenerator 1.0.1
* Fix connectors base
2022-05-03 21:31:12 -07:00
Brian Lai
ceec454e30
update python cdk tutorial with updates to exchange rates api ( #12427 )
...
* update python cdk tutorial with updates to exchange rates api
* remove extra comment in source.py tutorial sample code
2022-04-29 13:44:04 -07:00
Pedro S. Lopez
53799cb9a2
🎉 CDK: support loading spec from yaml file ( #12104 )
...
* support loading spec from yaml file
* formatting
* remove commented code
* update comment
* remove unused file
* raise correct exception types
* bump version, update changelog
2022-04-20 13:18:46 -07:00
Pedro S. Lopez
a6a6be2588
allow importing IncrementalMixin from airbyte_cdk.sources.streams ( #11858 )
...
* allow importing IncrementalMixin from airbyte_cdk.sources.streams
* format file
* update docs with hint on where to import IncrementalMixin from
* bump cdk version to 0.1.54
* update changelog
* only build python 3.9
* update changelog with python dep bump
2022-04-14 11:42:57 -07:00
