1
0
mirror of synced 2025-12-25 02:09:19 -05:00
Commit Graph

6542 Commits

Author SHA1 Message Date
Tim Werner
32e767c9f6 🎉 Source File: add user-agent option (#14488)
* add user-agent option to source-file

* fix acceptance-test

* set version to 0.0 if version cannot be fetched

* vump connector version

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-13 13:46:05 -03:00
AJ
bda11b091a 🐛 Source Chargebee: fix transactions sort-by (#14672) 2022-07-13 18:02:11 +02:00
Alexandre Girard
a4c51cdc54 set default paginator (#14678) 2022-07-13 08:54:42 -07:00
Benoit Moriceau
8dc558d496 Bump local json connector (#14641)
* Bump local json connector

* update doc

* Fix publish

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-13 08:06:08 -07:00
wjwatkinson
19df797ac0 New Source: Kyriba (#12748)
* remove redundant error status retry logic

* added kyriba source

* implemented account incremental sync

* remove unnecessary scheams and rename accounts

* added cash flows

* allow for different kyriba domains

* added account balances

* added bank balances and intraday streams

* refactored with account sub stream class

* airbyte cannot accomodate nested incremental date fields

* fix nested date fields in schema

* fix cash flow schema and logic

* specify max retries

* change 401 status code, so it gets retried

* moved to requests native auth

* compare tokens

* update request auth on 401

* set request auth header on 401

* increase cash flow page size to 1000

* add retries to login

* only sync cash balances yesterday and earlier

* corrected account schema

* increased max retries

* create uuid for eod bank balances

* add standard cash balance sync

* add standard bank balance sync

* fixed cash flow schema

* full refresh cash balances to enable proper ELT

* full refresh bank balances to enable proper ELT

* add end date

* fix formatting None end date

* don't require end_date in regex

* fixed cash balance range increment

* added unit tests

* added tests updated update date time parameter formatting

* added unit tests

* removed modification

* formatted

* fix integration tests

* fix test connection integration test

* added lastKnownBalanceDate

* produce STATE with abnormal future state

* formatted

* added source kyriba

* added kyriba build

* removed unnecessary files

* address PR comments to improve code

* make api version name consistent

* decrease size of cash flow slices

* improve cash flow sync performance with 1 day stream slices

* reduce number of retries as slices checkpoint state

* currently only one version of the API

* fixed pr link

* fixed static code check issues

* added source kyriba

* correct doc

* remove redundant error status retry logic

* added kyriba source

* implemented account incremental sync

* remove unnecessary scheams and rename accounts

* added cash flows

* added account balances

* added bank balances and intraday streams

* airbyte cannot accomodate nested incremental date fields

* fix cash flow schema and logic

* specify max retries

* change 401 status code, so it gets retried

* compare tokens

* set request auth header on 401

* add retries to login

* increased max retries

* create uuid for eod bank balances

* add standard cash balance sync

* add standard bank balance sync

* fixed cash flow schema

* full refresh cash balances to enable proper ELT

* full refresh bank balances to enable proper ELT

* added unit tests

* added tests updated update date time parameter formatting

* removed modification

* formatted

* fix integration tests

* fix test connection integration test

* added source kyriba

* removed unnecessary files

* address PR comments to improve code

* currently only one version of the API

* fixed static code check issues

* correct mypy checks

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-13 11:11:07 -03:00
Marcos Marx
a2ec5cda13 rollback publish command skip-test option (#14643) 2022-07-13 09:45:54 -03:00
Yevhen Sukhomud
8abe9d92c2 13524 Resolved host port for mac os (#14663)
* 13524 Resolved host port for mac os

* 13524 Uodated testcontainer version
2022-07-13 18:36:49 +07:00
Yiyang Li
48f8f2f703 source-okta: add custom roles stream (#14610) 2022-07-13 10:54:13 +02:00
Denis Davydov
2570f9ce73 Source file: remove nested properties (#14535)
* #13787 source file: remove nested properties

* #13787: upd changelog

* #13787 revert processing json array, add unit test

* #13787 source file: upgrade pyarrow to allow builds on aarch64

* #13787 source file: upd version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-13 11:29:51 +03:00
Brian Lai
cf71ccb460 simple update to add a lookback_window to the datetime stream slicer (#14609) 2022-07-12 19:49:15 -04:00
Jimmy Ma
d0649c993c Add legacy update test (#14432)
* Add acceptance test for legacy state when updating a connection with new streams.
2022-07-12 14:48:42 -07:00
Lake Mossman
5d4d8aded0 fix precision of timestamps in comparison (#14645) 2022-07-12 14:33:32 -07:00
Tim Roes
0c80644fe6 🪟🐛 Keep refetching debug infos while job is running (#14341)
* Keep refetching debug infos while job is running

* Set delay to 2 minutes
2022-07-12 23:19:10 +02:00
Nilkamal
bfa54aca50 🎉 New Source: Elasticsearch (#14118)
* New source Elasticsearch

* Update README.md

* Update tests

* File additions and changes in test

* Deleting spec.json

* File additions and changes in test

* Fixed multiple streams bug in discover

* run format

* format files

* eof and update doc

* correct spec test

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-12 18:08:17 -03:00
Anne
767992e591 Remove withRefreshedCatalog param from updateConnection endpoint (#14477)
* New webBackendConnectionUpdate endpoint with withRefreshedCatalog param removed
2022-07-12 14:00:33 -07:00
Serhii Chvaliuk
c87068bd39 expected_records.txt updated (#14627)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-07-12 21:19:46 +03:00
Vladimir
428e482513 🪟 🎨 Redesign Sign-in/up pages (#14103)
* convert styled components in Auth component to scss

* move GitBlock to separate component to the left side

* fix color variables

* replace styled component FormTitle with scss modules

* create new PersonQuoteCover component

* update styles for LogIn button

* update login title

* remove Airbyte logo from auth page

* update related components after changes in FormTitle component

* increase size of "forgot password" text

* add cart.com company logo

* add all necessary elements and styles

* polished css for PersonQuoteCover component

* completed PersonQuoteCover component and moved to separate folder

* move FormTitle to separate folder

* move GitBlock to separate folder

* add tests for PersonQuoteCover component

* update Header component:
- remove "Back" button
- replace styled components with css

* leave comment for News component

* update GitBlock component:
- extend to support props
- add tests

* update GitBlock css classes order

* update SpecialBlock component

* update SignupPage with optional props

* set full width for Sign Up button

* replace News with PersonQuoteCover component

* Update airbyte-webapp/src/packages/cloud/views/auth/components/FormTitle/index.tsx

Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>

* move testimonials photo and logo to separate folder

* update tests

* Fixes after comments to PR:
- fixed not grayed image in Firefox
- update test snapshots

* remove alt text on company logo

* remove all conditionally rendered text (preparation for LaunchDarkly)

* update tests

* simplify implementation of title style

* Update quote company logo testid

* Move emoji from css to en file in signup specialblock component

* Update accept email invite heading and button to match styles
Add status message to signup page

Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>
Co-authored-by: Edmundo Ruiz Ghanem <edmundo@airbyte.io>
2022-07-12 14:16:41 -04:00
Yiyang Li
92b2d9cbc1 🎉 Source Okta: add GroupMembers stream (#14380)
* add Group_Members stream to okta source

- Group_Members return a list of users, the same schema of Users stream.
- Create a shared schema users, and both group_members and users sechema use it as a reference.
- Add Group_Members stream to source connector

* add tests and fix logs schema

- fix the test error: None is not one of enums though the enum type includes both string and null, it comes from json schema validator
ddb87afad8/jsonschema/_validators.py (L279-L285)
- change grouop_members to use id as the cursor field since `filter` is not supported in the query string
- fix the abnormal state test on logs stream, when since is abnormally large, until has to defined, an equal or a larger value
- remove logs stream from full sync test, because 2 full sync always has a gap -- at least a new log about users or groups api.

* last polish before submit the PR

- bump docker version
- update changelog
- add the right abnormal value for logs stream
- correct the sample catalog

* address comments::

- improve comments for until parameter under the logs stream
- add use_cache on groupMembers

* add use_cache to Group_Members

* change configured_catalog to test

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-12 14:49:57 -03:00
Christophe Duong
43f905b8f9 🎉 Add YAML format to source-file reader (#14588)
* Add yaml reader

* Update docs

* Bumpversion of connector

* bump docs

* Update pyarrow dependency

* Upgrade pandas dependency

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-12 19:34:17 +02:00
Marcos Marx
fc2a590d5b Helm Chart: Add external temporal option (#14597)
* conflict env configmap and chart lock

* reverting lock

* add eof lines and documentation on values yaml

* conflict json file

* rollback json

* solve conflict

* correct minio with new version

Co-authored-by: Guy Feldman <gfeldman@86labs.com>
2022-07-12 13:39:27 -03:00
Bas Beelen
ef49424a07 source-hubspot: change mentioning of Mailchimp into HubSpot doc (#14620) 2022-07-12 17:36:17 +02:00
Amruta Ranade
c1b9f82239 Docs: Fixed broken links (#14622)
* fixing broken links

* more broken links
2022-07-12 10:12:54 -04:00
Serhii Chvaliuk
67dc38b014 🎉 Source Github: improve schema for stream pull_request_commits added "null" (#14613)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-07-12 16:17:19 +03:00
Sherif A. Nada
743e6c2d6c [Low-code CDK] Add ability to remove fields (#14402) 2022-07-11 19:04:57 -07:00
Tobias Macey
f4524e34c5 version: Update Parquet library to latest release (#14502)
The upstream Parquet library that is currently pinned for use in the S3 destination plugin is over a year old. The current version is generating invalid schemas for date-time with time-zone fields which appears to be addressed in the `1.12.3` release of the library in commit c72862b613
2022-07-11 15:25:37 -07:00
Topher Lubaway
9c6c092a22 Revert "Improving docusaurus sidebar generation (#1927) (#14369)" (#14596)
This reverts commit a2c194a11f.
2022-07-11 15:27:14 -05:00
Baz
c13e35821c 🐛 Source Intercom: fix conversation_parts stream schema (#14554) 2022-07-11 22:37:35 +03:00
Edmundo Ruiz Ghanem
900c21230e Fix md and lg scss var names (#14579) 2022-07-11 13:29:08 -04:00
Anna Lvova
4e80a67d89 🐛Source Freshdesk: Update parsing of start_date param (#14049)
* Upd parsing of start_date

* Bump version

* Upd unit tests

* Upd timeout

* Remove timeout

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-11 18:56:01 +03:00
Baz
7221842ecb 🐛 Source Notion: fix normalization (#14505) 2022-07-11 15:23:32 +03:00
Sunny
0bfc7e1925 doc: fix deadlink in repo top level readme.md (#14564) 2022-07-11 12:15:46 +02:00
Serhii Chvaliuk
94e3e0ea27 🐛 Source Amazon Ads: Fix SAT test fails - updated expect_records (#14533)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-07-11 11:05:26 +03:00
Liren Tu
078f5fcdd8 🎉 Postgres source: retrieve only the tables in the publication under cdc mode (#14447)
* Move helper methods to dedicated class

* Add unit tests for existing methods

* Move constant to debezium event util

* Include only publicized tables under cdc mode

* Add comments

* Add unit test

* Format code

* Move isCdc method out of postgres source

* Bump versions

* Update doc

* Add comment in unit test

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-09 21:54:00 -07:00
Greg Solovyev
97eddb087e Add code owners for normalization and jdbc-based connectors (#14520) 2022-07-08 16:59:19 -07:00
Brian Lai
374e265fcb [Low Code CDK] configurable oauth request payload (#13993)
* configurable oauth request payload

* support interpolation for dictionaries that are not new subcomponents

* rewrite a declarative oauth authenticator that performs interpolation at runtime

* formatting

* whatever i don't know why factory gets flagged w/ the newline change

* we java now

* remove duplicate oauth

* add some comments

* parse time properly from string interpolation

* move declarative oauth to its own package in declarative module

* add changelog info
2022-07-08 16:49:16 -04:00
Lake Mossman
f8092708bb CDC Acceptance Test (#14370)
* Use Debezium Postgres image for CDC tests

* Formatting

* add cdc acceptance tests

* make method public

* format

* add update destination definition version method

* wait for successful job

* add acceptance tests for incremental with old dest, and for delete + some refactoring

* format

* fix assignment of sourceDbConfig

* fix init

* remove logs

* increase timeout on deleteConnection test to prevent transient failures

* TEMPORARY add logs to help with debugging CI failures

* add -i to acceptance test for debugging

* add back destinationPsql start to see if it fixes the issue

* Revert "add -i to acceptance test for debugging"

This reverts commit 693a955041.

* Revert "TEMPORARY add logs to help with debugging CI failures"

This reverts commit a30efd2c54.

* fix timestamp comparison

Co-authored-by: jdpgrailsdev <jpearlin1@gmail.com>
2022-07-08 13:45:44 -07:00
Mykyta Serbynevskiy
a2c194a11f Improving docusaurus sidebar generation (#1927) (#14369)
* Improving docusaurus sidebar generation (#1927)

* Added "Career & open positions" folder to sidebar, adjusted "Project overview" folder

* Deleted "career-and-open-positions" folder from sidebar
2022-07-08 14:18:27 -05:00
Lake Mossman
e1c2ca234e remove CONTAINER_ORCHESTRATOR env var from acceptance tests as it is unnecessary (#14541) 2022-07-08 11:38:55 -07:00
Vladimir
1e808ec7ee 🪟 🐛 Update connection name - fix minor issues (#13550)
* align text to center in edit mode

* fix firing onChange event twice

* fix wrong prop drilling

* fix issues:
- unable to delete last chart in input
- delete the whole name by 'select all' command
- trim entered names
- don't allow to save empty names

* remove debug from test

* remove unused import

* update tests due to changes in base Input component

* reorder default input data-testId in order to have ability to overwrite it

* fix e2e tests

* Improvements:
- replace "props: any" with right types
- rename 'addEnterEscFuncForInput' to more general and self-describable name

* replace styled components with scss modules

* add eslint-plugin-css-modules rules

* Fixes:
- turn on eslint css modules rule as error
- remove unused styles

* minor fix after master merge

* Revert "Fixes:"

This reverts commit d7fb68abc6.

* Revert "add eslint-plugin-css-modules rules"

This reverts commit 0333a70108.

* apply suggested changes after PR review

* add updates due to changes in PR #13914

* add react-app-rewired to tweak webpack config and fix the css order issue

* remove test flag "CI=true"

* Revert "add react-app-rewired to tweak webpack config and fix the css order issue"

This reverts commit 81d7ba42
2022-07-08 21:11:58 +03:00
Teal Larson
df24831cd4 🪟🔧Update Segment tracking event names (#13859)
* wip

* Update high priority segment calls

* other segment calls use new format

* allow for non-legacy segment calls

* comment

* cleanup

* use single TrackActionAction

* renames from code review

* typo fix

* WIP connect hook to react hooks testing library

* test parsing of legacy namespace

* cleanup
2022-07-08 13:35:34 -04:00
Nataly Merezhuk
111cccef95 🪟🐛 Fixes connector status icon alignment (#14465)
* Changes vertical alignment on icon value to 'bottom' when the status is 'inactive'.

* Update StatusIcon storybook to test adding a value

Co-authored-by: Edmundo Ruiz Ghanem <edmundo@airbyte.io>
2022-07-08 19:00:43 +02:00
Mark Berger
addf57cc83 Fixed disabled and cycled scrollbar on Connector Set Up page (#14531) 2022-07-08 19:55:32 +03:00
Lake Mossman
d2ea57499d Run container orchestrator acceptance tests (#13699)
* Add CONTAINER_ORCHESTRATOR=true flag to kube acceptance tests

* Add CONTAINER_ORCHESTRATOR=true flag to GKE acceptance tests too

* fix downtime test

* fix the other orchestrator tests

* move container orchestrator acceptance tests to their own class
2022-07-08 09:27:01 -07:00
Alexandre Girard
01fc7a48c7 move jinja macros to their own file (#14515)
* move jinja macros to their own file

* format

* export as dict

* format
2022-07-08 07:56:06 -07:00
Teal Larson
cd1d960b0f 🪟 🎨 Front end/Design tweaks (#14257)
* updates from pairing

* cleanup from review
2022-07-08 10:46:00 -04:00
Alexandre Girard
1cf89c63c3 add reference docs for declarative source (#14501)
* generate pdoc

* set python version to 3.8

* mimic doc

* set the os

* Revert "generate pdoc"

This reverts commit 82bdeabe8f.

* add missing docs

* add declarative to index

* missing init files

* generate rst files
2022-07-08 07:26:27 -07:00
Serhii Lazebnyi
391e97f377 Bumped release stage version (#14538) 2022-07-08 16:50:21 +03:00
Serhii Lazebnyi
f896c574d1 🎉 Source Amazon S3: Fix docs link issue (#14397)
* Fix UI connector name and link issue

* Revert name to S3
2022-07-08 15:13:00 +03:00
So-shi Nakachi
806c93922f doc: fix typo in deploy on kubernetes (#14489) 2022-07-08 12:41:11 +02:00
Denis Davydov
fde2068c43 Source Monday: timeout errors (#14443)
* #61 alpha beta. Source Monday: timeout errors

* #61 alphabeta: source monday - upd changelog

* #61 Source Monday: adjust SATs

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-08 10:56:50 +03:00