1
0
mirror of synced 2026-01-04 00:04:25 -05:00
Commit Graph

7603 Commits

Author SHA1 Message Date
Sophia Wiley
821c98e66c Added Google OAuth login info to docs (#17070)
* added google login info

* edited doc
2022-09-26 15:43:03 -07:00
Benoit Moriceau
bbb59d8675 Bmoric/temporal cleaning cron (#16414)
* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Add the injected temporal client to airbyte-cron

* Update cron to make it work

* Remove useless Bean factory

* PR comments

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Improve conditional bean check

* Match existing Optional functionality

* Add notEquals check

* Add missing env var to Helm chart

* Fix typo

* Mark LogConfigs as Singleton

* WIP Convert airbyte-workers to Micronaut framework

* Rebase cleanup

* Fix broken tests

* Simplify code

* Support control vs data plane configuration

* make WORFKLOW_PROXY_CACHE non-static to avoid cacheing mocks in unit tests

* Formatting

* Pairing on Worker Micronaut (#16364)

* add RouteToSyncTaskQueue activity

* use new route activity in connection manager workflow

* format

* call router service for task queue

* Revert temporal proxy changes

* Formatting

* Fix default value

* register new route activity in test

* fix SyncWorkflowTest now that it isn't doing any routing

* Update dependencies

* More dependency updates

* Update dependencies

* Improve conditional bean check

* Match existing Optional functionality

* Add notEquals check

* Add missing env var to Helm chart

* Fix typo

* Mark LogConfigs as Singleton

* Env vars for log/state storage type

* Remove use of Optional in bean declarations

* Fix typo in config property name

* Support Temporal Cloud namespace

* Change to @Value

* Use correct value for conditional check

* Upgrade Micronaut

* Fix merge conflict

* Formatting

* Add missing env var

* Use sync task queue environment variable

* Handle sync task queue as set

* format and force http

* Handle case where sync task queue is empty

* Add correct path to config property

* Remove unused import

* Remove conflict

* Remove unused parameter

* Formatting

* Use pattern for condition process factory beans

* Cleanup

* PR feedback

* Revert hack for testing

* Fix temporal restart by status (#16447)

* Update application.yml

* Re add worker dep

* Add missing env var

* PR comments

* Bmoric/move temporal client (#16778)

* tmp

* tmp

* View diff

* Move part of the temporal client

* tmp

* copy Temporal Utils test

* Uniq Temporal Utils

* Uniq Temporal Workflow Utils

* Move CancellationHandler

* Move commons-temporal to being shared between platform, connector and CLI

* Rm worker dependency from cron

* Fix build

* Update with right value for cron.

* Fix dep conflict

Co-authored-by: jdpgrailsdev <jpearlin1@gmail.com>
Co-authored-by: pmossman <parker@airbyte.io>
Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io>
2022-09-26 15:14:35 -07:00
Teal Larson
c49a5320f9 🪟 🐛 disable launch button for disabled connections (#17172) 2022-09-26 17:06:51 -04:00
Pedro S. Lopez
8c6313374e SentryJobErrorReportingClient: set event platform from parsed stacktrace (#16906)
* set sentry event platform from parsed stacktrace

* fix pmd

* use an enum

* use enum in test

* fix imports

* fix imports

* format

* fix imports
2022-09-26 16:49:22 -04:00
Lake Mossman
c4fb6b7f9e 🪟 🎨 Connector Field UX Improvements (#16152)
* use Source name and Destination name

* move label messages into info tooltips

* pass tooltip props through IntoTooltip and set placement

* add error message to ControlLabels

* refactor to reduce duplication and add optional indicator

* add label as header in info tooltip and fix tooltip word wrapping

* remove placeholders and style tooltip a bit better

* make example background darker

* remove unnecessary style

* put examples in flex box

* use property label in labeled switch

* use info tool tip for condition section

* remove unused class name

* remove colon

* add styling to lists inside of tooltip

* move errors below inputs and add error flag to control input

* remove label css

* use new property label for array section as well

* fix css

* make examples a bit more circular

* put optional text into translatable string

* fix TextWithHTML props and export

* simplify CSS selector

* rename IProps to Props

* fix export

* move examples into its own component

* remove unnecessary var

* conditionSection feedback

* update design of group sections

* fix empty group control styling

* add comment

* fix position and sizing of group control sections

* adjust margin

* add main changed components to storybook

* linting fixes

* fix group section styling and clean up storybook

* fix stylelint errors

* wrap storybook components in cards

* use card prop to fix build error

* fix info tooltip positioning and prevent wrapping on group label

* adjust infotooltip styling again

* fix info tooltip position on safari as well

* fix styling of condition section dropdown

* small spacing adjustment

* another small padding adjustment

* switch display to block

* simplify scss classes

* rename props class

* fix casing of scss variables

* use scss variables in some places

* move PropertyError to Property folder

* replace displayError with hasError

* rename infoMessage to infoTooltipContent

* move stories to other folders

* fix display and local var

* update snapshots

* properly set htmlFor on group control label and fix nest group control styling

* fix tooltip border radius

* fix group controls border radius

* fix spacing of field forms

* fix input border and background colors

* fix display types in property section

* update snapshot classes
2022-09-26 13:19:22 -07:00
Eugene
7ce39ed8a1 🐛Destination-snowflake: added connection string identifier (#17115)
* [16767] Destination-snowflake: added connection string identifier
2022-09-26 22:09:40 +03:00
Subodh Kant Chaturvedi
99746082a4 mysql source : implement support for snapshot of new tables in cdc mode (#16954)
* mysql source : implement support for snapshot of new tables in cdc mode

* undo unwanted changes

* add more assertions

* format

* fix build

* fix build

* revert acceptance test changes

* bump version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-27 00:14:07 +05:30
Sajarin
cf0c082e47 chore: generate source_specs.yaml file (#17160) 2022-09-26 14:34:09 -04:00
Edmundo Ruiz Ghanem
9db329eb6c 🪟 🧹 Move minor UI components to compoents/ui (#17109)
* SingletonCard to ui/components/Toast

* Move Spinner to components/ui

* Move SlickSlider to components/ui

* Move ProgressBar to components/ui and add storybook

* Move Markdown to components/ui

* Move CatalogDiffModal story to its own group

* Move CodeEditor to components/ui, add storybook

* Move TextWithHTML to components/ui

* Move StepsMenu to components/ui, add Storybook

* Add args to CodeEditor and TextWithHTML storybooks

* Move SideMenu to components/ui

* Fix progressbar storybook args

* Add ReleaseStateBadge story

* Move ImageBlock to components/ui

* Move BarChart to components/ui and add storybook

* Move Breadrumbs to components/ui

* Move LoadingBackdrop to components/ui

* Move InfoBox to components/ui and add storybook

* Fix unit tests
2022-09-26 14:30:43 -04:00
Björn Schuberg
05bd34f477 🎉 New Source: Adjust (#16051)
* Initial version of the adjust source connector

* source-adjust: add bootstrap.md

* adjust-source: add setup guide

* adjust-source: update integration READMEs

* source-adjust: better stream name

* source-adjust: fix sample conf

* source-adjust: add spec order metadata

* source-adjust: improve spec dimension doc

* source-adjust: warn on custom metric cast failure

* source adjust: Update source_definitions.yaml

* source adjust: Update documentation url
2022-09-26 14:25:51 -04:00
Eugene
4e381abbd2 🐛Source-snowflake: added connection string identifier (#17116)
* [16767] Source-snowflake: added connection string identifier
2022-09-26 21:19:56 +03:00
Xiaohan Song
9abcbadd93 Data plane needs gcp secret hydrator and persistence for replication activity (#17149)
* use prototype for jwt signature because we need a new one

* add comments on why using provider and prototype:

* secret hydrator fix for data plane

* style fix
2022-09-26 11:05:21 -07:00
Davin Chia
517466eb66 Remove unused webbackend search code. (#17008)
* Remove unused webbackend search code.

* Remove unused request body.

* Remove unused request body.

* Remove failing test.

* Remove failing test.

* Remove failing test.

Co-authored-by: Parker Mossman <parker@airbyte.io>
2022-09-26 10:46:01 -07:00
Yurii Bidiuk
e07bd6bee3 Destination BigQuery: add user-agent header (#16890)
* Destiantion BigQuery: add user-agent header

* fixed comments

* fix tests

* refactor code

* refactoring

* bump version

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-26 20:31:27 +03:00
Krisjan O
3939ff34cd Source: Amazon Seller Partner - Updated Report API version + New reports (#16629)
* Updated Report API version + New reports

* Fixed typo and added ability to set fetch period to a fixed range

* Removed old code + updated connection test doc

* minor fix for python format

* Bump version to 0.2.26 and updated docs

* Docs update, fix typo in PR number

Co-authored-by: Tuan Nguyen <anhtuan.nguyen@me.com>
Co-authored-by: Sajarin <sajarindider@gmail.com>
2022-09-26 13:19:06 -04:00
Yatsuk Bogdan
8f8ab19ebf 🪟 🎨 Adds reset streams details to status list (#16600)
* Adds reset streams details to status list

* Updates styled-components into SASS module; Refactors pluralization logic; Adds truncating for long list of reset streams

* Applies isOpen logic for reset streams details; Fixes TS job property in job object

* Moves ResetIcon to components/icons folder

* Fix comments

* Fixes SCSS variables; Removes testing code

* Fixes style and localization

* Fixes error state of reset streams

* Updates job item design

* Fixes localization logic

* Fixes comments

* Fix comments

* Update airbyte-webapp/src/components/JobItem/components/MainInfo.tsx

Co-authored-by: Krishna (kc) Glick <krishna@airbyte.io>

* Fixes styles; Fixes label and statusIcon getter logic

* Fixes ResetStreamDetails names prop

* Fixes else if statement

Co-authored-by: Krishna (kc) Glick <krishna@airbyte.io>
2022-09-26 19:32:01 +03:00
Subodh Kant Chaturvedi
cb7076bffe allow nullable columns to be used as cursor (#17131)
* allow nullable columns to be used as cursor

* add test

* fix test

* bump version

* Revert "bump version"

This reverts commit a3cf9aff84.

* bump version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-26 21:20:53 +05:30
Tim Roes
4ca32a4051 Fix minor issues in connection/connector tables (#17074) 2022-09-26 17:04:44 +02:00
Tim Roes
871929de15 Improve styling of sidebar (#17072) 2022-09-26 16:48:58 +02:00
Serhii Chvaliuk
e3553fea22 🎉 Source mixpanel: Service Accounts added (#16915)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-26 16:34:45 +03:00
Artem Inzhyyants
a9630e3fee 🐛 Source Twilio: add test data for Message Media and Conferences (#17134) 2022-09-26 14:57:26 +02:00
Arsen Losenko
d32a0f1a79 Source Klaviyo: change releaseStage to GA (#17044) 2022-09-26 15:32:39 +03:00
Michael Siega
32a984123e convert the workspace update api to patch style (#16739)
* convert the workspace api to patch style

* check for empty email in workspace update
2022-09-26 11:29:51 +02:00
darynaishchenko
b8a647270a Chargebee: Added "custom fields" to the relevant Chargebee source data streams. (#17056)
* Chargebee: added custom fields

* Chargebee: added value types json schema

* Chargebee: added docs, updated docker label, refactored utils

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-26 11:16:16 +03:00
Liren Tu
d14b4423ea Publish R2 destination (#17122)
* Format code

* Update doc

* Add r2 to build doc

* Update readme

* Add r2 to destination definitions

* Move test resource to base-java-s3

* Remove redundant dependency

* Add missing dependency

* Format json

* Remove s3-destination-base-integration-test setting.gradle root

* Add missing lzo dependency

* Fix import in destination jdbc
2022-09-25 21:45:00 -07:00
Nugusbayev Kanagat
3494405078 🎉 New Destination: R2 Cloudflare (#15296)
* added destination-r2 spec

added base-java-s3 module

updated common s3 lib

local

changed dependency of s3 in all realted modules

minor style fixes

added common module for s3 integration tests

* fix configuration for r2 integration tests

* fix configuration for r2 integration tests

* added docs for destination-r2

* minor import fixes

* mark test as disabled

* fixed imports in destination-snowflake

* restored styling task

* added tests for S3DEstinationConfig

* added upload threads count configuration for r2(due to limitations)

* deleted parquet format type

* Fix import in gcs

* Fix import in redshift

* Fix import in snowflake

* Fix one more import in gcs

* Fix one more import in redshift

* Fix import in databricks

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
2022-09-25 19:20:47 -07:00
Roman Yermilov [GL]
b261d1fe77 Source Harvest: update release stage (#17029)
* Source Harvest: update release stage

* Updated CDK version to latest

Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
2022-09-23 22:45:18 +02:00
Tim Roes
caa03fde9d Prevent rerendering on stale queries (#17106) 2022-09-23 21:53:30 +02:00
Edmundo Ruiz Ghanem
c943b8d3b5 Show "source defined" as cursor for CDC streams, move strings to i18n (#16999)
* Update check on PathPopout to ensure source defined appears when the path provided is empty

* i18n source defined message

* Update source defined message format
Move destination namespace names to i18n
2022-09-23 15:33:35 -04:00
Denys Davydov
68cae56610 Source Facebook Marketing: limit time range to 37 months (#17027)
* #481 oncall source fb marketing: limit time range to 37 months

* source fb marketing: upd changelog

* #391 source facebook-marketing: retry on getting error 960

* source facebook marketing: format code

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-23 21:57:27 +03:00
Denys Davydov
d62f88196f Source Salesforce: perform call to objects list in connection check (#17094)
* #629 oncall - source salesforce: perform call to objects list in connection check

* source salesforce: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-23 21:57:11 +03:00
Tim Roes
d4b7f4ef6d 🪟 🐛 Fix error message with missing docs (#17071)
* Fix error message with missing docs (#17014)

* Reverse type check logic
2022-09-23 19:48:14 +02:00
Davin Chia
4fe6a8bac2 Make listing all connections for workspace faster. (#17004)
Instead of always listing all the connections in the workspace and filtering deleted connections in memory, we want to do this in the database.
2022-09-23 12:20:07 -05:00
Edmundo Ruiz Ghanem
c1981e1add 🪟 🧹 Replace most usages of styled headings with <Text /> component (#17048)
* Move WorkspacesPage to scss, remove use of title components

* Move Onboarding Final step component to SCSS

* Replace H1 with Title in ProgressBlock

* Move TitlesBlock to SCSS

* Fix text size in TitlesBlock

* Move PreferncesPage modules to scss

* Reduce Prefernces Page description size

* Replace H6 with Text in ConnectionPageTitle
Remove H6 component

* Replace H2 with Text in ErrorOcurredView

* Replace H4 with Text in Transformation view when it's not supported
Update not supported text

* Update TryAfterErroBlock to use scss

* Migrate EmptyResourceListView to SCSS modules

* Replace H3 usages in TableItemTitle with Text component

* Remove H2 from the Titles components. Mark H3 and H5 as deprecated

* Migrate TableItemTitle to scss and remove H3 usage

* Update new scss with spacing vars

* Update test snapshots
2022-09-23 11:09:29 -04:00
Octavia Squidington III
8dfb8b0c92 Bump Airbyte version from 0.40.8 to 0.40.9 (#17093)
Co-authored-by: terencecho <terencecho@users.noreply.github.com>
v0.40.9
2022-09-23 09:46:09 -05:00
Topher Lubaway
6ddf32a84c More clear direction on flaky test (#17085) 2022-09-23 09:31:59 -05:00
Baz
45503de7d9 Source Netsuite: mention API method used to fetch the data for connector (#17082) 2022-09-23 13:51:53 +03:00
Baz
5928b48c41 🐛 Source Recharge: fix Charges.total_weight field data-type (#17080) 2022-09-23 13:26:23 +03:00
andriikorotkov
dbe25f6588 updated DB2 source documentation (#17078) 2022-09-23 12:10:31 +03:00
Jimmy Ma
89558542f5 Fix format (#17066) 2022-09-22 15:23:20 -07:00
Krishna (kc) Glick
31e6179c9f Revert "Fix error message with missing docs (#17014)" (#17063)
This reverts commit e931b69cba.
2022-09-22 17:02:09 -04:00
Edward Gao
23c8efa2b2 Destination Bigquery / Bigquery-denormalized: Only override dataset ID if stream namespace is null/empty (#17054)
* stop overriding namespace?

* set namespace if needed

* also check for empty namespace

* version bump + changelog

* auto-bump connector version [ci skip]

* sanitize dataset id

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 12:41:47 -07:00
Teal Larson
39d819297f 🪟 🐛 Schedule type cron bug fix (#17058)
* remove inaccurate and unneeded setting of schedule type

* Fixing & adding tests

Co-authored-by: KC <krishna@airbyte.io>
2022-09-22 13:57:05 -05:00
Oleksandr Sheheda
6465aae0dc [16617] Source BigQuery: added user agent header (#16902)
* [16617] Source BigQuery: added user agent header

* [16617] Source BigQuery: added user agent header

* [16617] Source BigQuery: added user agent header
updated image tag, added changelog

* [16617] Source BigQuery: added user agent header
fixed NPE

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 21:45:06 +03:00
Mark Berger
e6858830ef Sidebar headless UI menu (#16333)
* Replaced sidebar menu with headlessui

* Removed unnecessary tag li

* Removed unnecessary Sidebar Popout

* Changed export type

* Added keyboard support

* Removed styled-components

* Removed commented code

* Disabled ESLint rule css-modules/no-undef-class

* Review fixes

* Fixed shorthand property

* Review fixes

* Review fixes #2

* Review fixes #3

* Review fixes #4

* Review fixes #5

* Review fixes #6

* Review fixes #7

* Update airbyte-webapp/src/packages/cloud/views/layout/SideBar/SideBar.module.scss

Co-authored-by: Vladimir <volodymyr.s.petrov@globallogic.com>

* Update airbyte-webapp/src/packages/cloud/views/layout/SideBar/SideBar.module.scss

Co-authored-by: Vladimir <volodymyr.s.petrov@globallogic.com>

Co-authored-by: Vladimir <volodymyr.s.petrov@globallogic.com>
2022-09-22 21:33:41 +03:00
Cole Snodgrass
e662cf0d3a rm airbyte-scheduler; migrate existing client to airbyte-server (#17051)
* wip; move scheduler client to server package

* remove additional references to airbyte-scheduler

* removed more references to airbyte-scheduler
2022-09-22 10:25:52 -07:00
Anne
90ceb1bacb Change break to continue (#17015)
* Change break to continue
2022-09-22 10:04:12 -07:00
Yashkumar Makwana
abed1ce8b6 🎉New Source: Zenefits (#14809)
* New Source: Zenefits

* updated Readme.md

* Update spec.json

* Update request headers

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update source.py

* removed sample_files

* 1. updated source_definition
2. created a new file zenefits.md

* Delete TODO.md

* fix: remove .class files

* fix: added back source_definition

* fix: rebase with origin master

* fix: updated specs

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
2022-09-22 22:25:21 +05:30
Dustin Shapiro
6ddcb5df05 fix: update ingress template value (#16931)
Use $ as the Helm the range macro is used.

Co-authored-by: Dusty Shapiro <dusty@rvvup.com>
2022-09-22 11:26:17 -05:00
Yurii Bidiuk
bb39b36016 Destiantion Kinesis: add required fields (#16952)
* Destiantion Kinesis: add required fields

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 18:03:26 +03:00