104 Commits

Author	SHA1	Message	Date
Joe Bell	db37aadf67	Improve Authentication Error handling (#21087) ... * refactor checks * Add Auth Error Checks * rebase from master * some weird versioning issue * 🪟🎉 Connector builder: Substream slicer and cartesian slicer (#20861) * improve some types * improve further * clean up a bit more * refactor loading state * move loading state up * remove isLoading references * remove unused props and make fetch connector error work * remove special component for name * remove top level state for unifinished flows * start removing uiwidget * Update airbyte-webapp/src/views/Connector/ConnectorCard/ConnectorCard.module.scss Co-authored-by: Tim Roes <tim@airbyte.io> * remove undefined option for selected id * remove unused prop * fix types * remove uiwidget state * clean up * adjust comment * handle errors in a nice way * do not respect default on oneOf fields * rename to formblock * reduce re-renders * pass error to secure inputs * simplify and improve styling * align top * code review * remove comment * review comments * rename file * be strict about boolean values * add example * track form error in error boundary * review comments * handle unexpected cases better * enrich error with connector id * 🪟🎉 Add copy stream button (#20577) * add copy stream button * review comments * rename prop * 🪟🎉 Connector builder: Integrate connector form for test input (#20385) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * fix small stuff * add warning label * review comments * adjust translation Co-authored-by: lmossman <lake@airbyte.io> * use request_body_json instead of request_body_data * 🪟 🎨 Move `Add` button into the line of Connector Builder key value list fields (#20699) * move add button into line * add stories for empty with control, and content + control * change button name to Control * 🪟🎉 Connector builder: Allow defining inputs (#20431) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * handle stored form values that don't contain new fields properly * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * 🪟🎉 Connector builder authentication (#20645) * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * fix keys * 🪟🎉 Connector builder: Session token and oauth authentication (#20712) * session token and oauth authentication * fill in session token variable * typos * make sure validation error does not go away * 🪟🎉 Connector builder: Always validate inputs form (#20664) * validate user input outside of form * review comments Co-authored-by: lmossman <lake@airbyte.io> Co-authored-by: lmossman <lake@airbyte.io> * fix merge conflict with dropdown prop being renamed to control * [Connector Builder] Add paginator (#20698) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * handle stored form values that don't contain new fields properly * session token and oauth authentication * fill in session token variable * fix merge of default values * add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts * add cursor and primary key fields to ui * save * add page size and token option inputs * fixes after rebase * add pagination * fix pagination types * handle empty field_name better * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * typos * make sure validation error does not go away * make primary key and cursor optional, and reorder * save toggle group progress * fix style of toggle label * handle empty values better * fix page size/token option field validation and rendering * handle cursor pagination page size option correctly Co-authored-by: Joe Reuter <joe@airbyte.io> * [Connector Builder] Add stream slicer (#20748) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * handle stored form values that don't contain new fields properly * session token and oauth authentication * fill in session token variable * fix merge of default values * add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts * add cursor and primary key fields to ui * save * add page size and token option inputs * fixes after rebase * add pagination * fix pagination types * handle empty field_name better * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * typos * make sure validation error does not go away * make primary key and cursor optional, and reorder * save toggle group progress * fix style of toggle label * handle empty values better * fix page size/token option field validation and rendering * handle cursor pagination page size option correctly * save stream slicer progress * finish stream slicer * fix stream slicer fields and validation Co-authored-by: Joe Reuter <joe@airbyte.io> * debounce form builder values update to reduce load * 🪟🔧 Connector builder: use new lowcode manifest (#20715) * use new manifest yaml * Update airbyte-webapp/src/components/connectorBuilder/types.ts Co-authored-by: Lake Mossman <lake@airbyte.io> * use updated manifest types Co-authored-by: Lake Mossman <lake@airbyte.io> * add substream slicer * add substream and cartesian slicer * debounce validation as well * akways show stream test button in error state if there are errors * fix type of oauth input * add validation schema for add stream form * validate all views on test click * add type to prevent console warning * do not allow path for substream slicer request option * do not show request option for substream slicer * rewrite stream slice field tooltip Co-authored-by: Tim Roes <tim@airbyte.io> Co-authored-by: lmossman <lake@airbyte.io> * 🪟🔧 Connector builder: Performance improvements (#20620) * improve some types * improve further * clean up a bit more * refactor loading state * move loading state up * remove isLoading references * remove unused props and make fetch connector error work * remove special component for name * remove top level state for unifinished flows * start removing uiwidget * Update airbyte-webapp/src/views/Connector/ConnectorCard/ConnectorCard.module.scss Co-authored-by: Tim Roes <tim@airbyte.io> * remove undefined option for selected id * remove unused prop * fix types * remove uiwidget state * clean up * adjust comment * handle errors in a nice way * do not respect default on oneOf fields * rename to formblock * reduce re-renders * pass error to secure inputs * simplify and improve styling * align top * code review * remove comment * review comments * rename file * be strict about boolean values * add example * track form error in error boundary * review comments * handle unexpected cases better * speed up some bits * more changes * enrich error with connector id * 🪟🎉 Add copy stream button (#20577) * add copy stream button * review comments * rename prop * 🪟🎉 Connector builder: Integrate connector form for test input (#20385) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * fix small stuff * add warning label * review comments * adjust translation Co-authored-by: lmossman <lake@airbyte.io> * use request_body_json instead of request_body_data * 🪟 🎨 Move `Add` button into the line of Connector Builder key value list fields (#20699) * move add button into line * add stories for empty with control, and content + control * change button name to Control * 🪟🎉 Connector builder: Allow defining inputs (#20431) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * handle stored form values that don't contain new fields properly * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * 🪟🎉 Connector builder authentication (#20645) * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * fix keys * 🪟🎉 Connector builder: Session token and oauth authentication (#20712) * session token and oauth authentication * fill in session token variable * typos * make sure validation error does not go away * 🪟🎉 Connector builder: Always validate inputs form (#20664) * validate user input outside of form * review comments Co-authored-by: lmossman <lake@airbyte.io> Co-authored-by: lmossman <lake@airbyte.io> * fix merge conflict with dropdown prop being renamed to control * [Connector Builder] Add paginator (#20698) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * handle stored form values that don't contain new fields properly * session token and oauth authentication * fill in session token variable * fix merge of default values * add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts * add cursor and primary key fields to ui * save * add page size and token option inputs * fixes after rebase * add pagination * fix pagination types * handle empty field_name better * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * typos * make sure validation error does not go away * make primary key and cursor optional, and reorder * save toggle group progress * fix style of toggle label * handle empty values better * fix page size/token option field validation and rendering * handle cursor pagination page size option correctly Co-authored-by: Joe Reuter <joe@airbyte.io> * [Connector Builder] Add stream slicer (#20748) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * handle stored form values that don't contain new fields properly * session token and oauth authentication * fill in session token variable * fix merge of default values * add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts * add cursor and primary key fields to ui * save * add page size and token option inputs * fixes after rebase * add pagination * fix pagination types * handle empty field_name better * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * typos * make sure validation error does not go away * make primary key and cursor optional, and reorder * save toggle group progress * fix style of toggle label * handle empty values better * fix page size/token option field validation and rendering * handle cursor pagination page size option correctly * save stream slicer progress * finish stream slicer * fix stream slicer fields and validation Co-authored-by: Joe Reuter <joe@airbyte.io> * debounce form builder values update to reduce load * 🪟🔧 Connector builder: use new lowcode manifest (#20715) * use new manifest yaml * Update airbyte-webapp/src/components/connectorBuilder/types.ts Co-authored-by: Lake Mossman <lake@airbyte.io> * use updated manifest types Co-authored-by: Lake Mossman <lake@airbyte.io> * debounce validation as well * akways show stream test button in error state if there are errors * fix type of oauth input * review comments * fix more * add validation schema for add stream form * validate all views on test click * add type to prevent console warning * review comment * make sure testing state and form state stay consistent * improve builder errors * remove test state from streamconfig view * remove console log * remove unnecessary positive index check Co-authored-by: Tim Roes <tim@airbyte.io> Co-authored-by: lmossman <lake@airbyte.io> * Remove workspace helper from fetchConfigActivity (#21048) * Remove workspace helper and replace with workspaceApi * Publish new version of destination-redshift (#21083) * Update changelog * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Progress Bar Read APIs (#20937) Follow up PR to #20787 . Make stats available to the read apis so these are available to the webapp. After this, all that is left is writing these stats as the job progresses. Add the required logic in JobHistoryHandler.java. Took the chance to also rename our internal Attempt models field from id to attemptNumber to better reflect that the field stores not the row's database id, but the job's attempt number. Most of the files changes here are due to that rename. * Rm temporal version (#21045) * Rm temporal version * Remove temporal version * Update the replayed workflow * Format * Fix pmd * 🪟🎉 Connector builder: Available inputs dropdown (#20983) * improve some types * improve further * clean up a bit more * refactor loading state * move loading state up * remove isLoading references * remove unused props and make fetch connector error work * remove special component for name * remove top level state for unifinished flows * start removing uiwidget * Update airbyte-webapp/src/views/Connector/ConnectorCard/ConnectorCard.module.scss Co-authored-by: Tim Roes <tim@airbyte.io> * remove undefined option for selected id * remove unused prop * fix types * remove uiwidget state * clean up * adjust comment * handle errors in a nice way * do not respect default on oneOf fields * rename to formblock * reduce re-renders * pass error to secure inputs * simplify and improve styling * align top * code review * remove comment * review comments * rename file * be strict about boolean values * add example * track form error in error boundary * review comments * handle unexpected cases better * enrich error with connector id * 🪟🎉 Add copy stream button (#20577) * add copy stream button * review comments * rename prop * 🪟🎉 Connector builder: Integrate connector form for test input (#20385) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * fix small stuff * add warning label * review comments * adjust translation Co-authored-by: lmossman <lake@airbyte.io> * use request_body_json instead of request_body_data * 🪟 🎨 Move `Add` button into the line of Connector Builder key value list fields (#20699) * move add button into line * add stories for empty with control, and content + control * change button name to Control * 🪟🎉 Connector builder: Allow defining inputs (#20431) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * handle stored form values that don't contain new fields properly * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * 🪟🎉 Connector builder authentication (#20645) * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * fix keys * 🪟🎉 Connector builder: Session token and oauth authentication (#20712) * session token and oauth authentication * fill in session token variable * typos * make sure validation error does not go away * 🪟🎉 Connector builder: Always validate inputs form (#20664) * validate user input outside of form * review comments Co-authored-by: lmossman <lake@airbyte.io> Co-authored-by: lmossman <lake@airbyte.io> * fix merge conflict with dropdown prop being renamed to control * [Connector Builder] Add paginator (#20698) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * handle stored form values that don't contain new fields properly * session token and oauth authentication * fill in session token variable * fix merge of default values * add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts * add cursor and primary key fields to ui * save * add page size and token option inputs * fixes after rebase * add pagination * fix pagination types * handle empty field_name better * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * typos * make sure validation error does not go away * make primary key and cursor optional, and reorder * save toggle group progress * fix style of toggle label * handle empty values better * fix page size/token option field validation and rendering * handle cursor pagination page size option correctly Co-authored-by: Joe Reuter <joe@airbyte.io> * [Connector Builder] Add stream slicer (#20748) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * handle stored form values that don't contain new fields properly * session token and oauth authentication * fill in session token variable * fix merge of default values * add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts * add cursor and primary key fields to ui * save * add page size and token option inputs * fixes after rebase * add pagination * fix pagination types * handle empty field_name better * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * typos * make sure validation error does not go away * make primary key and cursor optional, and reorder * save toggle group progress * fix style of toggle label * handle empty values better * fix page size/token option field validation and rendering * handle cursor pagination page size option correctly * save stream slicer progress * finish stream slicer * fix stream slicer fields and validation Co-authored-by: Joe Reuter <joe@airbyte.io> * debounce form builder values update to reduce load * 🪟🔧 Connector builder: use new lowcode manifest (#20715) * use new manifest yaml * Update airbyte-webapp/src/components/connectorBuilder/types.ts Co-authored-by: Lake Mossman <lake@airbyte.io> * use updated manifest types Co-authored-by: Lake Mossman <lake@airbyte.io> * debounce validation as well * akways show stream test button in error state if there are errors * fix type of oauth input * available inputs dropdown * add validation schema for add stream form * validate all views on test click * add type to prevent console warning * improve styling * make sure padding is set correctly * make sure focus is set right * comment * use correct state hook * add tooltip to user input button and fix wording for new user input option Co-authored-by: Tim Roes <tim@airbyte.io> Co-authored-by: lmossman <lake@airbyte.io> * Fixed appstore docs link (#21098) * fill in all default values on switch (#21059) * 🐛Source Freshdesk: Fix schema types (#20416) * Fix schema types * Updated version * Updated acceptance tests * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * CDK: Add schema inferrer class (#20941) * fix stuff * Update schema_inferrer.py * Update schema_inferrer.py * bump version * review comments * code style * fix formatting * improve tests * Source Facebook Marketing: Docs update (#21105) * Source Amazon Ads: fix bug with handling: "Report date is too far in the past." (partial revert of #20662) (#21082) * Revert "Source Amazon Ads: fix bug with handling: "Report date is too far in the past." (#20662)" This reverts commit ec995959f7. * fix Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * revert master Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * revert to master Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * bump 0.1.27 Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * fix Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * amazon-ads.md updated Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * amazon-ads.md updated Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * auto-bump connector version Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> Co-authored-by: Augustin <augustin@airbyte.io> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Topher Lubaway <asimplechris@gmail.com> * Revert "Progress Bar Read APIs (#20937)" (#21115) Breaks when there is no config present https://github.com/airbytehq/airbyte/issues/21112 This reverts commit 3a2b0405c4. * Remove unneeded margin top and bottom (#21111) * feat(Platform): update actor configuration when receiving control messages from connectors during sync (#19811) * track latest config message * pass new config as part of outputs * persist new config * persist config as the messages come through, dont set output * clean up old implementation * accept control messages for destinations * get api client from micronaut * mask instance-wide oauth params when updating configs * defaultreplicationworker tests * formatting * tests for source/destination handlers * rm todo * refactor test a bit to fix pmd * fix pmd * fix test * add PersistConfigHelperTest * update message tracker comment * fix pmd * format * move ApiClientBeanFactory to commons-worker, use in container-orchestrator * pull out config updating to separate methods * add jitter * rename PersistConfigHelper -> UpdateConnectorConfigHelper, docs * fix exception type * fmt * move message type check into runnable * formatting * pass api client env vars to container orchestrator * pass micronaut envs to container orchestrator * print stacktrace for debugging * different api host for container orchestrator * fix default env var * format * fix errors after merge * set source and destination actor id as part of the sync input * fix: get destination definition * fix null ptr * remove "actor" from naming * fix missing change from rename * revert ContainerOrchestratorConfigBeanFactory changes * inject sourceapi/destinationapi directly rather than airbyteapiclient * UpdateConnectorConfigHelper -> ConnectorConfigUpdater * rm log * fix test * dont fail on config update error * pass id, not full config to runnables/accept control message * add new config required for api client * add test file * fix test compatibility * mount data plane credentials secret to container orchestrator (#20724) * mount data plane credentials secret to container orchestrator * rm copy-pasta * properly handle empty strings * set env vars like before * use the right config vars * 🐛Source Looker: Fix schema transformation issue (#20182) * Fix schema transformation issue * Updated PR number * Added unittest * Add a test case that uses the recursion * Unhide from cloud * Bumed seed version * [ISSUE #20322] add datetime_granularity logic to DatetimeStreamSlicer… (#20717) * [ISSUE #20322] add datetime_granularity logic to DatetimeStreamSlicer and migrate duration to ISO8601 * [ISSUE #20322] fix tests * [ISSUE #20322] code review based on clnoll's comments and fixed tests * [ISSUE #20322] fix flake8 error * [ISSUE #20322] fix source tests * [ISSUE #20322] fixing yet another error in source * [ISSUE #20322] code review * [ISSUE #20322] adding new sources using datetime slicer * [ISSUE #20322] fixing source-datascope and increasing version * [ISSUE #20322] regenerate component schema * [ISSUE #20322] fixing source-datascope * [ISSUE #20322] extra field error * Source Slack: update schema; data from openapi spec (#20767) * Source Slack: update schema; data from openapi spec * Source Slack: Docs update * Source Slack: Update schema * Source Slack: Update schema (timestamp returned in POSIX time) * Source Slack: Disable backward compatibility test for discovery * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * make sure name is rendered in first position (#21101) * Source Github: Raise Error if no organizations or repos are available (#21084) * Source Github: Raise Error if no organizations or repos are available * Source GitHub: Update docs; bump version * Source GitHub: Add test * Source GitHub: Exclude empty streams * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Convert the server to micronaut (#19194) * Extract Operation API * Extract scheduler API * Format * extract source api * Extract source definition api * Add path * Extract State API * extract webbackend api * extract webbackend api * extract workspace api * Extract source definition specification api * Remove configuration API * tmp * Checkstyle * tmp * tmp * Inject but don't resolve Bean * tmp * Tmp * fix build * TMP * Tmp * Clean up * better thread pool * Change port to 8080 * Fix port * Rm unused * Cors filter * Format * rename * Tmp * Config based * Rm health controller ref * tmp * Pool size * Mock healthcheck * Revert "Mock healthcheck" This reverts commit 466677636b. * Revert "Revert "Mock healthcheck"" This reverts commit 267094ae14. * Restore health check * Tmp * format * Rm deprecated * Fix PMD * Tmp * Fix proxy test * Remove useless annotation * set auto commit as false * Clean up and PR comments * Bmoric/convert attempt micronaut (#19847) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * Comments and banner * Non related files * rm tmp * Fix build * Format * Hit the micronaut server directly * micronaut OperationApiController (#20270) * micronaut OperationApiController * pass micronaut client to OperationApi * Bmoric/convert connection micronaut (#20211) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix build * Format * Remove media type * Format Co-authored-by: Cole Snodgrass <cole@airbyte.io> * Bmoric/convert destination controller micronaut (#20269) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * Tmp * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix bean * Add JsonSchemaValidator as a Bean * Fix build * Format * Format * Test fix * Pr comments * Remove media type * Format * Remove media type * Format * format * Add missing airbyte api client Co-authored-by: Cole Snodgrass <cole@airbyte.io> * Bmoric/convert destination definition controller micronaut (#20277) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * Tmp * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix bean * Add JsonSchemaValidator as a Bean * Fix build * Format * Format * Test fix * Pr comments * Remove media type * Format * Remove media type * Format * Remove media type * Format * api client * missing annotation * format Co-authored-by: Cole Snodgrass <cole@airbyte.io> * convert StateApiController to Micronaut (#20329) * convert to micronaut * nginx updates * format * Move dest oauth to micronaut (#20318) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix build * Format * Remove media type * Format * Move dest oauth to micronaut * Pr comments * format Co-authored-by: Cole Snodgrass <cole@airbyte.io> * Bmoric/convert source micronaut (#20334) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix build * Format * Remove media type * Format * Tmp * tmp * Build * missing bean * format Co-authored-by: Cole Snodgrass <cole@airbyte.io> * Migrate to micronaut (#20339) * Migrate source to micronaut * convert SchedulerApiController to Micronaut (#20337) * wip; SchedulerApiController * remove @Named * remove @Singleton * add back todo message * Bmoric/convert source definition micronaut (#20338) * tmp * Fix build * tmp * Tmp * tmp * tmp * Tmp * tmp * tmp * Clean up * tmp * Convert Connection Api Controller * PR Comments * convert openapiapicontroller to micronaut (#20258) * convert openapiapicontroller to micronaut * merge health/openapi locations into one entry * Fix build * Format * Remove media type * Format * Tmp * tmp * Build * missing bean * Tmp * Add Beans * fix Bean * Add passthrough * Clean up * Missing path * FIx typo * Fix conflicts * for mat Co-authored-by: Cole Snodgrass <cole@airbyte.io> * update SourceOauthApiController to Micronaut (#20386) * convert SourceOauthApiController to Micronaut * remove SourceOauthApi reference * convert WorkspaceApiController to micronaut (#20214) * wip; broken * convert WorkspaceApiController to micronaut * remove test controller * format * format * add @Body to SourceOauthApiController * consolidate nginx settings * remove unnecessary factories * Bmoric/convert jobs micronaut (#20382) * Convert jobs to micronaut * Nit * Format * Bmoric/convert source definition specification micronaut (#20379) * Migrate source definition specifications to micronaut * Format * Format * convert database assert call to Micronaut (#20406) * remove dupe config section; add DatabaseEventListener * move eventlistner to correct package; update implementation * convert NotificationsApiController to Micronaut (#20396) * convert NotificationsApiController to Micronaut * format * Migrate logs to micronaut (#20400) * Bmoric/convert webbackend micronaut (#20403) * Convert jobs to micronaut * Nit * Format * Migrate the webbackend to micronaut * Add missing Bean * Cleanup (#20459) * Cleanup * More cleanup * Disable in order to test cloud * Restore missing files * Fix test * Format and fix pmd * Add transactional * Fix version * Tentative * Cleanup the cleanup * Rm reference to the micronaut server * format * pmd * more pmd * fix build * Delete logs API * Revert "Delete logs API" This reverts commit fcb271dcdf. * Rm flaky test * Format * Try to fix test * Format * Remove optional * Rm import * Test sleep * Simplify injection * update import * Remove sleep * More injection * Remove more requirement * imports * Remove more requirement * Fix yaml * Remove unused conf * Add role * Test acceptance test * Update env * Revert "Update to Micronaut 3.8.0 (#20716)" This reverts commit a28f93747f. * Update helm chart * Fix helm chart * Convert Application Listener * Format * Add explicit deployment mode * Change check port * Update version and bump version to the right value * Cleanup * Update FE end to end test * Allow head request * Fix controller * Format * Fix http client Bean * Format Co-authored-by: Cole Snodgrass <cole@airbyte.io> * 🪟 🐛 🎨 Fix stream table icon checkboxes and icons (#21108) * Update new stream table icon positioning and style * Add custom icons from design file * Add opacity animation * Fix checkbox sizing in current streams table * Update test snapshots * Use explicit configuration for Micronaut endpoints (#20759) * process config control messages during `check` and `discover` (#20894) * track latest config message * pass new config as part of outputs * persist new config * persist config as the messages come through, dont set output * clean up old implementation * accept control messages for destinations * get api client from micronaut * mask instance-wide oauth params when updating configs * defaultreplicationworker tests * formatting * tests for source/destination handlers * rm todo * refactor test a bit to fix pmd * fix pmd * fix test * add PersistConfigHelperTest * update message tracker comment * fix pmd * format * move ApiClientBeanFactory to commons-worker, use in container-orchestrator * pull out config updating to separate methods * add jitter * rename PersistConfigHelper -> UpdateConnectorConfigHelper, docs * fix exception type * fmt * move message type check into runnable * formatting * pass api client env vars to container orchestrator * pass micronaut envs to container orchestrator * print stacktrace for debugging * different api host for container orchestrator * fix default env var * format * fix errors after merge * set source and destination actor id as part of the sync input * fix: get destination definition * fix null ptr * remove "actor" from naming * fix missing change from rename * revert ContainerOrchestratorConfigBeanFactory changes * inject sourceapi/destinationapi directly rather than airbyteapiclient * UpdateConnectorConfigHelper -> ConnectorConfigUpdater * rm log * fix test * dont fail on config update error * process control messages for discover jobs * process control messages for CHECK * persist config updates on check_connection_for_update * get last config message rather than first * fix pmd * fix failing tests * add tests * source id not required for check connection (create case) * suppress pmd warning for BusyWait literal * source id not required for checkc onnection (create case) (p2) * pass id, not full config to runnables/accept control message * add new config required for api client * add test file * remove debugging logs * rename method (getLast -> getMostRecent) * rm version check (re-added this in by mistake on merge) * fix test compatibility * simplify * 🪟🧹 Connector builder: Fast fields performance improvements (#20957) * improve some types * improve further * clean up a bit more * refactor loading state * move loading state up * remove isLoading references * remove unused props and make fetch connector error work * remove special component for name * remove top level state for unifinished flows * start removing uiwidget * Update airbyte-webapp/src/views/Connector/ConnectorCard/ConnectorCard.module.scss Co-authored-by: Tim Roes <tim@airbyte.io> * remove undefined option for selected id * remove unused prop * fix types * remove uiwidget state * clean up * adjust comment * handle errors in a nice way * do not respect default on oneOf fields * rename to formblock * reduce re-renders * pass error to secure inputs * simplify and improve styling * align top * code review * remove comment * review comments * rename file * be strict about boolean values * add example * track form error in error boundary * review comments * handle unexpected cases better * speed up some bits * more changes * enrich error with connector id * 🪟🎉 Add copy stream button (#20577) * add copy stream button * review comments * rename prop * 🪟🎉 Connector builder: Integrate connector form for test input (#20385) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * fix small stuff * add warning label * review comments * adjust translation Co-authored-by: lmossman <lake@airbyte.io> * use request_body_json instead of request_body_data * 🪟 🎨 Move `Add` button into the line of Connector Builder key value list fields (#20699) * move add button into line * add stories for empty with control, and content + control * change button name to Control * 🪟🎉 Connector builder: Allow defining inputs (#20431) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * handle stored form values that don't contain new fields properly * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * 🪟🎉 Connector builder authentication (#20645) * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * fix keys * 🪟🎉 Connector builder: Session token and oauth authentication (#20712) * session token and oauth authentication * fill in session token variable * typos * make sure validation error does not go away * 🪟🎉 Connector builder: Always validate inputs form (#20664) * validate user input outside of form * review comments Co-authored-by: lmossman <lake@airbyte.io> Co-authored-by: lmossman <lake@airbyte.io> * fix merge conflict with dropdown prop being renamed to control * [Connector Builder] Add paginator (#20698) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * handle stored form values that don't contain new fields properly * session token and oauth authentication * fill in session token variable * fix merge of default values * add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts * add cursor and primary key fields to ui * save * add page size and token option inputs * fixes after rebase * add pagination * fix pagination types * handle empty field_name better * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * typos * make sure validation error does not go away * make primary key and cursor optional, and reorder * save toggle group progress * fix style of toggle label * handle empty values better * fix page size/token option field validation and rendering * handle cursor pagination page size option correctly Co-authored-by: Joe Reuter <joe@airbyte.io> * [Connector Builder] Add stream slicer (#20748) * move connector builder components into the same shared components/connectorBuilder directory * move diff over from poc branch * save current progress * add modal for adding streams * focus stream after adding and reset button style * add reset confirm modal and select view on add * style global config and streams buttons * styling improvements * handle long stream names better * pull in connector manifest schema directly * add box shadows to resizable panels * upgrade orval and use connector manifest schema directly * remove airbyte protocol from connector builder api spec * generate python models from openapi change * fix position of yaml toggle * handle no stream case with better looking message * group global fields into single object and fix console error * confirmation modal on toggling dirty form + cleanup * fix connector name display * undo change to manifest schema * remove commented code * remove unnecessary change * fix spacing * use shadow mixin for connector img * add comment about connector img * change onSubmit to no-op * remove console log * clean up styling * simplify sidebar to remove StreamSelectButton component * swap colors of toggle * move FormikPatch to src/core/form * move types up to connectorBuilder/ level * use grid display for ui yaml toggle button * use spread instead of setting array index directly * add intl in missing places * pull connector manifest schema in through separate openapi spec * use correct intl string id * throttle setting json manifest in yaml editor * use button prop instead of manually styling * consolidate AddStreamButton styles * fix sidebar flex styles * use specific flex properties instead of flex * clean up download and reset button styles * use row-reverse for yaml editor download button * fix stream selector styles to remove margins * give connector setup guide panel same corner and shadow styles * remove blur from page display * set view to stream when selected in test panel * add placeholder when stream name is empty * switch to index-based stream selection to preserve testing panel selected stream on rename * handle empty name in stream selector * make connector form work in connector builder * wip * fix small stuff * add basic input UI * user inputs * make most of inputs configuration work * fix a bunch of stuff * handle unknown config types * add warning label * fix label * fix some styling * review comments * improve state management and error handling * allow auth configuration * check for conflicts with the inferred inputs * fix invisible inputs * handle stored form values that don't contain new fields properly * session token and oauth authentication * fill in session token variable * fix merge of default values * add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts * add cursor and primary key fields to ui * save * add page size and token option inputs * fixes after rebase * add pagination * fix pagination types * handle empty field_name better * Update airbyte-webapp/src/locales/en.json Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * inputs editing weirdness * input form reset * using the Label component * reduce redundancy and hide advanced input options for inferred inputs * unnecessary validation * typo * unnecessary effect hook * build spec even for invalid forms but do not update stream list * typos * make sure validation error does not go away * make primary key and cursor optional, and reorder * save toggle group progress * fix style of toggle label * handle empty values better * fix page size/token option field validation and rendering * handle cursor pagination page size option correctly * save stream slicer progress * finish stream slicer * fix stream slicer fields and validation Co-authored-by: Joe Reuter <joe@airbyte.io> * debounce form builder values update to reduce load * 🪟🔧 Connector builder: use new lowcode manifest (#20715) * use new manifest yaml * Update airbyte-webapp/src/components/connectorBuilder/types.ts Co-authored-by: Lake Mossman <lake@airbyte.io> * use updated manifest types Co-authored-by: Lake Mossman <lake@airbyte.io> * debounce validation as well * akways show stream test button in error state if there are errors * fix type of oauth input * review comments * fix more * start implementing fast fields * a few more fastfields * fix some stuff * complete fast fields work * memoize a bit more * remove irrelevant change * fix types * some more improvements * prevent all out-of-line validation calls * use derivedJsonManifest when in UI view Co-authored-by: Tim Roes <tim@airbyte.io> Co-authored-by: lmossman <lake@airbyte.io> * In Cloud, out of date connectors call always returns 0 (#21126) * isCloudApp returns a boolean now * Connector service never makes the check updates call in Cloud * Move hadoop-lzo to test dependency (#21085) * remove hadoop-lzo from gradle dependencies in destination-s3 project * move hadoop-lzo from build time to test time dependencies in base-s3 project * Update Connector Builder docs to reflect configuration UI additions (#21129) * update connector builder docs for UI MVP release * remove note about separately declaring schemas * Revert "Convert the server to micronaut (#19194)" (#21132) This reverts commit 0b153d11f9. * 🐛 Source Google Ads: publish missing `0.2.6` (#21103) * Remerge Progress Bar Read API. (#21124) Let's try #20937 again, this time with better test for error cases. See original PR for description. This PR adds testing and logic to handle empty/bad job input. * Bump Airbyte version from 0.40.26 to 0.40.27 (#21135) Co-authored-by: lmossman <lmossman@users.noreply.github.com> * add import rule changes for Kotlin code (#21136) * Source Okta: OAuth2.0 support - disabled (#20877) * Disabled OAuth Okta * Updated acceptance tests * Updated PR number * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Remove the quarantine status (#21088) * Rm temporal version * Remove temporal version * Update the replayed workflow * Remove quarantine information * normalization: delete supportsDbt and supportsNormalization from DestinationDefinitionSpecificationRead (#21005) * delete supportsDbt and supportsNormalization from DestinationDefinitionSpecificationRead * FE changes * add new fields to the DestinationDefinitionRead API model * update handlers * update api doc * update handlers * remove debug loggin * implement suggestions * update octavia-cli tests * fix supported check * update octavia-cli * fix integration tests * fix mocks * fix forgotten renaming Co-authored-by: Evan Tahler <evan@airbyte.io> * Allow for custom requesters to be defined in low-code manifests (#21001) * Allow for custom requesters to be defined in low-code manifests * add test for custom requester component * bump versions and changelog * adjust readme (#20945) * 🪟🎨 Connector form: Improve logs look and feel (#20951) * improve logs look and feel in connector form * add button to toggle * lokk and feel * Adjust GitHub flow for FE chapter (#20813) * Adjust GitHub flow for FE chapter * Change labeler config * Source: Google Analytics 4 (GA4) - improve config validation and SAT (#20889) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * Simplify messaging for sync canceled jobs (#20999) * Hide stats * Show cancelled icon when it's multiple attempts * Update attempts count to gray * Extract shared attempt functions into utils file * Cleanup component export and scss imports * Fix height glitch when opening and closing log * Remove deprecated getter (#21089) * Rm temporal version * Remove temporal version * Update the replayed workflow * Remove quarantine information * Remove deprecated getter * Import error broke master (#21165) * allow the cursors and primary keys to be deselected when the sync mode does not require it (#21117) * Connector builder server: Add inferred schema to read API response (#20942) * fix stuff * add inferred schema to API * fix yaml changes * fix yaml formatting * add whitespace back * reorder imports * New destination: databend (community PR #19815) (#20909) * feat: Add databend destination Co-authored-by: hantmac <hantmac@outlook.com> Co-authored-by: josephkmh <joseph@airbyte.io> Co-authored-by: Sajarin <sajarindider@gmail.com> * Update postgres.md (#21170) * Source Facebook Marketing: Update schema ad_account (#21149) * Source Facebook Marketing: Update schema ad_account * Source Facebook Marketing: Update docs; bump version * Source Facebook Marketing: disable backward compatibility * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Revert "Revert Convert the server to micronaut" (#21133) * Revert "Revert "Convert the server to micronaut (#19194)" (#21132)" This reverts commit 31c65f8eea. * Fix the cors * Fix cloud * Amazon Seller Partner: Validate connections without orders data (#20896) * amazon seller partner validate connection without orders data * amazon seller partner bump version in docs * amazon seller partner fix redundant if statement * auto-bump connector version Co-authored-by: Sajarin <sajarindider@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Source Harvest: Skip 403 FORBIDDEN for all streams (#21151) * Source Harvest: Skip 403 FORBIDDEN for all streams * Source Harvest: update docs * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * test for behavior when a column is removed in an incremental sync (#20033) * test for behavior when a column is removed in an incremental sync * fixes in dat test for dropping a column * only run drop-one-column test for the exchange rates dataset * re-enable tests that were disabled during development * remove unused import * update test to new method for checking spec capabilities * use config directly instead of parameterized test Co-authored-by: Michael Siega <michael@airbyte.io> Co-authored-by: Michael Siega <109092231+mfsiega-airbyte@users.noreply.github.com> * Source Mailchimp - fix the way request params are built (#20765) * 1058 source mailchimp - fix the way request params are built * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🐛 🎉 Source Airtable: migrate to the `Metadata API` for dynamic schema generation (#20846) * Extend connection list filtering (#21094) * Update listWorkspaceStandardSyncs to support lists of sources/destinations * Update API spec * test cleanup (#21178) * removed error (#20612) * Adjust connector icons (#20547) * Adjust connector icons * Update icons * Add missing icons * Remove missing icon from spec * Add trailing newline * Fix test * Use jest directly as test runner (#21174) * 🪟 🎨 Refining BulkEditPanel component (#20474) * Adds handling for source defined and unavailable stream paths inside PillSelect * Adds handling of unavailable cursor and primary key field for PillButton components * Adds useMemo to SyncPathSelect component * Add InfoText component for cursor field and primary key to match table design in Figma * Removes animation for BulkEditPanel; Adds StreamPathSelect new property pillIfChangeUnavailable for cases when we need or not to render pill under unavailable to change path text * Disables pointer event while PillButton is disabled * Adds renderDisabledState property to control PillSelect content while disabled * Changes renderDisabledState -> disabledLabel * 🪟 🔧 Add tests for BulkEditService (#20820) * Add tests for BulkEditService * Add act function to test cases while updating state * Add tests full description; Adds type to MOCK_NODES array; Fixes import relative path * Connector builder: E2e tests (#21122) * wip * wip * e2e tests for connector builder server * rename function * clean up * clean up a bit more * fix path * fix and add documentation * more documentation * stabilze * review comments * 🎉 Source Airtable: update `releaseStage` to `beta` (#20935) * 🪟 🎉 Disable deselection of cursor field/primary key in the UI (#20844) * 🪟 🔧 Update frontend typing of cloud api response (#21086) * 🪟 🔧 Use CSS Custom Properties for colors (#19344) * use css custom properties for colors * separate tokens by hue * adjust stylelint * fix code editor color transformations * remove blue-transparent from color palette * scope opacity to bg only * rename to theme-light * avoid transparent blue, use palette color instead * bring back expanding hex values * replace tooltip colors * define overlay color in theme * fix tooltip colors * slightly darker link color * remove unused SC colors * add box shadow variables * light tooltip table variant * inline one-off colors for login * add new colors to theme-light * fix import name * fix merge conflict with box shadows * fix overlay bg color Co-authored-by: lmossman <lake@airbyte.io> * ci-connector-ops: change required reviewers logic (#21158) * [ISSUE-20322] updating tutorial documentation (#21163) * [ISSUE-20322] updating tutorial documentation * [ISSUE-20322] updating test and improving docs * Connector builder server: Fix unit tests (#21107) * fix stuff * add inferred schema to API * fix yaml changes * fix yaml formatting * add whitespace back * fix tests * reformat * add unit tests * Destination connector for teradata vantage (rebased community PR) (#21160) * Destination connector for teradata vantage is implemented * add teradata destination to seed yaml * Add teradata destination to specs seed file Co-authored-by: Satish Chinthanippu <satishchinthanippu@gmail.com> * 🎉 Updates for edit password field in connector (#20723) * Moves Cancel button to the left. * changes Edit button to outline - secondary variant. * Updates button size to match input height. * convert EnvVariableFeatureFlag to an injected dependency (#21171) * inject feature-flag client into integration-launcher * inject feature-flag client into message-tracker * inject feature-flag client into DefaultAirbyteSource * missed reference in message-tracker * Updated ISO certification date (#21181) * added info * edited wording * Source google ads: add more logs (#20755) * #1148 source google ads: add more logs * #1148 source google ads: upd changelog * #1148 source google ads: flake fix * #1148 source google ads - fix SATs * #1148 source Google Ads: bump version * #1148 source google ads: upd expected records * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 Destination Local CSV: add custom delimiter (#17998) * Adds delimeter to the spec file. Adds function to get delimeter from the config. * New delimiter works, but only checked raw airbyte tables. * Fixes testWriteFailure(), testWriteSuccess() still broken. * Corrects CSVFormat and now all tests pass. * Implements tab separator. * Corrects tooltip on destination settings page. * Creates CSV file with delimiters and prints it as stirng. * Adds try catch block for assertion. Deletes file after test run. * Removes separate format for tab dleimiter, it is not needed. * Cleans up code. * Adds missing bracket. * Adds files from incorrect rebase. * Corrects imports. * Fixes connectors base build. * Corrects Dockerfile version bump. Adds changelog. * Corrects getProtocolVersion method and makes CSVDataArgumentsProvider class static. * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Source Airtable: fix field names (#21215) * Source S3: update block size for json (#21210) * Source S3: update block size for json * Source S3: update docs * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🪟 🧹 Custom connectors in Cloud UI updates (#21034) * check if doc url is a valid url in cloud, but don't require it * field label and placeholder * cleanup * match casing for other form labels * cleanup imports * hide documentation panel if URL is empty * only show documentation panel if url is from our docs * make docs url optional, don't put a link in conncectorsview if empty * Update airbyte-webapp/src/views/Connector/ConnectorDocumentationLayout/ConnectorDocumentationLayout.tsx Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com> * use <FormattedMessage />, cleanup from Joey's commit in review comment * Use <Text/> component in ImageCell, get styled components out of there while I'm at it * add "required" labels * show validation error regardless of `touched` for documentationUrl * use old placeholder instead Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com> * lowcode docs: add link to YAML reference (#21231) * Update platform-workflow-labels.yml (#21225) * Update README.md (#21236) Update connector release stage docs * Enforce HTTPS (#21182) * Enforce HTTPS * 🪟🐛 Connector builder UI: Fix datetime stream slicer (#21161) * fix datetime stream slicer * Update airbyte-webapp/src/components/connectorBuilder/Builder/StreamSlicerSection.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> * Update airbyte-webapp/src/components/connectorBuilder/Builder/StreamSlicerSection.tsx Co-authored-by: Lake Mossman <lake@airbyte.io> Co-authored-by: Lake Mossman <lake@airbyte.io> * [ISSUE #19981] testing version bump (#21106) * [ISSUE #19981] testing version bump * [ISSUE #19981] debugging part-to-bump * [ISSUE #19981] further debugging on part-to-bump * [ISSUE #19981] further debugging on part-to-bump yet again * [ISSUE #19981] further debugging on part-to-bump yet again and again * [ISSUE #19981] improving on first successful attempt * [ISSUE #19981] adding changelog and concurrency * [ISSUE #19981] update version in .bumpversion.cfg * [ISSUE #19981] testing Slack notification with working channel * [ISSUE #19981] documentation and clean up before PR * [ISSUE #19981] make changelog-message optional * [ISSUE #19981] align version with new release * [ISSUE #19981] code review * [Low-Code CDK] Handle forward references in manifest (#20893) [Low-Code CDK] Handle forward references in manifest * Update form field labels in connection form to match design (#21036) * Update form field labels in connection form to match design - Replaced ControlLabels message property with infoTooltipContent - Replaced property "flex-start" to "center" for FlexContainer (New Connection) * Hide no credits banner for no billing accounts (#21218) * Fix auto detect schema change backdrop color CSS (#21246) * Source notion: fix schema (#20639) * #1047 oncall - Source Notion: fix Pages stream schema * #1047 oncall - Source Notion: fix Pages stream schema * #1047 oncall - Source Notion: upd changelog * #1047 source notion - remove ignored fields * #1047 source notion: fix SATs * #1047 source notion: upd expected records * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * [Connector-builder server] Allow client to specify record limit and enforce max of 1000 (#20575) * Upgrade to Micronaut 3.8.0 and related dependencies (#21077) * Upgrade to Micronaut 3.8.0 and related dependencies * Define hostname/subdomain to be compatible with updated AWS SDK * Use bucket name for hostname * Pin dependencies to avoid behavior change * Add comment * Source Google Analytics Data API: slicer updated, unit tests added (#21169) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * Bmoric/restore old interface (#21235) * Re introduce old exception mapper * Use mapper * Switch CI to use npm ci (#21259) * postgres-source-cdc: handle null values for array data types (#21003) * postgres-source-cdc:handle null values for array data types * fix test * upgrade version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🪟 🐛 Fix broken switch UI state - when checked state provided as a "value" (#21219) * fix broken switch state - when checked state provided as a "value" * Update airbyte-webapp/src/components/ui/Switch/Switch.tsx Co-authored-by: Krishna (kc) Glick <krishna@airbyte.io> Co-authored-by: Krishna (kc) Glick <krishna@airbyte.io> * 🪟 🔧 Use experiment for new streams table design (#21230) * Use experiment for new streams table design * Update formConfig validationSchema to use a hook to get the feature flags directly * Add useNewTableDesignExperiment helper hook * Rename isNewStreamsTableEnabled -> isNewTableDesignEnabld * Cleanup tidyFormValues to take the existing schema in * Fix var name in CatalogTreeTableHeader * Source Snapchat Marketing: fix error response (#21267) * Source Snapchat Marketing: fix error response * Source Snapchat Marketing: update docs * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * [Low-Code CDK] Parse incoming manifest objects into Pydantic models (#20747) * initial work to parse manifest objects into pydantic models * pr feedback and some other validations * rerun manifest schema generation * remove field constraint due to bug * add a few tests for the model to component factory * add catch * fix a bug where propagated schema doesn't work with old factory * add additionaProperties: false to the handwritten schema * regenerate pydantic models * fix test * [Low Code CDK] Add type annotations and cleanup usage of `visited` set (#21268) * [Low Code CDK] Add type annotations and cleanup usage of `visited` set * 🤖 Bump minor version of Airbyte CDK * feat: mention user on failed master branch build (#21201) * 🪟 🔧 Fixed passed props to `<CheckBoxControl />` which cause error in console (#20711) * fixed sourceDefinitionId and destinationDefinitionId * clean up props * Connector builder server: Fail on failing tests (#21198) * fail on failing command * fix test * Source Google Ads: Update docs (#21308) * Source Ads: docs * Source Ads: docs * Migrate airbyte-bootloader to Micronaut (#21073) * Migrate airbyte-bootloader to Micronaut * PR feedback * More PR feedback * Rename variable for clarity * Add properties to support cloud * Formatting * Use default values for env vars * Re-organization to support cloud overrides * Disable conditional logging * More singleton cleanup * test: try CI without fluentbit * Revert "test: try CI without fluentbit" This reverts commit 8fa0f74106. * test: enable SSH on EC2 runner * Revert "test: enable SSH on EC2 runner" This reverts commit e4867aae09. * Avoid early database connection on startup * Fix compile issues from refactor * Formatting Co-authored-by: perangel <perangel@gmail.com> * 🪟 🧹 Reorganize connections pages (#20845) * Reorganize connection pages Flatten structure Simplify names * Consolidate all the Connection routes to a single file * Extract job types and utilities to own files within JobItem Move JobsWithJobs to JobItem/utils * Move ConnectionName component to components/connection * Move StatusMainInfo components and rename to components/connection/ConnectionInfoCard * Move ConnectionBlock to components/connection * Clean up ConnectionPage structure * Update style import in ConnectionInfoCard test * Clean up ConnectionRoutePaths enum * Apply suggestions from code review - Update export default style Co-authored-by: Krishna (kc) Glick <krishna@airbyte.io> * Update ConnectionInfoCard test description name * Update test snapshots Co-authored-by: Krishna (kc) Glick <krishna@airbyte.io> * Add bootloader/server to published images for Cloud (#21325) * 🪟 🚦 E2E: Fix duplicated database object warning (#20926) * replace "cypress-postgres" with "pg-promise" npm package * update cypress config * update userDefineConnection type and naming * Update airbyte-webapp-e2e-tests/cypress/plugins/index.ts Co-authored-by: Mark Berger <mark.berger@globallogic.com> Co-authored-by: Mark Berger <mark.berger@globallogic.com> * Run Airbyte CI on pull_request (#21266) * Run Airbyte CI on pull_request as well * also only run on pushes to master * 🎉 Source bigcommerce: add brands and categories streams (#20518) * Add product info to the orders stream * Update docs * Refactor order_products into its own stream * formatting * Update docs and docker version * remove duplicate field * Add brands and categories streams to bigcommerce * remove duplicate state * remove duplicate source * update docs * update docs * bump docker version * update markdown * lint * auto-bump connector version Co-authored-by: Sajarin <sajarindider@gmail.com> Co-authored-by: Haithem SOUALA <haithem.souala@woopit.fr> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 New Destination: Weaviate (#20094) * Add Weaviate Destination #20012 * Fix formatting and standards * Fix flake issue * Fix unused client variable * Add support for int based ID fields * Ensure stream name meets Weaviate class reqs * add integration test for using pokemon as source * handle nested objects by converting to json string * create schema for transforming data to weaviate * Add docs for weaviate destination * Remove pokemon-schema external dependency * Remove pikachu integration test external dep * Add large batch test case * add test for second sync * Fix issue with fields starting with uppercase * add more checks to line_break test * Update README for Weaviate * Make batch_size configurable with 100 as default * Add support for providing vectors * Update docs * Add test for existing Weaviate class * Add trying to create schema in check connection * Add support for mongodb _id fields * Add support for providing custom ID * remove unused file * fix flow of is_ready() check * Move standalone functions to utils.py * Support overwrite mode * Add regex based stream_name_class_name conversion * remove unneeded print statement * Add "airbyte_secret" : true to password config * add support for array of arrays * remove unneeded variable declaration * change to MutableMapping since we use del * change name from queued_write to buffered_write * add retry on partial batch error * Fix partial batch retry and add tests * fix ID generation * Clean up recursive retry logic * fix flake tests * ran flake reformat * add definitions Co-authored-by: Ivica Taseski <ivica.taseski94@gmail.com> Co-authored-by: itaseski <itaseski@debian-BULLSEYE-live-builder-AMD64> * Make schema field in source-snowflake mean a subset of the specified o… (#20465) * Make schema field in source-postgres mean a subset of the specified of schema when during discover(). update UI * Add missing file * Fix failing acceptance test * sanity * update doc * typo * version bump and release note * Fix failing test * fix format * Ensure Local Secret Persistence is Initialized (#21329) * Formatting * Ensure that local persistence is initialized * December release notes docs (#21270) * added december release notes * edits * Add dependencies to docker-compose (#19321) * Revert "Revert "Add dependencies to docker-compose (#19257)" (#19306)" This reverts commit b16f28fd4d. * Bump docker-compose version * test * Revert "test" This reverts commit c3b28c9ec3. * Bump docker compose version to 3.8 * test * test * Use a newer docker for platform build * test * Use docker compose v2 * Update docker for frontend e2e * Use docker compose v2 for e2e tests * Update tools to use docker compose v2 * Update more docker compose references to v2 * Update docker compose usage in docs * Update deploying airbyte docs * Fix permission * Case insensistive check * Add extra mention for docker compose in upgrading airbyte * Improve upgrade docker note * store env-var response (#21327) * store env-var response * fix formatting * fix pmd issue * change log message to debug * fix file change error * Better escape the select query for a case a schema name starts with a number (#21051) * Better escape the select query for a case a schema name starts with a numebr. * improve test * fix unrelated build error * manually bump source definition version (#21349) * 🐛 Source Jira: `check_connection` fails if no projects are defined (#20739) Co-authored-by: Evan Tahler <evan@airbyte.io> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com> Co-authored-by: Augustin <augustin@airbyte.io> * airbyte-common-workers: Collect trace message on failed connection_status (#20721) * 🪟 🎉 Verify auth status on tab focus (#21175) * Make documentation_url optional in a declarative connector spec (#21347) * Make documentation_url optional in a declarative connector spec * simplify if statement * bump patch version of cdk * Revert "bump patch version of cdk" This reverts commit 1854bf3be1. * 🤖 Bump patch version of Airbyte CDK * Revert "🤖 Bump patch version of Airbyte CDK" This reverts commit 85d5a989e2. * 🤖 Bump patch version of Airbyte CDK Co-authored-by: lmossman <lmossman@users.noreply.github.com> * Avoid parsing a message if this message is too big (#21090) * Tmp * Format * TMP * TMP * Inject max memory * Clean up * Improve error message * PR comments * Unrelated changes * Fix pmd * JDBC Destinations: improve error message for conflicting streams (#21342) * catch conflicting streams as configerror * add test * bump version + changelog * derp, fix test setup * derp * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Bump Airbyte version from 0.40.27 to 0.40.28 (#21359) Co-authored-by: lmossman <lmossman@users.noreply.github.com> * Add records committed to job stats (#21364) * [Low-Code CDK] Construct declarative components from Pydantic models (#21050) * initial work to parse manifest objects into pydantic models * pr feedback and some other validations * rerun manifest schema generation * remove field constraint due to bug * initial work to construct most components from greenhouse * custom components parse subcomponent fields correctly and adding a few more component constructors * construct components from gnews * first pass at posthog.yaml * Handle nested custom components with list values. Also includes updates to posthog.yaml, including autoformatting changes. * adding constructors for slicers, filters, and transformations and a few bug fixes * make sed work across multiple OS * add NoAuth component * fix handling of custom components with nested list * Autogenerate `TYPE_NAME_TO_MODEL` mapping * Handle default kwargs not defined on model for custom components * Re-add `options` for CartesianProductStreamSlicer for backwards compat with custom stream slicers * add basic unit tests for the model component factory * add back defaults and extra parameters like options to retain compatibility with legacy flow and backwards compatibility * Remove `_get_defaults`; using actual default values on classes instead * Add backoff strategy component creation functions * add back defaults and extra parameters like options to retain compatibility with legacy flow and backwards compatibility * add lots of tests to construct components from the pydantic models and a few bug fixes * add a few tests for the model to component factory * add catch * fix a bug where propagated schema doesn't work with old factory * clean up a few files * add type inference for custom components, more tests and some refactoring of the model factory * template, docs, manifest updates, pr feedback and some cleanup * pr feedback and polish schema a bit * fix tests from the latest rebase of master * fix the last few bugs I found and adjust a few sources that weren't perfectly compatible with the new component flow * fix CheckStream bug cleanup and a few small tweaks and polish * add additional test to cover bug case * fix formatting * 🤖 Bump minor version of Airbyte CDK Co-authored-by: Catherine Noll <noll.catherine@gmail.com> Co-authored-by: brianjlai <brianjlai@users.noreply.github.com> * update destination-python generator (#21298) * fixed current_value in usage_triggers stream (#21304) * 🪟 🔧 Add stylelint plugin & no-color-variables-in-rgba rule (#21269) * fix e2e tests (#21380) * 🪟🎉 Connector builder: Schema inferrer UI (#21154) * fix stuff * add inferred schema to API * fix yaml changes * fix yaml formatting * add whitespace back * basic ui * advanced UI * Remove unused one * reset package lock * resolve merge conflicts * styling * show button and icon in the normal schema tab * restructure * handle yaml view * small fix * review comments * make monaco resize * review comments * 🎉 New Source: Dremio (#19912) * Remove `supportsNormalization` `supportsDbt` from non GA destination spec (#21319) * SAT: check `future_state` only for incremental streams. (#21248) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * 🎉 Typeform - beta certification (#21145) * fixed SAT, fix specs, updated docs * fix incremental * fix incremental * fix incremental * updated connector version in source_definitions.yaml * updated version in source_specs.yaml * Update databend.md (#21179) * Update databend.md * <Callout /> component (#21273) * genericize alert component * add className prop to alert * rename infobox "callout", swap in place for scss module * fix imports * use callout in place of alert * add blue variant, remove icon prop * remove font-size (this is already the default) * remove single use prop, use a classname instead * use semantic names, remove empty line * cleanup from rebase * Update to Micronaut 3.8.1 (#21388) * Source Slack: retry timeout error (#21321) * Source Slack: retry timeout error * Source Slack: update docs * Source Slack: update retry * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Source Facebook Marketing: Videos stream remove filtering (#21385) * Source Facebook Marketing: Videos stream remove filtering * Source Facebook Marketing: update docs * Source Facebook Marketing: Ref * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * Source Klaviyo: enable SAT high strictness (#21374) * Source Klaviyo: change configuration of acceptance-test-config.yml for high strictness * Add metrics stream to configured_catalog, add metrics expected records * Update future state * Remove expected records for flows stream * Source File: enable high strictness level (#21243) * Source File: enable high strictness level * Addedend line Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com> * introduce code coverage report step (#21376) * introduce code coverage report step * remove airbyte-queue jacoco report since it isn't being generated by the build yet * leave pointer to wildcard support and tweak coverage threshold * added edits (#21346) * 🪟 🐛 Fix jest tests to fail build correctly (#21408) * Fix jest tests to fail build correctly * Add failing test for demo * Update StatusCell.test.tsx * Allow the OSS server only on OSS (#21416) * Update create connection page to full width, and fix trivial UI issues (#21352) * Update big prop in FormPageContent to expand to full width * Fix Normalization & Tranformation heading size * Update FormPageContent to scss and add bottom padding * Remove max-width from ConnectionReplicationPage * Improvements to edge cases of CheckStream (#21404) * Add test for failure case * Except StopIteration - make test pass * Don't attempt to connect to a stream if we get no stream slices * Make helper method for getting first record for a slice * Add comments and exit early if stream to check isn't in list of source streams * move helpers to helper module * Clarify what it means when StopIteration is returned by helper methods * Add AllowedHosts to Actor Definitions and Config Database (#21363) * Add AllowedHosts to actor_definitions and database * use objects for better null-ness handling * Tables.ACTOR_DEFINITIO * Update on-gcp-compute-engine.md (#21429) Use curl instead of wget since we already install curl. * refactor checks * rebase from master * PR Feedback * PR feedback * revert unintended version downgrade * add version updates and changelog * auto-bump connector version * auto-bump connector version * formatting * bump snowflake destination version * manual version bump for s3 * auto-bump connector version * auto-bump connector version Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> Co-authored-by: Joe Reuter <joe@airbyte.io> Co-authored-by: Tim Roes <tim@airbyte.io> Co-authored-by: lmossman <lake@airbyte.io> Co-authored-by: Anne <102554163+alovew@users.noreply.github.com> Co-authored-by: Greg Solovyev <grishick@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Davin Chia <davinchia@gmail.com> Co-authored-by: Benoit Moriceau <benoit@airbyte.io> Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com> Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com> Co-authored-by: Artem Inzhyyants <36314070+artem1205@users.noreply.github.com> Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com> Co-authored-by: Augustin <augustin@airbyte.io> Co-authored-by: Topher Lubaway <asimplechris@gmail.com> Co-authored-by: Krishna (kc) Glick <krishna@airbyte.io> Co-authored-by: Pedro S. Lopez <pedroslopez@me.com> Co-authored-by: Maxime Carbonneau-Leclerc <maxi297@users.noreply.github.com> Co-authored-by: Cole Snodgrass <cole@airbyte.io> Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com> Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io> Co-authored-by: Baz <oleksandr.bazarnov@globallogic.com> Co-authored-by: Octavia Squidington III <90398440+octavia-squidington-iii@users.noreply.github.com> Co-authored-by: lmossman <lmossman@users.noreply.github.com> Co-authored-by: Evan Tahler <evan@airbyte.io> Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com> Co-authored-by: Michael Siega <109092231+mfsiega-airbyte@users.noreply.github.com> Co-authored-by: hantmac <hantmac@outlook.com> Co-authored-by: josephkmh <joseph@airbyte.io> Co-authored-by: Sajarin <sajarindider@gmail.com> Co-authored-by: Zaza Javakhishvili <zaza.javakhishvili@gmail.com> Co-authored-by: Michael Siega <michael@airbyte.io> Co-authored-by: Denys Davydov <davydov.den18@gmail.com> Co-authored-by: Jimmy Ma <gosusnp@users.noreply.github.com> Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com> Co-authored-by: Yatsuk Bogdan <yatsukbogdan@gmail.com> Co-authored-by: Joey Marshment-Howell <josephkmh@users.noreply.github.com> Co-authored-by: Satish Chinthanippu <satishchinthanippu@gmail.com> Co-authored-by: Nataly Merezhuk <65251165+natalyjazzviolin@users.noreply.github.com> Co-authored-by: Teal Larson <LARSON.TEAL@GMAIL.COM> Co-authored-by: Sherif A. Nada <snadalive@gmail.com> Co-authored-by: Catherine Noll <clnoll@users.noreply.github.com> Co-authored-by: Mark Berger <mark.berger@globallogic.com> Co-authored-by: Subodh Kant Chaturvedi <subodh1810@gmail.com> Co-authored-by: Vladimir <volodymyr.s.petrov@globallogic.com> Co-authored-by: Volodymyr Pochtar <v.pochtar@gmail.com> Co-authored-by: perangel <perangel@gmail.com> Co-authored-by: Conor <cpdeethree@users.noreply.github.com> Co-authored-by: Philip Corr <PhilipCorr@users.noreply.github.com> Co-authored-by: Haithem SOUALA <haithem.souala@woopit.fr> Co-authored-by: Sam Stoelinga <sammiestoel@gmail.com> Co-authored-by: Ivica Taseski <ivica.taseski94@gmail.com> Co-authored-by: itaseski <itaseski@debian-BULLSEYE-live-builder-AMD64> Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com> Co-authored-by: Philippe Boyd <philippeboyd@users.noreply.github.com> Co-authored-by: Edward Gao <edward.gao@airbyte.io> Co-authored-by: Catherine Noll <noll.catherine@gmail.com> Co-authored-by: brianjlai <brianjlai@users.noreply.github.com> Co-authored-by: darynaishchenko <80129833+darynaishchenko@users.noreply.github.com> Co-authored-by: Arsen Losenko <20901439+arsenlosenko@users.noreply.github.com> Co-authored-by: midavadim <midavadim@yahoo.com> Co-authored-by: Roman Yermilov [GL] <86300758+roman-yermilov-gl@users.noreply.github.com> Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com> Co-authored-by: Ella Rohm-Ensing <erohmensing@gmail.com>	2023-01-23 09:12:27 -08:00
Eugene	8ed229ffb6	🐛Destination-Bigquery: Added an explicit error message if sync fails due to a config issue (#21144) ... * [19998] Destination-Bigquery: Added an explicit error message in sync fails due to a config issue	2023-01-18 22:54:46 +00:00
Rodi Reich Zilberman	6a68d1f4a7	Map number(integer) into an integer rather than a float (#20730) ... * Fix failing test * Bigquery-denormalized update spec plus destination_definitions.yaml to fix integration tests * sanity * Update version number and release note * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2023-01-04 15:27:25 +05:30
Greg Solovyev	a12ca28d72	Do not swallow GCS failure when checking BigQuery config (#20501) ... * Do not swallow GCS failure when checking BigQuery config * bump version of BigQuery denormalized * add changelog * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-12-15 12:56:47 -08:00
Eugene	be530bbf2e	[18713] Destination-bigquery: updated Check method to test for non-billable project (#19489) ... * [18713] Destination-bigquery: updated Check method to test for non-billable project * fixed typo in comments * Bumped version * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-11-28 09:36:03 -08:00
Eugene	6640a47c02	[888]Destination-Bigquery: fixed check method to capture mismatch dataset location (#19358) ... * [888]Destination-Bigquery: fixed check method to capture mismatch dataset location	2022-11-11 23:07:03 +02:00
Eugene	df2b061095	🐛Destination-bigquery: added additional verifications to Check method (#18554) ... * [16992] Destination-bigquery: added additional verifications to Check connection method	2022-11-10 15:44:40 +02:00
Edward Gao	85e93cb006	Destinations BigQuery and GCS: update docs about bucket encryption (#18315)	2022-10-21 13:34:48 -07:00
Edward Gao	e21d54fbab	Destination Bigquery: log the full stacktrace rather than just message (#18162) ... * log the full stacktrace rather than just message * version bumps + changelog * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-10-19 09:23:49 -07:00
Yurii Bidiuk	e07bd6bee3	Destination BigQuery: add user-agent header (#16890) ... * Destiantion BigQuery: add user-agent header * fixed comments * fix tests * refactor code * refactoring * bump version * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-09-26 20:31:27 +03:00
Edward Gao	23c8efa2b2	Destination Bigquery / Bigquery-denormalized: Only override dataset ID if stream namespace is null/empty (#17054) ... * stop overriding namespace? * set namespace if needed * also check for empty namespace * version bump + changelog * auto-bump connector version [ci skip] * sanitize dataset id * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-09-22 12:41:47 -07:00
Yurii Bidiuk	10e4aae891	Use LOG4J2 to wrap connectors logs to JSON format (#15668) ... * Use LOG4J2 to wrap connectors logs to JSON format * log connector's stack trace directly as a message * add stack_trace field to json template * bump versions * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * revert versions for destinations: postgres, mssql, mysql Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-09-14 23:27:13 +03:00
VitaliiMaltsev	022b8db345	Destination: BigQuery (denormalized typed struct) - Could not evaluate union, field is expected to be one of these: NULL, STRING. (#16401) ... * Fixed bucket naming for S3 * removed redundant configs * BigQuery Denormalized: Could not evaluate union on Mongo formatted type * add test * fixed BigQuery tests * updated changelog * replaced star import * bump version * auto-bump connector version [ci skip] * fixed compilation error * bump mongodb-strict-encrypt * auto-bump connector version [ci skip] * removed unused method Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-09-13 19:10:09 +03:00
Andrii Leonets	40c84476d7	Destination BigQuery : Increase version for BQ PR #14023 (#16494) ... * Increase version for BQ PR #14023 * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-09-08 19:03:27 -07:00
Liren Tu	fd547c64d7	🐛 Avro related destinations: fix field name clashing with combined restrict (#16243) ... * Add test case for allOf and oneOf * Bump version * Add pr id * Bump gcs version * Bump dest jdbc * Bump redshift * Bump snowflake * Bump databricks * Bump bigquery * Revert "Bump dest jdbc" This reverts commit f10497e96a. * Use a new pat to avoid api rate limit * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * Revert databricks bump * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-09-01 18:06:27 -07:00
Jagruti Tiwari	8288c16485	fix: replace airbyte oss with airbyte open source (#15885) ... Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>	2022-08-24 01:01:53 -03:00
VitaliiMaltsev	3aefd3308f	Bigquery Destinatioin throw exception if job failed (#15787) ... * Fixed bucket naming for S3 * removed redundant configs * BigQuery Destination throw exception if job failed * bump version * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-08-23 12:46:27 +03:00
Andy Jih	e8a0d5c851	Docs updates for destinations affected by per-stream state (#15578) ... * Docs updates for destinations affected by per-stream state * adding -alpha tag	2022-08-11 15:37:22 -07:00
Liren Tu	69972d606d	Bigquery destination: fix changelog and publish new version (#15409) ... * Fix bigquery changelog * Bump bigquery denormalized * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-08-08 11:05:23 -07:00
Greg Solovyev	95aeb39325	Fix typo in change log (#15343) ... * Fix typo in several change logs Co-authored-by: andrii.leonets <aleonets@gmail.com>	2022-08-05 09:13:24 -07:00
Marcelo Castro	76c4053282	Destination BigQuery: Enabling Application Default Credentials (#14784) ... * Enabling different bigquery authentications Current implementation only accepts Service Accounts. For developers it is very desirable that we can login with Application Default Credentials (ADC) https://cloud.google.com/sdk/gcloud/reference/auth/application-default * bump version and update doc * auto-bump connector version [ci skip] Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-08-03 15:18:34 -03:00
Andrii Leonets	e7b9e00949	Fix multiply log bindings (#14801) ... * Fix multiply log bindings * Exclude slf4j-reload4j * Exclude slf4j-log4j12 for debezium * Increase version for debezium related sources and json converter related destinations * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] * rebump bigquery versions * mark destinations s3, gcs as unpublishd * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] Co-authored-by: subodh <subodh1810@gmail.com> Co-authored-by: Greg Solovyev <grishick@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Edward Gao <edward.gao@airbyte.io>	2022-08-03 09:36:30 -07:00
Amruta Ranade	cc7e850a42	Update bigquery.md	2022-08-03 12:08:53 -04:00
Benoit Moriceau	8bf5986a0b	Fix bq standard mode (#15180) ... * Fix * Add version and changelog * auto-bump connector version [ci skip] * auto-bump connector version [ci skip] Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-08-02 10:00:12 -07:00
Topher Lubaway	9c6c092a22	Revert "Improving docusaurus sidebar generation (#1927) (#14369)" (#14596) ... This reverts commit a2c194a11f.	2022-07-11 15:27:14 -05:00
Mykyta Serbynevskiy	a2c194a11f	Improving docusaurus sidebar generation (#1927) (#14369) ... * Improving docusaurus sidebar generation (#1927) * Added "Career & open positions" folder to sidebar, adjusted "Project overview" folder * Deleted "career-and-open-positions" folder from sidebar	2022-07-08 14:18:27 -05:00
VitaliiMaltsev	d3f4fdd8fc	BigQuery Denormalized "airbyte_type": "big_integer" to INT64 (#14079) ... * BigQuery Denormalized "airbyte_type": "big_integer" to INT64 * updated changelog * added unit test * removed star import * fixed checkstyle * bump version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-06-29 19:47:15 +03:00
VitaliiMaltsev	11ea066f47	Remove "additionalProperties": false from specs for connectors with staging (#14114) ... * Remove "additionalProperties": false from spec for connectors with staging * Remove "additionalProperties": false from spec for Redshift destination * bump versions * auto-bump connector version * auto-bump connector version * auto-bump connector version * auto-bump connector version * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-06-24 20:04:04 +03:00
VitaliiMaltsev	c283d9d159	Deprecate PART_SIZE_MB in connectors using S3/GCS storage (#13753) ... * Removed part_size from connectors that use StreamTransferManager * fixed S3DestinationConfigTest * fixed S3JsonlFormatConfigTest * upadate changelog and bump version * auto-bump connector version * auto-bump connector version * auto-bump connector version * auto-bump connector version * upadate changelog and bump version for Redshift and Snowflake destinations * auto-bump connector version * fix GCS staging test * fix GCS staging test * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-06-20 14:56:33 +03:00
George Claireaux	e63458fabb	Publishing java connectors with better formatted stacktrace in Trace messages (#13852) ... * start * revert * azblob * bq * bq denorm * megapublish baaaabyyyy * fix needs * matrix connectors * auto-bump connector version * dont failfast and max parallel 5 * multi runno * minor * testing matrix agents * name * testing multi agents * tmp fix * new multi agents * multi test * tryy * let's do this * magico * fix * label test * couple more connector bumps * temp * things * check this * lets gooo * more connectors * Fix md and lg spacing scss vars (#13886) * Delete TEMP-testing-command.yml * auto-bump connector version * Delete publish-command.yml * added comment describing bash part * running single thread * catch sentry cli * auto-bump connector version * Update _variables.scss Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Edmundo Ruiz Ghanem <168664+edmundito@users.noreply.github.com>	2022-06-17 16:15:24 +01:00
Amruta Ranade	7ecfa13ee0	Updated BigQuery, Google Sheets, Facebook Marketing, Salesforce docs (#13717) ... * initial changes * Edited google Sheets doc * More edits * edited the intro and prereqs for BigQuery * edited the data loading section * more edits * Grammatical edits * Formatting edits	2022-06-13 14:25:41 -04:00
LiRen Tu	e996c2c14f	🐛 BigQuery destination: always run GCS destination check for GCS staging method (#13579) ... * Run gcs destination check all the time * Add missing permissions to the status message * Add more error messages * Bump bigquery version * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-06-07 17:52:49 -07:00
Yurii Bidiuk	d6b50c1ce0	Destination BigQuery: refactored spec and doc (#13424) ... * change spec * refactored readme file for bigquery * change spec order * bump version	2022-06-07 18:36:40 +03:00
LiRen Tu	d6f8c2627b	📝 Update bigquery doc about hmac key troubleshooting (#12895)	2022-05-16 15:37:07 -07:00
LiRen Tu	e00f487d98	📝 Destination bigquery: mark service account as required for cloud (#12768) ... * Update spec * Update doc * Bump version and update changelog * Modify wording * Add sample service account key json * Add screenshots and common permission issues * Refactor service account helper method * Update log message * Update version date in changelog * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-05-15 01:28:36 -07:00
George Claireaux	440f3db8bb	Destinations BigQuery & Snowflake: use latest base-java to emit AirbyteTraceMessage on errros (#12805) ... * bumping BQ and Snowflake dests to get AirbyteTraceMessage from base-java * add PR link to changelogs * auto-bump connector version * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-05-12 14:23:29 +01:00
LiRen Tu	bd59726668	Publish new connectors to log offending json fields in json to avro conversion for debugging (#12578) ... * Publish new connectors to log debugging info in json to avro conversion * Add pull request id * auto-bump connector version * auto-bump connector version * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-05-04 11:47:07 -07:00
Alexandre Girard	1021428998	Destination Bigquery: Update Dataset location field description (#12528) ... * Update field description * reset * Bump * reset * auto-bump connector version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-05-02 17:45:36 -07:00
Alexandre Girard	553024780f	Destination Bigquery: Dataset location is a required field (#12477) ... * Dataset location is required * Update doc * Bump versions * reset * Bump * downgrade * Revert "downgrade" This reverts commit b482c54d6f. * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-04-29 15:34:43 -07:00
VitaliiMaltsev	0f8e11e243	🐛 BigQuery Destination failed conditional binding (#12068) ... * BigQuery Destination check conditional binding * fixed checkstyle * BigQuery Destination fixed bug with contitional binding * update CHANGELOG * update CHANGELOG * add explaining comment * bump version * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-04-15 19:17:46 +03:00
Eugene	281e0eec35	🎉Destination-bigquery-denorm: fixed emittedAt timestamp (#11978) ... * [11067] Destination-bigquery-denorm: fixed emittedAt timestamp	2022-04-15 17:39:51 +03:00
LiRen Tu	8bd2d9b518	🎉 BigQuery destination: use serialized buffer for gcs staging (#11776) ... * Rebase bigquery changes to master * Add comments * Uncomment test code * Format code * Bump versions * Fix denormalized destination target table name * Fix avro schema for denormalized destination * Remove unnecessary params from consumer factory * Add back previous version * Add warning about standard mode * auto-bump connector version * Bump version for bigquery in seed Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>	2022-04-07 16:59:19 -07:00
andriikorotkov	72dcd8126a	🐛 Destination bigquery denormalize "allOf" and "anyOf" fix (#11166) ... * fix bug with anyOf and allOf json blocks * updated tests and formatter * clean code * updated tests * updated destination_specs yaml file * updated version of connector and docs	2022-04-05 21:52:13 +03:00
Oleksandr Sheheda	24d7ab1f58	10148 BigQuery Destination/Denormalized Destination spec updates (#11620) ... * 10148 BigQuery Destination/Denormalized Destination spec updates * 10148 updated changelog * 10148 updated docker image version * 10148 updated docker image version	2022-04-02 20:24:39 +03:00
Eugene	2008970c7c	🎉Destination-Bigquery-denormalized: added tests for schema selection (#11636) ... * [8895] Destination-Bigquery-denormalized: added tests for schema selection	2022-04-01 20:34:30 +03:00
Eugene	69841cee73	🎉BigQuery denormalized Destination: Integration test enhancement for picking test-data and schemas (#11454) ... * [8870] BigQuery denormalized Destination: Integration test enhancement for picking test-data and schemas	2022-03-30 21:35:37 +03:00
VitaliiMaltsev	2ed4c05a19	🎉 BigQuery Destination: improve check performance (#11350) ... * fix for jdk 17 * add CHANGELOG * bump redshift version * add CHANGELOG * Update airbyte-integrations/connectors/destination-bigquery/src/main/java/io/airbyte/integrations/destination/bigquery/BigQueryDestination.java * bump version Co-authored-by: vmaltsev <vitalii.maltsev@globallogic.com> Co-authored-by: Sherif A. Nada <snadalive@gmail.com>	2022-03-24 20:36:17 +02:00
andriikorotkov	8771ba99fb	🎉 BigQuery destination mlp (#11238) ... * fix bug with anyOf and allOf json blocks * updated spec and documentation * rollback last changes * updated spec * updated spec * updated bigquery permissions in doc * updated spec * updated spec * updated spec * updated tests * fixed remarks * updated tests * updated tests * added new bigquery version * updated spec * updated version * updated definitions	2022-03-23 13:46:40 +02:00
LiRen Tu	21ec23cc31	🐞 Fix invalid char in snowflake & bigquery namespace (#10793) ... * Add namespace test for snowflake * Enable namespace test for bigquery * Format code * Capitalize test case id * Update exception message to point to test case file * Update snowflake name transformer to prepend underscore * Override convertStreamName instead of getIdentifier * Add missing state message * Remove unused import * Disable more namespace test cases We don't want to introduce changes that will affect existing connections for now. * Dry method that mutates namespace * Pass through null * Normalize namespace * Fix test case * Revert consumer factory changes * Normalize namespace in catalog * Revert catalog normalization * Enable namespace test for all snowflake destination tests * Test namespace for both bigquery destination tests * Add unit test for bigquery name transformer * Transform bigquery schema name * Fix avro name transformer * Normalize avro namespace * Standardize namespace in gcs utils * Bump version for snowflake and bigquery * Enable namespace test for bigquery denormalized * Dry bigquery denormalized acceptance test * Revert some of the variable scope change * Fix unit test * Bump version * Introduce getNamespace method * Implement getNamespace method for bigquery * Switch to getNamespace methods * Update comments * Fix bigquery denormalized acceptance test * Format code * Dry bigquery destination test * Skip partition test for gcs mode * Bump version	2022-03-19 17:47:24 -07:00
Christophe Duong	b333f4fc75	🐛 Handle try/catch in BigQuery destination consumers (#10755) ... * Handle try/catch in BigQuery destination consumers * Remove parallelStream * Bumpversion of connector * update changelogs * update seeds	2022-03-03 16:31:42 +01:00