* addlog
* fix applicaion.yml
* remove logging
* var name for boolean
* test setup
* test
* more fix for testing
* self review
* remove unrelated changes
* remove unwanted cdk changes
* more clean ups
* Add /v1/web_backend/check_updates
* Add constant for fallback cases
* Uses the new get out of date connectors endpoint
* Move connector service to its own file. Invalidate count on version change.
* Better upgrade all functionality
* Cleanup
Co-authored-by: Jimmy Ma <jimmy@airbyte.io>
* Add cancelled state and update error to use crossicon instead of fontawesome
* Update cancelled icon to use fontawesome minus
* Show cancelled icon in connections job list and list table
* Update snapshots
* Add i18n for disabling the connection, limit supported preferences
* Remove custom style from NonBreakingChangsPreferenceField control label - not needed
* improve some types
* improve further
* clean up a bit more
* refactor loading state
* move loading state up
* remove isLoading references
* remove unused props and make fetch connector error work
* remove special component for name
* remove top level state for unifinished flows
* start removing uiwidget
* Update airbyte-webapp/src/views/Connector/ConnectorCard/ConnectorCard.module.scss
Co-authored-by: Tim Roes <tim@airbyte.io>
* remove undefined option for selected id
* remove unused prop
* fix types
* remove uiwidget state
* clean up
* adjust comment
* handle errors in a nice way
* do not respect default on oneOf fields
* rename to formblock
* reduce re-renders
* pass error to secure inputs
* simplify and improve styling
* align top
* code review
* remove comment
* review comments
* rename file
* be strict about boolean values
* add example
* track form error in error boundary
* review comments
* handle unexpected cases better
* enrich error with connector id
* rename prop
* use proper validation in array section
* fix test
* rename variable
Co-authored-by: Tim Roes <tim@airbyte.io>
* Remove uiWidgetsInfo state by finding the enum/const field defining the selected oneOf and adding it to the condition form block to check the selected value by looking at the formik state
* Only do default value calculation once before rendering the formik component the first time (remove PatchInitialValuesWithWidgetConfig hack)
* Build yup schema once by flattening the oneOf conditions into a single object and adding when conditions on the found condition key (remove RevalidateOnValidationSchemaChange hack)
The snowflake destination in version <=0.4.40 does not work together with the changes on this PR - existing connections will continue to work fine, but it's not possible to change the configuration. Please update the snowflake destination connector to 0.4.41.
oneOf properties not following the rules described in the documentation will stop working in the UI - the form will crash and a meaningful error is shown which also links to the documentation:
* convert request body
* fix tests
* test body data
* more tests
* more tests
* _
* return stacktrace
* pretty print
* Revert "pretty print"
This reverts commit 091253803c.
* Revert "Revert "pretty print""
This reverts commit b6f62d645b.
* replace \n
* missing type hint
* Increased default buffer count and introduces configurable value for destination redshift
* Updates logic to ensure filebuffers are at minimum the default number
* Reverted changes to increase the file buffer default but allow for users to increase with adequate warnings
* Bumps version number and updates changelog
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* [faker] decouple stream state
* add PR #
* commit Stream instantiate changes
* fixup expected record
* skip backward test for this version too
* Apply suggestions from code review
Co-authored-by: Augustin <augustin@airbyte.io>
* lint
* Create realistic datasets of 10GB, 100GB, and 1TB in size (#20558)
* Faker CSV Streaming utilities
* readme
* don't do a final pipe to jq or you will run out or ram
* doc
* Faker gets 250% faster (#20741)
* Faker is 250% faster
* threads in spec + lint
* pass tests
* revert changes to record helper
* cleanup
* update expected_records
* bump default records-per-slice to 1k
* enforce unique email addresses
* cleanup
* more comments
* `parallelism` and pass tests
* update expected records
* cleanup notes
* update readme
* update expected records
* auto-bump connector version
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
The Java 19 toolchain doesn't like sneaky throws. Not entirely sure why. However, I think it's better practice to not use sneaky throws as it makes it clearer what is throw and where.
Example error message when trying to compile the current codebase with Java 19:
error: Error during the transformation of 'io.airbyte.validation.json.JsonSchemaValidatorTest'; post-compiler 'lombok.bytecode.SneakyThrowsRemover' caused an exception: java.lang.IllegalArgumentException: Unsupported class file major version 63
at org.objectweb.asm.ClassReader.<init>(ClassReader.java:199)
at org.objectweb.asm.ClassReader.<init>(ClassReader.java:180)
at org.objectweb.asm.ClassReader.<init>(ClassReader.java:166)
at lombok.bytecode.AsmUtil.fixJSRInlining(AsmUtil.java:37)
at lombok.bytecode.SneakyThrowsRemover.applyTransformations(SneakyThrowsRemover.java:46)
at lombok.core.PostCompiler.applyTransformations(PostCompiler.java:44)
at lombok.core.PostCompiler$1.close(PostCompiler.java:87)
at jdk.compiler/com.sun.tools.javac.jvm.ClassWriter.writeClass(ClassWriter.java:1508)
at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.genCode(JavaCompiler.java:738)
Implement the persistence layer changes following #19191.
This PR handles writing and reading stats to the new stream stat_table and columns in the existing sync_stats table.
At the same time we introduce upserts of stats records - i.e. merge updates into a single record - in preparation for real time stats updates vs the current approach where a new stat record is always written.
There will be two remaining PRs after this:
- First PR will be to fully wire up and test the API.
- Second PR will be to actually save stats while jobs are running.
Add the orchestrator label to orchestrators so we can better differentiate orchestrator pods.
This is useful since orchestrator pods are the only pods in the job namespace with a need to talk to the main Airbyte application pods. These labels allow us to apply more granular network filtering.
Also took the chance to do some clean up of labels.
* Refactor BigQuery Destination Integration tests
to reduce code duplication and move configuration
logic from code into config files.
This refactoring will make it easier to add more
test cases for configuration variations such as
data location, accounts with various permission combinations,
and account impersonation
* More refactoring
* fix typo
* Change secret file names to avoid conflict with current tests on master
* remove copy-pasted credential file paths
* more copu-pasta reduction
