* ensure workspace webhook configs can be correctly passed between API and persistence layers
* remove unnecessary logging
* add unit tests to workspace webhook config handling
* additional testing and style cleanup around workspace webhook config handling
* introduce webhook operations to the operations API and persistence
* add unit tests for webhooks in operations endpoint handler
* fixes and additional testing in webhook operations handler
* cleanup refactor around operations handling to reduce duplicate code
* wip
* handle webhook configs in workspaces endpoint and split/hydrate secrets
* style improvements to documentation around webhook configs
* Clarify documentation around webhook auth tokens
* More documentation clarification around webhook configs
* Format.
* unit test coverage for webhook config handling
* use common json parsing libraries around webhook configs
* clean up around testing webhook operation configs
Co-authored-by: Davin Chia <davinchia@gmail.com>
* progress on adding geography throughout api
* fix workspace handler test
* more progress
* implement workspace defaulting and add/update more tests
* fix bootloader tests
* set defaultGeography in missing places
* add Geography column when reading Connection record from DB
* fix pmd
* add more comments/description
* format
* description
* Use PATCH Api for toggling connections
* remove catalog from web backend connection list, and move icon to source/destination field in response
* Adjust FE code
* comment out failing tests
* Resolve merge conflict
* add back in tests and make them pass with new list items
* format
* leave repository layer alone for now, just remove catalog from API response
* format
* load the icon when returning
* update sourceHandler and destinationHandler test to account for icons
* add icon to source and destination in webBackendConnectionHandlerTest
* change icon test to actually load an svg rather than using static mocks
* also fix icon test for WebBackendConnectionsHandlerTest
* make PMD happy
Co-authored-by: Tim Roes <tim@airbyte.io>
Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: KC <krishna@airbyte.io>
* remove operationIds from WebBackendConnectionUpdate, just use operations
* refactor connection updates to patch-style update, where null fields remain unchanged
* better comment and arg name
* format
* make sure we are still 'dual-writing' to the old schedule column, even when the patch doesn't specify anything for it
* update acceptance test to update with new schedule syntax
* add catalog sorting to preserve stream order during patch, and more tests
* format
* add description, throw runtime exception for impossible branches, move streamReset to private helper
* PR suggestions
* add nested test classes and write a test for the catalog sorting method
* format
* add comment clarifying that the catalog sort is for UX, and isn't critical
* format
* format
* update acceptance tests to send proper catalog patches instead of whole new catalog
* format
* format
* simplify catalog patching - now, if a catalog is present on the request, replace the entire catalog with it.
Otherwise, if catalog on the request is null, leave the catalog unchanged
* format
* format
* Revert "update acceptance tests to send proper catalog patches instead of whole new catalog"
This reverts commit 71922648b4e070f46ff6c468813b7ab8dd9d6651.
* adjust description
* add jobInfoLight API endpoint that excludes attempt information, which can be enormous as it includes all log lines
* update replication activity to call new light endpoint
* start implementation of new persistence method
* add includingJobId and totalJobCount to job list request
* format
* update local openapi as well
* refactor queries into JOOQ and return empty list if target job cannot be found
* fix descriptions and undo changes from other branch
* switch including job to starting job
* fix job history handler tests
* rewrite jobs subqueries in jooq
* fix multiple config type querying
* remove unnecessary casts
* switch back to 'including' and return multiple of page size necessary to include job
* undo webapp changes
* fix test description
* format
* Update protocol version from actor defs API operations
* Implement default airbyte protocol version support
* Add version parsing
* Add acceptance tests
* Fix Acceptance Tests
* format
* Make test package private
* add new Configs for multi-cloud
* add api endpoints for setting workflow attempt id and createOrUpdate state
* update activities to call APIs instead of persistence
* workerApp refactor to separately initialize control or data plane dependencies
* modify syncWorkflow to call new activity that decides which task queue for data plane tasks
* misc to get build working
* move StateConverter to worker, so that server and worker can both access without needing to introduce any new dependencies
* update configs - remove extranneous helpers, clarify naming and comments, removed COMBINED value
* forgot to actually remove COMBINED enum value, this removes it
* add WorkerApp todo for breaking API Client into a scoped client
* rename decideTaskQueueActivity var to routeToTaskQueueActivity
* pr comments
* naming fix
* refactor secretHydrator instantiation
* WorkerApp PR feedback: move API client logic to separate class, use updated configs, etc
* add a RouterService class that is injected into RouteToTaskQueueActivityImpl
* AttemptApi cleanup and added unit test coverage
* fix confusion between AttemptId and AttemptNumber in new AttemptApi
* remove unused getDataPlaneSecretsHydrator
* remove unused import
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
* save
* clean up more usages and remove withRefreshedCatalog
* make webapp use correct endpoint
* add back intercept
* fix acceptance test
* fix log
* remove 'new' from test name
* fix: clone api doesn't take update configurations
* fix: you will be able to create clone in different workspace
* fix: added description to source/destination body
* Display addtional failure information when sync is expanded
* rename
* Update airbyte-webapp/src/components/JobItem/components/FailureDetails.tsx
Co-authored-by: Tim Roes <tim@airbyte.io>
* fix bad merge
* jump to timestamp
* cleat timestamp when logs colapsed
* speed up search
* rename
* Do what @pedroslopez says
* Rebase from master after big API update
* `floor` matchtimes for greater range matching
* Update airbyte-webapp/src/components/JobItem/components/ErrorDetails.tsx
Co-authored-by: Tim Roes <tim@airbyte.io>
* Update airbyte-webapp/src/components/JobItem/components/ErrorDetails.tsx
Co-authored-by: Tim Roes <tim@airbyte.io>
* Update airbyte-webapp/src/components/JobItem/components/Logs.tsx
Co-authored-by: Tim Roes <tim@airbyte.io>
* Update airbyte-webapp/src/components/JobItem/JobItem.tsx
Co-authored-by: Tim Roes <tim@airbyte.io>
* replace regexp with `dayJs`
* Extract dayjs into globals module
* mach time in full-second resolution
* revert `dayJs` and use `Date.parse`
* Just show failure timestamp rather than scroll
Co-authored-by: Tim Roes <tim@airbyte.io>
* Set SourceCatalogId during connectionUpdate operation
* Return catalogId when get a connection
* Fix db operation of standardSync.sourceCatalogId
- value is not set correctly during update operation
- value is not read
* UI modification to set the sourceCatalogId
* remove sourceCatalogId from diff computation
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
* workspaceId should be part of spec request
* address review comment
* fix test
* format
* update octavia according to API changes
* create integration test for definition generation
* fix test
* fix test
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
* listActorDefinitions apis now require workspaceId
- listSourceDefinitions takes workspaceId
- listDestinationDefinitions takes workspaceId
* actorDefinitionCreate apis now require workspaceId
- SourceDefinitionCreate takes workspaceId
- DestinationDefinitionCreate takes workspaceId
* source_definitions/list_opt_in
* destination_definitions/list_opt_in
* create/delete source definition grants
* create/delete destination definition grants
* PR feedback
- remove use of opt-in
- update description of definition list apis
* new create/list source definition routes
- create new routes instead of making breaking changes to existing ones
- existing create/list routes will be repurposed as admin only routes
* new create/list destination definition routes
- create new routes instead of making breaking changes to existing ones
- existing create/list routes will be repurposed as admin only routes
* revert acceptance test changes
* new update/delete actor definition routes
- create new routes instead of making breaking changes to existing ones
- existing create/list routes will be repurposed as admin only routes
* rearrange routes in config.yaml
* PR feedback
* new create actor definition routes
- create new routes instead of making breaking changes to existing ones
- existing create/list routes will be repurposed as admin only routes
* typo
* typo
* Make SchedulerHandler store schema after fetching it
* Add `disable_cache` parameter to discover_schema API
* Return cached catalog if it already exists
* Address code review comments
* Add tests for caching of catalog in SchedulerHandler
* Format fixes
* Fix Acceptance tests
* New code review fixes
- Use upper case for global variable
- Inline definition and assignment of variable
* Feat: first cut to allow naming for connections
* fix
* fix: migration
* fix: migration
* fix: formatting
* fix: formatting
* fix: tests
* fix: -> is bit outside of what we do generally
* fix: tests are failing
* fix: tests are failing
* fix: tests are failing
* fix: tests are failing
* fix: tests are failing
* Feat: first cut to allow naming for connections
* fix
* fix: migration
* fix: migration
* fix: formatting
* fix: formatting
* fix: tests
* fix: -> is bit outside of what we do generally
* tombstone added to schema dump
* read/write actor_definition.tombstone in config persistence
* add tombstone field when loading from yaml seed persistence
* add tombstone to normalized_tables_schema.txt
* add test to ensure tombstone defaults to false
* add release_date and release_stage to actor_definition
- migration to add columns
- update json schema to add new fields
- update API schema to include new fields in definition reads
- update tests
* rename definition creation methods to include custom
* add format:date for releaseDate, and define shared releaseStage enum
* handle null releaseDate
* add attempt failure info to api yml
* populate failureSummary in JobConverter
* tests and formatting
* add Partial Success to glossary section of docs
* failure summary naming/enum tweaks
* tweak enum values and add retryable boolean instead of transient failureType
* Allow updating workspace names
* Add additional unit test
* Fix code styling
* Update slug as well
* Update indentations
* Pull name update into separate endpoint
