* remove operationIds from WebBackendConnectionUpdate, just use operations
* refactor connection updates to patch-style update, where null fields remain unchanged
* better comment and arg name
* format
* make sure we are still 'dual-writing' to the old schedule column, even when the patch doesn't specify anything for it
* update acceptance test to update with new schedule syntax
* add catalog sorting to preserve stream order during patch, and more tests
* format
* add description, throw runtime exception for impossible branches, move streamReset to private helper
* PR suggestions
* add nested test classes and write a test for the catalog sorting method
* format
* add comment clarifying that the catalog sort is for UX, and isn't critical
* format
* format
* update acceptance tests to send proper catalog patches instead of whole new catalog
* format
* format
* simplify catalog patching - now, if a catalog is present on the request, replace the entire catalog with it.
Otherwise, if catalog on the request is null, leave the catalog unchanged
* format
* format
* Revert "update acceptance tests to send proper catalog patches instead of whole new catalog"
This reverts commit 71922648b4e070f46ff6c468813b7ab8dd9d6651.
* adjust description
* add jobInfoLight API endpoint that excludes attempt information, which can be enormous as it includes all log lines
* update replication activity to call new light endpoint
* start implementation of new persistence method
* add includingJobId and totalJobCount to job list request
* format
* update local openapi as well
* refactor queries into JOOQ and return empty list if target job cannot be found
* fix descriptions and undo changes from other branch
* switch including job to starting job
* fix job history handler tests
* rewrite jobs subqueries in jooq
* fix multiple config type querying
* remove unnecessary casts
* switch back to 'including' and return multiple of page size necessary to include job
* undo webapp changes
* fix test description
* format
* Update protocol version from actor defs API operations
* Implement default airbyte protocol version support
* Add version parsing
* Add acceptance tests
* Fix Acceptance Tests
* format
* Make test package private
* add new Configs for multi-cloud
* add api endpoints for setting workflow attempt id and createOrUpdate state
* update activities to call APIs instead of persistence
* workerApp refactor to separately initialize control or data plane dependencies
* modify syncWorkflow to call new activity that decides which task queue for data plane tasks
* misc to get build working
* move StateConverter to worker, so that server and worker can both access without needing to introduce any new dependencies
* update configs - remove extranneous helpers, clarify naming and comments, removed COMBINED value
* forgot to actually remove COMBINED enum value, this removes it
* add WorkerApp todo for breaking API Client into a scoped client
* rename decideTaskQueueActivity var to routeToTaskQueueActivity
* pr comments
* naming fix
* refactor secretHydrator instantiation
* WorkerApp PR feedback: move API client logic to separate class, use updated configs, etc
* add a RouterService class that is injected into RouteToTaskQueueActivityImpl
* AttemptApi cleanup and added unit test coverage
* fix confusion between AttemptId and AttemptNumber in new AttemptApi
* remove unused getDataPlaneSecretsHydrator
* remove unused import
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
* save
* clean up more usages and remove withRefreshedCatalog
* make webapp use correct endpoint
* add back intercept
* fix acceptance test
* fix log
* remove 'new' from test name
* fix: clone api doesn't take update configurations
* fix: you will be able to create clone in different workspace
* fix: added description to source/destination body
* Display addtional failure information when sync is expanded
* rename
* Update airbyte-webapp/src/components/JobItem/components/FailureDetails.tsx
Co-authored-by: Tim Roes <tim@airbyte.io>
* fix bad merge
* jump to timestamp
* cleat timestamp when logs colapsed
* speed up search
* rename
* Do what @pedroslopez says
* Rebase from master after big API update
* `floor` matchtimes for greater range matching
* Update airbyte-webapp/src/components/JobItem/components/ErrorDetails.tsx
Co-authored-by: Tim Roes <tim@airbyte.io>
* Update airbyte-webapp/src/components/JobItem/components/ErrorDetails.tsx
Co-authored-by: Tim Roes <tim@airbyte.io>
* Update airbyte-webapp/src/components/JobItem/components/Logs.tsx
Co-authored-by: Tim Roes <tim@airbyte.io>
* Update airbyte-webapp/src/components/JobItem/JobItem.tsx
Co-authored-by: Tim Roes <tim@airbyte.io>
* replace regexp with `dayJs`
* Extract dayjs into globals module
* mach time in full-second resolution
* revert `dayJs` and use `Date.parse`
* Just show failure timestamp rather than scroll
Co-authored-by: Tim Roes <tim@airbyte.io>
* Set SourceCatalogId during connectionUpdate operation
* Return catalogId when get a connection
* Fix db operation of standardSync.sourceCatalogId
- value is not set correctly during update operation
- value is not read
* UI modification to set the sourceCatalogId
* remove sourceCatalogId from diff computation
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
* workspaceId should be part of spec request
* address review comment
* fix test
* format
* update octavia according to API changes
* create integration test for definition generation
* fix test
* fix test
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
* listActorDefinitions apis now require workspaceId
- listSourceDefinitions takes workspaceId
- listDestinationDefinitions takes workspaceId
* actorDefinitionCreate apis now require workspaceId
- SourceDefinitionCreate takes workspaceId
- DestinationDefinitionCreate takes workspaceId
* source_definitions/list_opt_in
* destination_definitions/list_opt_in
* create/delete source definition grants
* create/delete destination definition grants
* PR feedback
- remove use of opt-in
- update description of definition list apis
* new create/list source definition routes
- create new routes instead of making breaking changes to existing ones
- existing create/list routes will be repurposed as admin only routes
* new create/list destination definition routes
- create new routes instead of making breaking changes to existing ones
- existing create/list routes will be repurposed as admin only routes
* revert acceptance test changes
* new update/delete actor definition routes
- create new routes instead of making breaking changes to existing ones
- existing create/list routes will be repurposed as admin only routes
* rearrange routes in config.yaml
* PR feedback
* new create actor definition routes
- create new routes instead of making breaking changes to existing ones
- existing create/list routes will be repurposed as admin only routes
* typo
* typo
* Make SchedulerHandler store schema after fetching it
* Add `disable_cache` parameter to discover_schema API
* Return cached catalog if it already exists
* Address code review comments
* Add tests for caching of catalog in SchedulerHandler
* Format fixes
* Fix Acceptance tests
* New code review fixes
- Use upper case for global variable
- Inline definition and assignment of variable
* Feat: first cut to allow naming for connections
* fix
* fix: migration
* fix: migration
* fix: formatting
* fix: formatting
* fix: tests
* fix: -> is bit outside of what we do generally
* fix: tests are failing
* fix: tests are failing
* fix: tests are failing
* fix: tests are failing
* fix: tests are failing
* Feat: first cut to allow naming for connections
* fix
* fix: migration
* fix: migration
* fix: formatting
* fix: formatting
* fix: tests
* fix: -> is bit outside of what we do generally
* tombstone added to schema dump
* read/write actor_definition.tombstone in config persistence
* add tombstone field when loading from yaml seed persistence
* add tombstone to normalized_tables_schema.txt
* add test to ensure tombstone defaults to false
* add release_date and release_stage to actor_definition
- migration to add columns
- update json schema to add new fields
- update API schema to include new fields in definition reads
- update tests
* rename definition creation methods to include custom
* add format:date for releaseDate, and define shared releaseStage enum
* handle null releaseDate
* add attempt failure info to api yml
* populate failureSummary in JobConverter
* tests and formatting
* add Partial Success to glossary section of docs
* failure summary naming/enum tweaks
* tweak enum values and add retryable boolean instead of transient failureType
* Allow updating workspace names
* Add additional unit test
* Fix code styling
* Update slug as well
* Update indentations
* Pull name update into separate endpoint
* add optional tombstone field to StandardSourceDefinition and update repository methods to use it
add new ConfigRepository unit tests to cover includeTombstone logic
whitespace nits
remove accidental import
let autoformat decide whitespace
set tombstone false when creating a SourceDefinition
set tombstone in a couple missed places
remove includeTombstone param from get repository methods
check for tombstone sourceDefinition in ConfigDumpImporter
add tombstone to DestinationDefinition as well
add sourceDefinition delete endpoint, handler, and tests. also fix up some variable names (source -> sourceDefinition) because of confusion and conflicts
add api endpoint to delete destination definitions
filter via stream instead of if-continue
fix comment formatting
* gradle format
After a sync is completed, we tag all the workspace with related to this sync as having a first sync completed.
This will then be forwarded to the UI in another PR.
This is part of #5884
* Enhance API for use by cloud to provide per-connector billing information
* Add listAllConnectionsForWorkspace to include deleted
* Config for list all connections
* Merged master, and formatting
* Formatting
* Name change per PR suggestion
* Name change per PR suggestion
* Formatting again
* Change OAuth API
* Change protocol for new OAuthConfigSpecification
* Refactor OAuth classes and tests
* Remove webbackend source/destination creation
* Change from webback to normal API
* Implement new protocol change with OAuth specs
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* format
* format
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
