Commit Graph

30 Commits

Author SHA1 Message Date
Harry
af17e20f99 feat(sandbox): implement sandbox archive upload/download endpoints and security enhancements
- Added sandbox archive upload and download proxy endpoints with signed URL verification.
- Introduced security helpers for generating and verifying signed URLs.
- Updated file-related API routes to include sandbox archive functionality.
- Refactored app asset storage methods to streamline download/upload URL generation.
2026-01-26 01:11:53 +08:00
Harry
c035133353 refactor(asset-storage): fix security problems 2026-01-25 03:44:36 +08:00
Harry
eed9faedaa refactor: replace AppAssetsInitializer with DraftAppAssetsInitializer and ensure assets directory creation in app_assets_initializer 2026-01-23 18:51:31 +08:00
Harry
248fa38c34 refactor(storage): unified storage cache layer and preasign interface
- Updated storage wrappers to utilize a new base class, StorageWrapper, for better delegation of methods.
- Introduced SilentStorage to handle read operations gracefully by returning empty values instead of raising exceptions.
- Enhanced CachedPresignStorage to support batch caching of download URLs, improving performance.
- Refactored FilePresignStorage to support both presigned URLs and signed proxy URLs for downloads.
- Updated AppAssetService to utilize the new storage structure, ensuring consistent asset management.
2026-01-23 17:01:10 +08:00
Harry
3165f3adbe feat(app_assets): enhance asset management with CachedPresignStorage
- Introduced CachedPresignStorage to cache presigned download URLs, reducing repeated API calls.
- Updated AppAssetService to utilize CachedPresignStorage for improved performance in asset download URL generation.
- Refactored asset builders and packagers to support the new storage mechanism.
- Removed unused AppAssetsAttrsInitializer to streamline initialization processes.
- Added unit tests for CachedPresignStorage to ensure functionality and reliability.
2026-01-23 16:10:28 +08:00
Harry
dde2bea2cc fix(llm-skill): prompt tool call
- Renamed `build_skill_artifact_set` to `build_skill_bundle` for improved clarity in asset management.
- Updated references in `SkillManager` to reflect the new method name and ensure consistent handling of skill bundles.
- Added `AppAssetsAttrsInitializer` to `SandboxManager` to enhance asset initialization processes.
- Implemented output truncation in `SandboxBashTool` to manage long command outputs effectively.
2026-01-22 23:36:32 +08:00
Harry
521b66c488 feat(app-bundle): implement app bundle import/export functionality
- Introduced AppBundleService for managing app bundle publishing and importing, integrating workflow and asset services.
- Added methods for exporting app bundles as ZIP files, including DSL and asset management.
- Implemented source zip extraction and validation to enhance asset import processes.
- Refactored asset packaging to utilize AssetZipPackager for improved performance and organization.
- Enhanced error handling for bundle format and security during import operations.
2026-01-22 22:46:57 +08:00
Harry
a43efef9f0 refactor(skill): transition from artifact set to bundle structure
- Replaced SkillArtifactSet with SkillBundle across various components, enhancing the organization of skill dependencies and references.
- Updated SkillManager methods to load and save bundles instead of artifacts, improving clarity in asset management.
- Refactored SkillCompiler to compile skills into bundles, streamlining the dependency resolution process.
- Adjusted DifyCli and SandboxBashSession to utilize ToolDependencies, ensuring consistent handling of tool references.
- Introduced AssetReferences for better management of file dependencies within skill bundles.
2026-01-22 22:46:57 +08:00
Yeuoly
d32996b8c9 feat: enhance sandbox initializers with async and sync support 2026-01-22 19:58:38 +08:00
Yeuoly
a3cf73b220 feat: refactor initializers to support async and sync execution 2026-01-22 19:54:54 +08:00
Yeuoly
55c588a03a feat: add async_initialize method to asset initializers 2026-01-22 19:49:41 +08:00
Yeuoly
3058415b4e feat: switch async 2026-01-22 19:42:09 +08:00
Yeuoly
c3b4029d0b feat: DraftAppAssetsInitializer 2026-01-22 19:18:46 +08:00
Yeuoly
5e16d85ff6 refactor(sandbox): async init and draft downloads
Reduce startup latency by deferring sandbox setup and downloading draft assets directly with cached presigned URLs.
2026-01-22 19:18:34 +08:00
Harry
9d80770dfc feat(sandbox): enhance sandbox management and tool artifact handling
- Introduced SandboxManager.delete_storage method for improved storage management.
- Refactored skill loading and tool artifact handling in DifyCliInitializer and SandboxBashSession.
- Updated LLMNode to extract and compile tool artifacts, enhancing integration with skills.
- Improved attribute management in AttrMap for better error handling and retrieval methods.
2026-01-22 17:26:09 +08:00
Harry
e7c3e4cd21 feat: introduce attribute management system for sandbox
- Added AttrMap and AttrKey classes for type-safe attribute storage.
- Implemented AppAssetsAttrs and SkillAttrs for managing application and skill attributes.
- Refactored Sandbox and initializers to utilize the new attribute management system, enhancing modularity and clarity in asset handling.
2026-01-22 17:26:09 +08:00
Harry
9ed83a808a refactor: consolidate sandbox management and initialization
- Moved sandbox-related classes and functions into a dedicated module for better organization.
- Updated the sandbox initialization process to streamline asset management and environment setup.
- Removed deprecated constants and refactored related code to utilize new sandbox entities.
- Enhanced the workflow context to support sandbox integration, allowing for improved state management during execution.
- Adjusted various components to utilize the new sandbox structure, ensuring compatibility across the application.
2026-01-21 20:42:44 +08:00
Harry
f831d3bbd6 fix(app_assets_initializer): specify output directory for unzip command to ensure proper asset extraction 2026-01-21 02:58:47 +08:00
Harry
7fd9ef3d22 fix(dify_cli): solve the permission error on e2b 2026-01-21 01:25:21 +08:00
Harry
705d4cbba9 feat(sandbox_provider): add default sandbox provider for CE 2026-01-21 00:37:38 +08:00
Harry
18a589003e feat(sandbox): enhance sandbox initialization with draft support and asset management
- Introduced DraftAppAssetsInitializer for handling draft assets.
- Updated SandboxLayer to conditionally set sandbox ID and storage based on workflow version.
- Improved asset initialization logging and error handling.
- Refactored ArchiveSandboxStorage to support exclusion patterns during archiving.
- Modified command and LLM nodes to retrieve sandbox from workflow context, supporting draft workflows.
2026-01-20 19:45:04 +08:00
Harry
1c76ed2c40 feat(sandbox): draft storage 2026-01-20 18:45:13 +08:00
Harry
f5a34e9ee8 feat(skill): skill support 2026-01-20 03:02:34 +08:00
Harry
956436b943 feat(sandbox): skill initialize & draft run 2026-01-19 18:15:39 +08:00
Harry
3bb9c4b280 feat(constants): introduce DIFY_CLI_ROOT and update paths for Dify CLI and app assets
- Added DIFY_CLI_ROOT constant for the root directory of Dify CLI.
- Updated DIFY_CLI_PATH and DIFY_CLI_CONFIG_PATH to use absolute paths.
- Modified app asset initialization to create directories under DIFY_CLI_ROOT.
- Enhanced Docker and E2B environment file handling to use workspace paths.
2026-01-19 18:15:39 +08:00
Harry
0de32f682a feat(skill): skill parser & packager 2026-01-19 12:41:01 +08:00
Harry
41baaca21d feat(sandbox): integrate ArchiveSandboxStorage into AdvancedChat and Workflow app generators 2026-01-15 17:20:10 +08:00
Harry
63b3e71909 refactor(sandbox): redesign sandbox_layer & reorganize import paths 2026-01-15 13:22:49 +08:00
Harry
94ff904a04 feat(sandbox): add AppAssetsInitializer and refactor VMFactory to VMBuilder
- Add AppAssetsInitializer to load published app assets into sandbox
- Refactor VMFactory.create() to VMBuilder with builder pattern
- Extract SandboxInitializer base class and DifyCliInitializer
- Simplify SandboxLayer constructor (remove options/environments params)
- Fix circular import in sandbox module by removing eager SandboxBashTool export
- Update SandboxProviderService to return VMBuilder instead of VirtualEnvironment
2026-01-15 00:13:52 +08:00
Harry
a0c388f283 refactor(sandbox): extract connection helpers and move run_command to helper module
- Add helpers.py with connection management utilities:
    - with_connection: context manager for connection lifecycle
    - submit_command: execute command and return CommandFuture
    - execute: run command with auto connection, raise on failure
    - try_execute: run command with auto connection, return result

  - Add CommandExecutionError to exec.py for typed error handling
    with access to exit_code, stderr, and full result

  - Remove run_command method from VirtualEnvironment base class
    (now available as submit_command helper)

  - Update all call sites to use new helper functions:
    - sandbox/session.py
    - sandbox/storage/archive_storage.py
    - sandbox/bash/bash_tool.py
    - workflow/nodes/command/node.py

  - Add comprehensive unit tests for helpers with connection reuse
2026-01-15 00:13:52 +08:00