Commit Graph
17 Commits
Author SHA1 Message Date
Dmitry Ng 39f122467d feat(config): add new embedding and rename database connection pool settings
- Introduced `EMBEDDING_MAX_TEXT_BYTES` to limit the maximum byte size of text sent to the embedding model.
- Renamed database connection pool settings: `DATABASE_MAX_OPEN_CONNS`, `DATABASE_MAX_IDLE_CONNS`, and `DATABASE_VECTOR_MAX_CONNS` for improved PostgreSQL connection management.
- Updated relevant documentation to reflect these new configuration options and their usage.
- Adjusted various components to utilize the new settings for enhanced performance and resource management.
2026-05-18 18:26:52 +03:00
Dmitry Ng 2ce863ec1a feat(toolcall): implement ToolCall logging functionality
- Added ToolCallLogProvider interface with methods for logging tool calls, updating success and failure statuses.
- Introduced proxyToolCallLogProvider to handle ToolCall logging operations.
- Updated flow execution components to integrate ToolCall logging, including flow workers and controllers.
- Enhanced GraphQL schema to support ToolCall logs, including queries and subscriptions for real-time updates.
- Updated documentation to reflect the new ToolCall logging features and their usage.
2026-05-18 11:21:56 +03:00
Dmitry Ng 077ddce476 feat(database): enhance PostgreSQL connection pooling and configuration
- Introduced shared connection pooling for PostgreSQL using `*sql.DB` for sqlc and GORM, optimizing resource usage.
- Added new environment variables: `DB_MAX_OPEN_CONNS`, `DB_MAX_IDLE_CONNS`, and `DB_VECTOR_MAX_CONNS` for configurable connection limits.
- Updated documentation to reflect new connection pooling strategy and provide operational commands for monitoring.
- Implemented shared `pgxpool` for pgvector stores to reduce connection overhead and improve performance.
- Adjusted various components to utilize the new connection pooling setup, ensuring efficient database interactions.
2026-05-18 11:13:43 +03:00
Dmitry Ng 1bb7f8a9a0 feat(flow): add WaitTaskCompletion method and associated tools for assistant
- Introduced WaitTaskCompletion method in FlowWorker interface to block until the current task completes or the context expires.
- Implemented signalTaskComplete to manage task completion signaling across goroutines.
- Added waitFlowCompletion tool to handle waiting for task completion with configurable timeout.
- Updated assistant provider to include wait functionality for flow completion.
- Enhanced templates and tool registry to support new wait functionality.
2026-05-16 22:52:51 +03:00
Dmitry Ng 3a52079278 feat(knowledge): add pgvector knowledge base management
- GraphQL/REST CRUD + semantic search for knowledge documents
- KnowledgeStore with admin/user-scoped filtering, re-embedding on update
- Real-time subscriptions (created/updated/deleted) per user and admin
- user_id tracking in all agent-stored documents (guide/answer/code/memory)
- sqlc queries, goose migrations, privilege grants, user_id backfill
- Memory cleanup on flow deletion; stale orphan purge via migration
- Unit tests for all KnowledgeStore operations including security cases
- Frontend GraphQL schema and TypeScript types regenerated
2026-05-05 01:09:20 +03:00
Dmitry Ng e370450dab feat(browser): enhance HTML and MD content handling with warnings for small content and errors for binary URLs
- Updated `getHTML` and `getMD` methods to return warnings for small content instead of errors.
- Implemented checks for binary URLs, returning descriptive errors when such URLs are encountered.
- Added new tests to validate the updated behavior for small and empty content handling.
2026-05-03 00:45:27 +03:00
Dmitry Ng b254ff6f90 refactor(flow_manager): improve error handling for running tasks 2026-05-03 00:43:51 +03:00
Dmitry NgandOctopus 956c11eadc feat: introduce engagement-log/technical-channel language policy across agent prompts
- Replaced ambiguous "user's language" guidance in tools/args.go with explicit engagement-log vs technical-channel markers per field, with strong English-only requirement for vector-store and search-engine queries.
- Added a unified LANGUAGE POLICY block to every agent prompt (primary_agent, assistant, pentester, coder, installer, searcher, memorist, generator, refiner, reporter, enricher), tailored per agent based on its actual tool set.
- Extended template variables and tool access (TerminalToolName, FileToolName) for coder, pentester, installer, memorist, generator, refiner, and enricher to match their runtime tool registrations.
- Fixed inverted UseAgents condition and removed misleading vector-store write references in assistant prompt; corrected MEMORY SYSTEM INTEGRATION for mode-specific tool references.
- Compressed COMPLETION REQUIREMENTS across templates and aligned closing-tool guidance with the channel mapping (engagement-log message vs technical-channel result).

Fixes #285.

Co-Authored-By: Octopus <liyuan851277048@icloud.com>
2026-04-30 15:29:56 +03:00
Dmitry Ng aa4b70eaba feat: add user resources system and flow integration
- UserResource model with MD5-deduplicated blob storage and virtual path filesystem
- REST API for resource CRUD (upload, mkdir, move, copy, delete, download)
- GraphQL query/mutations with resourceIds support on createFlow, putUserInput, createAssistant, callAssistant
- Resource → flow copy with hierarchy restore; incremental container sync (find missing, copy once)
- FlowWorker.PutResources delegates copy, docker push and flowFileAdded events
- Agent prompts updated with {{.UserFiles}} XML listing of /work/uploads and /work/resources
- Resource subscriptions: resourceAdded/Updated/Deleted
2026-04-28 17:00:17 +03:00
Dmitry Ng 7c4ebda2c2 feat: enhance Docker client with container file operations and API integration
- Added methods for non-recursive directory listing and file stat operations in the Docker client.
- Implemented a new API endpoint to retrieve files from a running container's directory.
- Updated documentation to reflect new file operations and API changes.
- Introduced data structures for container file metadata and integrated them into the flow file service.
- Enhanced flow file management capabilities with improved synchronization between local and container file systems.
2026-04-27 13:19:39 +03:00
Dmitry Ng 72b1c8489e feat: implement flow file management with upload and retrieval capabilities
- Added new endpoints for managing flow files, including listing, uploading, and deleting files within flow workspaces.
- Introduced FlowFile model to represent file metadata.
- Enhanced GraphQL schema to support flow file operations and subscriptions for real-time updates.
- Updated API documentation to reflect new flow file functionalities.
2026-04-27 10:03:42 +03:00
Dmitry Ng 5067e8f5a4 feat: add assistant flow management tools and summarizer cache
- new get_flow_status / stop_flow / submit_flow_input / patch_flow_subtasks tools in assistant executor, backed by FlowWorker callbacks
- flowStatusTool supports 5 detail levels with verbose mode, polling for task readiness, and per-size summarization
- summarizer LRU cache (1000 entries, 4 h TTL, SHA-256 key) on flowProvider to skip redundant LLM calls
- updated assistant.tmpl with full flow management protocol: state reference, decision guide, constraints
- updated flow_execution.md with terminal timeout config, new tools, and cache
2026-04-24 02:37:09 +03:00
Dmitry Ng 78e46bd77a refactor: update TERMINAL_TOOL_TIMEOUT to 1200 seconds with detailed documentation
- Changed default terminal tool timeout from 600 to 1200 seconds.
- Updated related documentation across .env.example, README.md, and config files to reflect the new timeout settings and their constraints.
- Enhanced descriptions in code comments and documentation to clarify timeout behavior, including clamping rules for values outside the accepted range.
2026-04-22 13:22:32 +03:00
Mason Kim(ZINUS US_SALES) 1d6da349d4 fix: clamp oversized terminal timeout 2026-04-15 13:14:16 -04:00
Mason Kim(ZINUS US_SALES) 86d7666c86 feat: add configurable terminal tool timeout
Signed-off-by: Mason Kim(ZINUS US_SALES) <mkim@zinus.com>
2026-04-15 12:34:03 -04:00
Dmitry Ng c8cd0e68f9 feat: add Docker host network mode support and improve agent terminal execution
- Add host network mode support in Docker client (DOCKER_NETWORK=host)
- Update documentation for network modes (bridge vs host)
- Enhance OOB port allocation guidance with mandatory directives
- Improve terminal command execution descriptions (detach, timeout)
- Fix MSF workflow issues: add process isolation rules and RPC daemon patterns
- Add terminal execution mechanics to adviser prompts for better monitoring
- Update installer locale with host network mode explanation

Fixes agent issues with msfconsole hanging, port conflicts, and process isolation.
2026-03-29 15:53:30 +03:00
Dmitry Ng b90ea4711e repo final state 2026-03-26 06:16:07 +03:00