The e2e suite and CI gates could report green without checking anything: the
lint/codegen gate never ran on fork PRs, the codegen freshness check diffed the
wrong range, the sticky comment claimed passes it never read, and several
visual/palette/contrast/a11y gates were written so they could not fail. Closing
those gates then exposed real defects.
**E2E and CI.** New coverage for the assistant lifecycle, flow interrupt, live
panels, tab deep links, the report route and its exports, session expiry, and
uploads/downloads (59 → 125 spec cases). Gates now fail on real regressions:
absolute pixel budget, per-tab panel sweeps, scoped a11y waivers, page-error
assertions. CI runs on `pull_request`, ranges from the PR base, routes comments
by PR identity, and redacts stand secrets from the public artifact.
**Markdown editor.** ~20 round-trip and toolbar fixes: table pipe escaping
(nested lists, Go template pipelines, headerless tables), fence validity,
setext and line-leading escapes, inline code containing a backtick, block
toggles under select-all, controls disabled where the command is unavailable or
a table cell cannot hold the block, theme-aware code surface. Two perf fixes cut
1 MB document load from 69s to 4s.
**Accessibility and UI.** Focus returns to whatever opened a dialog, sheet or
menu; icon-only buttons named; `aria-sort` on table columns; syntax tokens
retuned to clear AA in both themes; a dedicated link token. Detail and settings
pages now distinguish authz denials and partial errors from missing records,
survive background refetches without blanking or discarding unsaved edits, and
stop naming exported PDFs `*.pdf.pdf`.
**Backend.** Passwords capped at the 72 bytes bcrypt can hash (over-long ones
previously 500'd after passing validation); four agent options the GraphQL round
trip silently dropped are carried through; a flow whose worker fails to start is
no longer left in the listing; the Graphiti health check retries before
permanently disabling the client.
---------
Co-authored-by: Sergey Kozyrenko <sirozha_blr@tut.by>
- Added specific error handling for `ErrFlowNotFound` in `PatchAssistant` and `DeleteAssistant` methods, returning a warning-level response when the flow is not found.
- Updated documentation in templates to clarify requirements for `center_node_uuid` in search types, ensuring users understand the necessity of copying UUIDs from prior results.
- Improved JSON schema descriptions for `GraphitiSearchAction` to emphasize the mandatory nature of the `query` field across all search types.
- Introduced new tests to validate handling of empty timeout values and ensure proper parsing of temporal window timestamps, addressing potential production issues.
- Introduced `callWithSetupRetries` function to enhance error resilience during LLM prompt calls, allowing for retries on transient errors with backoff.
- Updated `NewFlowProvider` and `NewAssistantProvider` methods to utilize `callWithSetupRetries` instead of direct calls to `prv.Call`, improving stability in flow and assistant creation.
- Added comprehensive unit tests for `callWithSetupRetries`, covering immediate success, transient error handling, and context cancellation scenarios.
- Enhanced error messages in Graphiti search tools to provide actionable feedback for missing or malformed parameters.
- Updated templates to clarify search type requirements and taxonomy references.
- Added configuration options for an optional internal analytics engine in `.env.example`, `docker-compose.yml`, and related files.
- Updated documentation to include details about the new internal analytics engine, which scrapes and summarizes pages as a fallback for the web_search tool.
- Enhanced the `ftester` to support testing the new internal engine alongside existing search engines.
- Adjusted various components to integrate the new engine, including configuration forms and validation logic.
This feature allows users to enable a browser-based fallback for analytic queries, enhancing the flexibility of the web search capabilities.
- Added new configuration options in `.vscode/launch.json` for vllm and hcnsec providers.
- Updated `deepinfra.provider.yml` to reflect changes in model configurations and pricing.
- Introduced `hcnsec.provider.yml` with detailed model settings and pricing for HCNSec.
- Adjusted various test reports to reflect updated success rates and latencies for different models.
- Added `edit_file` to the file tool; diffs are parsed and applied in-memory via `github.com/sergi/go-diff`, exported as `tools.ApplyUnifiedDiff`.
- Made the hunk header lenient (position numbers optional) for models that omit them, with fuzzy content-based matching as fallback.
- Tuned `FileAction` field descriptions (layered, minimal-token) to steer models toward including context lines, based on live model testing via `ctester`.
- Added a `file_edit` multi-turn read_file → edit_file test case in `pkg/providers/tester`, using PentAGI's real tool schema, wired into the `advanced` test group.
- Fixed a bug where a single stateful test-case instance was shared across agent types, leaking conversation history and failures between independent runs.
- Extended the mock provider with sequential responses for multi-turn scenarios; added ctester integration tests.
Graphiti transport errors, an empty file path, and an omitted 'action' or
double-encoded 'questions' arg from the LLM all hard-failed the tool chain
and burned retries instead of degrading gracefully. Also: stopTaskTimeout
5s->60s (flow kept running after a false 500), and the routine "cookie
claim invalid" case now logs at Warn instead of Error.
Verified against the live /v2/search API: the response envelope only ever
populates data.web for this tool, and sources defaults to ["web"] when
omitted. The sources field carries a string-vs-object shape ambiguity across
the API/SDK/MCP layers (the Firecrawl MCP rejects string sources outright),
so dropping it removes a needless compatibility risk without changing
behaviour.
The summarizer prompt asked the model to 'directly answer the user query'
but never rendered the query — Firecrawl's /v2/search response, unlike
Tavily's, doesn't echo the query back, so it was dropped when adapting the
prompt. Every summarized result (the common path, since scrapeOptions
requests markdown) was therefore a generic page summary instead of a
query-focused one.
Thread the query through search -> parseHTTPResponse -> buildFirecrawlResult
-> getSummarizePrompt and restore the USER QUERY line, matching Tavily.
Add a test asserting the query is present in the prompt.
- Summarizer prompt now uses one-based source IDs (aligned with the numbered
links) and the metadata-fallback title/URL, and truncates each document's
markdown to the same per-result limit as the non-summarized path, so an
oversized or hostile page can't blow up the prompt.
- Reword tool description, searcher prompt and docs to say 'main-content
markdown' since the scrape uses onlyMainContent=true.
- config.md example now uses the real NewFirecrawlTool(...) constructor,
matching the searxng/sploitus entries.
Add Firecrawl as a network search provider alongside Tavily, Perplexity,
Traversaal, SearXNG and the others. Firecrawl's /v2/search endpoint combines
web search with page scraping, returning full-page markdown content for each
result, which the searcher agent summarizes like the other content-rich
engines.
- New firecrawl tool mirrors the Tavily provider layer-for-layer: registry
definition, tool-type mapping, factory wiring in both tool builders, and
ftester executor/args/mocks support.
- Configurable endpoint: FIRECRAWL_API_KEY plus an optional FIRECRAWL_API_URL
(default https://api.firecrawl.dev) so self-hosted Firecrawl instances work.
- Adds 'firecrawl' to the searchengine_type enum (goose migration + models)
and the search-log server model so searches are recorded per engine.
- Installer wizard, .env.example, docker-compose, prompts and docs updated.
The tool is only registered when FIRECRAWL_API_KEY is set, so existing
deployments are unaffected.
A single unreadable directory entry (dangling symlink, a file removed
between ls and stat, a transient /proc entry) used to fail the whole
listing with HTTP 500, blanking the file browser and discarding every
readable sibling. The frontend already expects /proc/sys to not fail
spuriously, but the backend did the opposite.
ListContainerDir now returns a ContainerDirListing{Files, Failures}: per-
entry stat errors no longer abort the batch. GetFlowContainerFiles serves
the readable entries as HTTP 200, carries the failures back in a new
ContainerFiles.Failures field, and logs each skipped entry (capped) plus a
degradation summary. Directory-level faults (not a dir, ls failed, container
gone) still return 500 — there is no partial to show. statContainerEntries
returns successes + failures instead of the lowest-index error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The tail reader's no-trailing-newline branch (a torn/truncated final
append, the case M3's fsync defends) and the over-window error branch
had no coverage. Add cases for a newline-free last line, a torn final
append that must be rejected, and a single line exceeding the 64KiB
window.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The comment narrated the replaced sync.Map design and defended the
choice; the fixed-stripe array plus FNV indexing is self-evident and
the rationale already lives in commit d12b984.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
record() called readLastEvidenceReceiptHash on every append, which re-read
and re-hashed the entire receipts.jsonl just to get the last hash and verify
the whole chain — O(N^2) over a flow's receipts, on the tool-call hot path
under the per-path lock (M2).
Read only the last line instead (windowed ReadAt from the end), failing
closed if that line is missing, wrong-schema, or hash-mismatched. This is
O(1) per append and stays correct across the multiple recorder instances
that write one path (L6): the file stays the single source of truth, so
there is no per-instance cache to go stale. Full-chain verification belongs
in a separate read-time verify tool, not on every write.
Off by default (EVIDENCE_RECEIPTS_ENABLED=false). Measured: 1024 concurrent
appends under -race dropped from ~49s to ~7.5s.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The append wrote the receipt line and returned without flushing, so a crash
between the write and the OS flushing its page cache could lose or truncate
the last receipt, after which the fail-closed chain check halts all further
writes until the file is repaired (M3). Sync the file before returning.
Off by default (EVIDENCE_RECEIPTS_ENABLED=false), so standard deployments are
unaffected; the cost lands only when the feature is explicitly enabled.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three guards for the L6 sharded mutex pool (d12b984):
- bounded: 100k distinct flows resolve to <= 256 mutexes, so reverting to a
per-path map (the original unbounded leak) fails the test.
- high concurrency: 16 cross-executor writers on one path keep the hash chain
intact under -race.
- stripe collision: two flows that hash to the same stripe keep their own
chains intact, confirming shared stripes add contention but not corruption.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A sync.Map holding one *sync.Mutex per flow path grew unbounded for the
whole server uptime. A fixed 256-stripe array keyed by FNV-32a hash of
the path caps memory at a constant cost while preserving per-path append
serialization required by the hash-chain invariant.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Added langfuse observability support to the graphiti search tool, allowing for detailed tracking of search operations.
- Introduced a mapping of search types to human-readable titles for better traceability in logs.
- Implemented a new method to build the input payload for langfuse retrievers, encapsulating relevant search parameters.
- Enhanced error handling to include retriever status updates on success and failure, improving observability and debugging capabilities.
- Introduced a process-level singleton for the anonymizer replacer to enhance performance and ensure thread safety.
- The replacer is built once using patterns loaded at startup, allowing it to be shared across all flow executor instances.
- Updated the flow tools executor to utilize the shared replacer, simplifying the creation process and improving efficiency.
Disabled-by-default audit feature (EVIDENCE_RECEIPTS_ENABLED): appends a
hash-chained JSONL receipt per finished/failed toolcall under
<DATA_DIR>/flow-<id>/evidence/receipts.jsonl, recording toolcall
provenance plus SHA-256 hashes of args/result (no raw content).
Integration adapted to current main: the original PR built the receipt
from a database.Toolcall returned by ce.db.UpdateToolcall*Result, which
the executor no longer uses after the ToolCallLogProvider (tclp) refactor.
Receipts are now built from the in-scope toolcall data at the tclp log
sites and recorded non-fatally, so a receipt failure is logged and never
fails an otherwise-successful toolcall.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Introduced `EMBEDDING_MAX_TEXT_BYTES` to limit the maximum byte size of text sent to the embedding model.
- Renamed database connection pool settings: `DATABASE_MAX_OPEN_CONNS`, `DATABASE_MAX_IDLE_CONNS`, and `DATABASE_VECTOR_MAX_CONNS` for improved PostgreSQL connection management.
- Updated relevant documentation to reflect these new configuration options and their usage.
- Adjusted various components to utilize the new settings for enhanced performance and resource management.
- Added ToolCallLogProvider interface with methods for logging tool calls, updating success and failure statuses.
- Introduced proxyToolCallLogProvider to handle ToolCall logging operations.
- Updated flow execution components to integrate ToolCall logging, including flow workers and controllers.
- Enhanced GraphQL schema to support ToolCall logs, including queries and subscriptions for real-time updates.
- Updated documentation to reflect the new ToolCall logging features and their usage.
- Introduced shared connection pooling for PostgreSQL using `*sql.DB` for sqlc and GORM, optimizing resource usage.
- Added new environment variables: `DB_MAX_OPEN_CONNS`, `DB_MAX_IDLE_CONNS`, and `DB_VECTOR_MAX_CONNS` for configurable connection limits.
- Updated documentation to reflect new connection pooling strategy and provide operational commands for monitoring.
- Implemented shared `pgxpool` for pgvector stores to reduce connection overhead and improve performance.
- Adjusted various components to utilize the new connection pooling setup, ensuring efficient database interactions.
- Introduced WaitTaskCompletion method in FlowWorker interface to block until the current task completes or the context expires.
- Implemented signalTaskComplete to manage task completion signaling across goroutines.
- Added waitFlowCompletion tool to handle waiting for task completion with configurable timeout.
- Updated assistant provider to include wait functionality for flow completion.
- Enhanced templates and tool registry to support new wait functionality.
- GraphQL/REST CRUD + semantic search for knowledge documents
- KnowledgeStore with admin/user-scoped filtering, re-embedding on update
- Real-time subscriptions (created/updated/deleted) per user and admin
- user_id tracking in all agent-stored documents (guide/answer/code/memory)
- sqlc queries, goose migrations, privilege grants, user_id backfill
- Memory cleanup on flow deletion; stale orphan purge via migration
- Unit tests for all KnowledgeStore operations including security cases
- Frontend GraphQL schema and TypeScript types regenerated
- Updated `getHTML` and `getMD` methods to return warnings for small content instead of errors.
- Implemented checks for binary URLs, returning descriptive errors when such URLs are encountered.
- Added new tests to validate the updated behavior for small and empty content handling.
- Replaced ambiguous "user's language" guidance in tools/args.go with explicit engagement-log vs technical-channel markers per field, with strong English-only requirement for vector-store and search-engine queries.
- Added a unified LANGUAGE POLICY block to every agent prompt (primary_agent, assistant, pentester, coder, installer, searcher, memorist, generator, refiner, reporter, enricher), tailored per agent based on its actual tool set.
- Extended template variables and tool access (TerminalToolName, FileToolName) for coder, pentester, installer, memorist, generator, refiner, and enricher to match their runtime tool registrations.
- Fixed inverted UseAgents condition and removed misleading vector-store write references in assistant prompt; corrected MEMORY SYSTEM INTEGRATION for mode-specific tool references.
- Compressed COMPLETION REQUIREMENTS across templates and aligned closing-tool guidance with the channel mapping (engagement-log message vs technical-channel result).
Fixes#285.
Co-Authored-By: Octopus <liyuan851277048@icloud.com>
- Added methods for non-recursive directory listing and file stat operations in the Docker client.
- Implemented a new API endpoint to retrieve files from a running container's directory.
- Updated documentation to reflect new file operations and API changes.
- Introduced data structures for container file metadata and integrated them into the flow file service.
- Enhanced flow file management capabilities with improved synchronization between local and container file systems.
- Added new endpoints for managing flow files, including listing, uploading, and deleting files within flow workspaces.
- Introduced FlowFile model to represent file metadata.
- Enhanced GraphQL schema to support flow file operations and subscriptions for real-time updates.
- Updated API documentation to reflect new flow file functionalities.
- new get_flow_status / stop_flow / submit_flow_input / patch_flow_subtasks tools in assistant executor, backed by FlowWorker callbacks
- flowStatusTool supports 5 detail levels with verbose mode, polling for task readiness, and per-size summarization
- summarizer LRU cache (1000 entries, 4 h TTL, SHA-256 key) on flowProvider to skip redundant LLM calls
- updated assistant.tmpl with full flow management protocol: state reference, decision guide, constraints
- updated flow_execution.md with terminal timeout config, new tools, and cache
- Changed default terminal tool timeout from 600 to 1200 seconds.
- Updated related documentation across .env.example, README.md, and config files to reflect the new timeout settings and their constraints.
- Enhanced descriptions in code comments and documentation to clarify timeout behavior, including clamping rules for values outside the accepted range.
