Commit Graph
199 Commits
Author SHA1 Message Date
Dmitry NgandSergey Kozyrenko 45c84c500f test(frontend): make the e2e suite falsifiable, and fix what it found (#380)
The e2e suite and CI gates could report green without checking anything: the
lint/codegen gate never ran on fork PRs, the codegen freshness check diffed the
wrong range, the sticky comment claimed passes it never read, and several
visual/palette/contrast/a11y gates were written so they could not fail. Closing
those gates then exposed real defects.

**E2E and CI.** New coverage for the assistant lifecycle, flow interrupt, live
panels, tab deep links, the report route and its exports, session expiry, and
uploads/downloads (59 → 125 spec cases). Gates now fail on real regressions:
absolute pixel budget, per-tab panel sweeps, scoped a11y waivers, page-error
assertions. CI runs on `pull_request`, ranges from the PR base, routes comments
by PR identity, and redacts stand secrets from the public artifact.

**Markdown editor.** ~20 round-trip and toolbar fixes: table pipe escaping
(nested lists, Go template pipelines, headerless tables), fence validity,
setext and line-leading escapes, inline code containing a backtick, block
toggles under select-all, controls disabled where the command is unavailable or
a table cell cannot hold the block, theme-aware code surface. Two perf fixes cut
1 MB document load from 69s to 4s.

**Accessibility and UI.** Focus returns to whatever opened a dialog, sheet or
menu; icon-only buttons named; `aria-sort` on table columns; syntax tokens
retuned to clear AA in both themes; a dedicated link token. Detail and settings
pages now distinguish authz denials and partial errors from missing records,
survive background refetches without blanking or discarding unsaved edits, and
stop naming exported PDFs `*.pdf.pdf`.

**Backend.** Passwords capped at the 72 bytes bcrypt can hash (over-long ones
previously 500'd after passing validation); four agent options the GraphQL round
trip silently dropped are carried through; a flow whose worker fails to start is
no longer left in the listing; the Graphiti health check retries before
permanently disabling the client.

---------

Co-authored-by: Sergey Kozyrenko <sirozha_blr@tut.by>
2026-07-29 16:34:29 +04:00
Dmitry Ng 762c6233ce feat(flow): enhance error handling and documentation for flow retrieval
- Added specific error handling for `ErrFlowNotFound` in `PatchAssistant` and `DeleteAssistant` methods, returning a warning-level response when the flow is not found.
- Updated documentation in templates to clarify requirements for `center_node_uuid` in search types, ensuring users understand the necessity of copying UUIDs from prior results.
- Improved JSON schema descriptions for `GraphitiSearchAction` to emphasize the mandatory nature of the `query` field across all search types.
- Introduced new tests to validate handling of empty timeout values and ensure proper parsing of temporal window timestamps, addressing potential production issues.
2026-07-26 14:40:01 +03:00
Dmitry Ng 2d87a1f8fe feat(retries): implement callWithSetupRetries for transient error handling
- Introduced `callWithSetupRetries` function to enhance error resilience during LLM prompt calls, allowing for retries on transient errors with backoff.
- Updated `NewFlowProvider` and `NewAssistantProvider` methods to utilize `callWithSetupRetries` instead of direct calls to `prv.Call`, improving stability in flow and assistant creation.
- Added comprehensive unit tests for `callWithSetupRetries`, covering immediate success, transient error handling, and context cancellation scenarios.
- Enhanced error messages in Graphiti search tools to provide actionable feedback for missing or malformed parameters.
- Updated templates to clarify search type requirements and taxonomy references.
2026-07-25 13:30:12 +03:00
Dmitry Ng d9b28ce8bf chore(dependencies): update langchaingo version and adjust model configurations
- Updated the `langchaingo` dependency in `go.mod` from version `v0.1.15-0.20260723091023-35da5c0f0620` to `v0.1.14-update.6`.
- Modified the model configurations in `config.yml` and `models.yml` to replace `claude-opus-4-8` with `claude-opus-5` for both the generator and refiner sections.
- Updated the model descriptions and pricing in `models.yml` to reflect the new `claude-opus-5` model and added new models `claude-sonnet-4-6` and `claude-haiku-4-5`.
- Adjusted the test report in `anthropic-report.md` to reflect the updated model names and improved average latencies across various tests.
2026-07-25 02:52:04 +03:00
Dmitry Ng 5a4f1f54dc feat(search): introduce internal analytics engine for web search
- Added configuration options for an optional internal analytics engine in `.env.example`, `docker-compose.yml`, and related files.
- Updated documentation to include details about the new internal analytics engine, which scrapes and summarizes pages as a fallback for the web_search tool.
- Enhanced the `ftester` to support testing the new internal engine alongside existing search engines.
- Adjusted various components to integrate the new engine, including configuration forms and validation logic.

This feature allows users to enable a browser-based fallback for analytic queries, enhancing the flexibility of the web search capabilities.
2026-07-25 01:16:18 +03:00
Dmitry Ng deaff9eafe chore(config): update launch.json and add new provider configurations
- Added new configuration options in `.vscode/launch.json` for vllm and hcnsec providers.
- Updated `deepinfra.provider.yml` to reflect changes in model configurations and pricing.
- Introduced `hcnsec.provider.yml` with detailed model settings and pricing for HCNSec.
- Adjusted various test reports to reflect updated success rates and latencies for different models.
2026-07-23 17:37:44 +03:00
Dmitry Ng c26cb0b41e feat(tools): add edit_file action with unified-diff support
- Added `edit_file` to the file tool; diffs are parsed and applied in-memory via `github.com/sergi/go-diff`, exported as `tools.ApplyUnifiedDiff`.
- Made the hunk header lenient (position numbers optional) for models that omit them, with fuzzy content-based matching as fallback.
- Tuned `FileAction` field descriptions (layered, minimal-token) to steer models toward including context lines, based on live model testing via `ctester`.
- Added a `file_edit` multi-turn read_file → edit_file test case in `pkg/providers/tester`, using PentAGI's real tool schema, wired into the `advanced` test group.
- Fixed a bug where a single stateful test-case instance was shared across agent types, leaking conversation history and failures between independent runs.
- Extended the mock provider with sequential responses for multi-turn scenarios; added ctester integration tests.
2026-07-23 17:19:22 +03:00
Dmitry Ng 796c7faf3d fix(agent): stop network and malformed-arg failures from hard-failing tools
Graphiti transport errors, an empty file path, and an omitted 'action' or
double-encoded 'questions' arg from the LLM all hard-failed the tool chain
and burned retries instead of degrading gracefully. Also: stopTaskTimeout
5s->60s (flow kept running after a false 500), and the routine "cookie
claim invalid" case now logs at Warn instead of Error.
2026-07-22 16:25:07 +03:00
Dmitry Ng 5b48fd9f8e Merge branch 'feature/frontend' into feature/next-release 2026-07-22 01:38:05 +03:00
Dmitry Ng 74c2663114 Merge branch 'integrate/open-prs-v2' into feature/next-release 2026-07-22 01:25:25 +03:00
Sergey KozyrenkoandClaude Opus 4.8 c635a9fe34 test(converter): cover per-agent extra_body round-trip
Guards the two-way mapping added with the extra_body field: a JSON object
survives GraphQL→pconfig→GraphQL intact, and an absent one stays nil rather
than becoming an empty map.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 04:03:42 +07:00
Sergey KozyrenkoandClaude Opus 4.8 cae0295e8b feat(providers): expose per-agent extra_body in the provider config
The backend already threaded a per-agent `extra_body` map into the LLM request
(pconfig.AgentConfig, openai.WithExtraBody), but it was reachable only from YAML
provisioning — the GraphQL type, the converter, and the settings form all
dropped it. Editing a YAML-provisioned provider through the UI therefore round-
tripped its config and silently stripped extra_body.

Add `extraBody: Map` to AgentConfig / AgentConfigInput, carry it through the
converter in both directions, and give each agent an "Extra Body" JSON field in
the provider form (validated as an object, serialized on load, parsed on save).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 03:57:23 +07:00
Dmitry Ng 2470f6a428 Merge pull request #364 from Osamaali313/fix/newbodypair-reverse-delete
fix(cast): delete pruned tool-call parts in reverse to avoid index shift
2026-07-18 22:00:58 +04:00
Dmitry Ng e772a5cd82 Merge pull request #373 from rakshith48/feat/firecrawl-search
feat(tools): add Firecrawl web search provider
2026-07-18 19:33:57 +04:00
Dmitry Ng 40705ebea8 Merge pull request #343 from mason5052/codex/issue-335-xsstrike-args
fix: avoid unsupported XSStrike flags in pentester prompt
2026-07-17 17:30:49 +04:00
Dmitry Ng af3df846ff Merge pull request #339 from mason5052/codex/issue-338-stream-zip-downloads
fix: stream ZIP downloads without buffering archives
2026-07-17 16:46:19 +04:00
Dmitry Ng 7bf8615b66 Merge pull request #328 from octo-patch/feature/upgrade-minimax-m3
feat: upgrade MiniMax default model to M3
2026-07-17 15:33:13 +04:00
Dmitry Ng 06f149246e Merge pull request #233 from salecharohit/feat/bedrock-config-path
Feat/bedrock config path
2026-07-17 15:08:06 +04:00
rramprakash bb44c9961d fix(firecrawl): omit sources from search request
Verified against the live /v2/search API: the response envelope only ever
populates data.web for this tool, and sources defaults to ["web"] when
omitted. The sources field carries a string-vs-object shape ambiguity across
the API/SDK/MCP layers (the Firecrawl MCP rejects string sources outright),
so dropping it removes a needless compatibility risk without changing
behaviour.
2026-07-16 12:53:11 +05:30
rramprakash b32dde8f01 fix(firecrawl): thread user query into summarizer prompt
The summarizer prompt asked the model to 'directly answer the user query'
but never rendered the query — Firecrawl's /v2/search response, unlike
Tavily's, doesn't echo the query back, so it was dropped when adapting the
prompt. Every summarized result (the common path, since scrapeOptions
requests markdown) was therefore a generic page summary instead of a
query-focused one.

Thread the query through search -> parseHTTPResponse -> buildFirecrawlResult
-> getSummarizePrompt and restore the USER QUERY line, matching Tavily.
Add a test asserting the query is present in the prompt.
2026-07-16 12:53:11 +05:30
rramprakash 7ffa30ddc9 address review: bound & align firecrawl summarizer prompt, fix docs
- Summarizer prompt now uses one-based source IDs (aligned with the numbered
  links) and the metadata-fallback title/URL, and truncates each document's
  markdown to the same per-result limit as the non-summarized path, so an
  oversized or hostile page can't blow up the prompt.
- Reword tool description, searcher prompt and docs to say 'main-content
  markdown' since the scrape uses onlyMainContent=true.
- config.md example now uses the real NewFirecrawlTool(...) constructor,
  matching the searxng/sploitus entries.
2026-07-16 12:53:11 +05:30
rramprakash 3f68960d98 feat(tools): add Firecrawl web search provider
Add Firecrawl as a network search provider alongside Tavily, Perplexity,
Traversaal, SearXNG and the others. Firecrawl's /v2/search endpoint combines
web search with page scraping, returning full-page markdown content for each
result, which the searcher agent summarizes like the other content-rich
engines.

- New firecrawl tool mirrors the Tavily provider layer-for-layer: registry
  definition, tool-type mapping, factory wiring in both tool builders, and
  ftester executor/args/mocks support.
- Configurable endpoint: FIRECRAWL_API_KEY plus an optional FIRECRAWL_API_URL
  (default https://api.firecrawl.dev) so self-hosted Firecrawl instances work.
- Adds 'firecrawl' to the searchengine_type enum (goose migration + models)
  and the search-log server model so searches are recorded per engine.
- Installer wizard, .env.example, docker-compose, prompts and docs updated.

The tool is only registered when FIRECRAWL_API_KEY is set, so existing
deployments are unaffected.
2026-07-16 12:53:11 +05:30
Sergey KozyrenkoandClaude Opus 4.8 2853c97549 feat(reasoning): add three-state On/Off/Default reasoning control
Add an explicit reasoning Off to the per-agent config, backed by the langchaingo
tri-state API. Off emits llms.WithReasoningDisabled() (the provider disable wire)
via BuildOptions, and UsesAdaptiveThinking is guarded so Off overrides the
adaptive-only auto-adaptive.

Per-model capability is derived at runtime from llms.ReasoningSupportFor and
surfaced through GraphQL ModelReasoningInfo, so the UI only offers Off where it
actually disables: cannotDisable reports when Off would be rejected (always-on
models) OR a silent no-op (an unclassified default-on model whose disable wire is
omitted), and capability is surfaced for any thinking-capable model (e.g. Gemini,
which declares thinking without a reasoning block). A new
ProviderType.ReasoningProvider() supplies the provider to the resolver.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 19:50:43 +07:00
Sergey KozyrenkoandClaude Opus 4.8 ffbbabb517 fix(docker,flows): surface truncated/systemerr exec streams, bound ftester drain, tidy failure logging
- demuxExecStdout errors on a header cut short mid-frame (was a silent EOF that
  dropped the tail) and surfaces a docker systemerr frame instead of discarding it.
- ftester drains telemetry on exit through the bounded observer.Drain instead of
  two unbounded ForceFlush calls, so an unreachable collector can't hang it at exit.
- container-listing failures are sorted for a deterministic skipped-entries preview,
  logged per-entry at Debug (the detail is already in the response and the endpoint
  is hit on every navigation) with names quoted so control bytes in a hostile
  filename can't inject into a log line; corrected the stat-failure comment
  (a dangling symlink lstats fine, it doesn't fail).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 00:06:21 +07:00
Sergey KozyrenkoandClaude Opus 4.8 7cd22ccbc9 test(docker,flows): cover the listing cap paths, bound the container-list path count
The truncation slice and the demux stdout byte-cap had no docker-layer tests —
only the handler's Truncated wiring was exercised through the fake, so a
mis-slice or a dropped cap would have gone unnoticed. Extract find-output
parsing into a pure parseFindEntries and take the byte cap as a demuxExecStdout
parameter, then unit-test both boundaries (at cap / cap+1 / over-limit stream).

Also bound how many paths one container-files request may list
(maxContainerListPaths), so the per-path entry cap can't be multiplied by an
attacker-chosen path count into a large fan-out or response body.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 23:22:33 +07:00
Sergey KozyrenkoandClaude Opus 4.8 18234f05a3 docs(docker): refresh the container-listing doc, drop a stale telemetry test comment
The container-listing section still described the removed `ls -1` parse,
`pkg/queue`, and the old `[]container.PathStat` return. Update it to the current
`find -print0` + errgroup fan-out returning `ContainerDirListing{Files, Failures,
Truncated}`. Also remove the stale `WithBlock`/`DefaultDialTimeout` paragraph left
above the non-blocking `grpc.NewClient` test — its own next paragraph already
contradicted it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 22:45:34 +07:00
Sergey KozyrenkoandClaude Opus 4.8 7b67b1c0ea fix(flows): return partial container listings instead of failing; stop the refetch skeleton flash
Container-listing polish on top of the partial-success work:

- A directory whose entries all fail to stat no longer 500s. The find exec
  already proved the container alive, so ListContainerDir returns the readable
  entries (possibly none) plus the per-entry failures, and the handler serves a
  200 partial listing; only a cancelled request or a dir that can't be listed at
  all still errors.

- Over the entry cap, list the first page and set a Truncated flag rather than
  erroring with end-user copy from the docker layer. The flag flows through to
  the Pull dialog, which now warns the user the directory was truncated instead
  of silently showing a subset.

- The Pull dialog no longer flashes a skeleton on an in-place refetch (Refresh,
  or after a Pull): the listing hook clears its rows only when the path actually
  changes, so a same-path reload keeps the current rows under the loading guard.

Tests: an all-entries-failed listing returns 200 with the failures; the
Truncated flag surfaces in the response. Swagger regenerated for the new field.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 22:38:17 +07:00
Sergey KozyrenkoandClaude Opus 4.8 2c524faea9 fix(observability,flows): drain telemetry sinks in parallel, stop leaking docker errors, dedup listing failures
Three defects on the shutdown and container-listing paths:

- Telemetry drain ran the langfuse and otel sinks serially under one deadline.
  langfuse's ForceFlush/Shutdown ignore the caller ctx and block on their own
  timeout, so a langfuse outage burned the whole budget and the healthy otel
  sink's final batch was never flushed. Add observer.Drain, which flushes then
  shuts down each sink in its own goroutine bounded by the deadline; main uses
  it, retiring the inline flush-then-shutdown dance.

- The partial-success listing put the raw stat/list error into the 200 body's
  Failures[].Message, exposing docker-layer detail (container ids, the daemon
  address) that response.Error only reveals under develop mode. Send a generic
  message to the client and keep the raw text dev-gated, logged server-side.

- A path read successfully by one query could also appear in Failures when
  another query failed on it, order-dependent. Reconcile at assembly: drop any
  failure whose path was read successfully, so no path is in both arrays.

Also bound the exec stdout buffered before parsing so a compromised sandbox
can't stream unbounded output into memory.

Tests: Drain delivers a healthy otel sink while a langfuse sink blocks and
returns within its deadline; the listing failure message is dev-gated; a path
never lands in both Files and Failures. Live: unreachable collector -> shutdown
stays bounded (~5s) and exits cleanly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 22:24:02 +07:00
Sergey KozyrenkoandClaude Opus 4.8 bc8714db6e fix(observability): flush telemetry on graceful shutdown, drain every sink
The graceful-shutdown branch only logged "cleaning up resources" and exited
without flushing telemetry, so the last buffered batch was dropped on every
SIGTERM. Drain it, with three corrections:

- main: flush then shut down the observer on shutdown, bounded by a 5s
  deadline in a goroutine so an unreachable collector can't stall exit.
  Flush must precede Shutdown because the langfuse observer's Shutdown
  cancels its context without flushing; its final batch drains only via
  ForceFlush while the context is still live.
- observer Flush/Shutdown: drain both the langfuse and otel sinks and join
  their errors instead of returning after the first, which skipped the
  second sink entirely when both were configured.
- telemetryClient Flush/Shutdown: attempt all three providers and always
  close the grpc connection, joining errors, so a stalled provider flush
  no longer skips the rest or leaks the connection.

Tests: the langfuse observer test proves Shutdown drops the buffered batch
while ForceFlush drains it to the sink; observer-level tests prove both
sinks drain and errors aggregate. Live: with an unreachable collector,
shutdown stays bounded (~5s) and exits cleanly instead of stalling ~10s.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:30:31 +07:00
Sergey KozyrenkoandClaude Opus 4.8 c0660b8398 fix(observability): connect the telemetry collector lazily, tear down cleanly
The startup dial used grpc.WithBlock with a 10s timeout, so a set-but-
unreachable collector stalled main() for up to 10s, and once that dial failed
the client was nil forever — a collector that came up later never connected
without a restart. Switch to grpc.NewClient (non-blocking): startup never
stalls and gRPC re-establishes the connection in the background on its own.
Also build all three exporters before any provider, so the exporter-error
cleanup (closing the connection) happens before any batch/reader goroutine
starts and can't leak one. Drops the deprecated WithBlock/WithReturnConnectionError.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 16:45:41 +07:00
Sergey KozyrenkoandClaude Opus 4.8 345b91e4c1 docs(swagger): regenerate for the container-files Failures field
ContainerFiles gained a Failures list when the container listing switched to
partial-success; regenerate the Swagger docs (swag v1.8.7, the pinned version)
so generated clients see the new field.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 16:37:18 +07:00
Sergey KozyrenkoandClaude Opus 4.8 a306b156a1 chore: gitignore backend/vendor, drop stale pkg/queue doc row, self-contained test messages
- Ignore the local `backend/vendor/` tree (187 MB, present only for the
  vendored-fork build) so it can't be accidentally staged.
- Remove the architecture-table row for `pkg/queue/`, which was deleted when
  the container stat fan-out moved to errgroup.
- Drop the internal "H4" report-id labels from the telemetry test — the
  sentences already say what they guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 16:37:17 +07:00
Sergey KozyrenkoandClaude Opus 4.8 780c2a722e fix(docker): drop the TTY from the dir listing to keep newline filenames intact
find -print0 already delimits with NUL, but the exec still ran under a TTY,
whose onlcr translation rewrites every \n in the stream to \r\n — including a
\n that is part of a filename — so a file literally named with a newline was
stat'd under the mangled name and reported unreadable. Run the exec without a
TTY and demux the resulting multiplexed stdout/stderr stream by hand (no new
dependency), so filename bytes reach the parser untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 16:20:36 +07:00
Sergey KozyrenkoandClaude Opus 4.8 93d36c9827 fix(flows): keep serving readable paths when one container path fails
The multi-path container listing still fail-fasted: a single bad top-level
path (nonexistent, not a directory, list failed) returned HTTP 500 and
blanked every path that had already listed. Record each such path as a
failure and keep going so the readable paths are still returned; only a
cancelled request aborts, and if every path fails at the directory level the
request is failed as a whole. Also drop a per-entry failure whose path was
already read successfully by an overlapping query.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 16:12:40 +07:00
Sergey KozyrenkoandClaude Opus 4.8 ac9a16517a fix(docker): list container dirs with find -print0, not ls parsing
Parsing `ls -1` output was unsafe: under the TTY the exec allocates, GNU
coreutils shell-quotes names — so a readable file with a space, quote or
non-ASCII byte was stat'd with the quotes and reported unreadable — and
busybox wraps every entry in ANSI colour codes, so on an alpine/busybox
image the whole listing came back unreadable. Switch to `find -maxdepth 1
-print0`: literal bytes, portable across GNU and busybox, and the NUL
delimiter survives filenames containing newlines. Cap the entry count so a
directory with hundreds of thousands of files can't fan out into that many
Docker API calls, and classify a cancelled request or an all-entries-failed
listing as a directory-level error instead of a misleading empty 200.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 16:12:38 +07:00
Sergey KozyrenkoandClaude Opus 4.8 1300f60a2b feat(docker): partial-success container listing instead of fail-fast
A single unreadable directory entry (dangling symlink, a file removed
between ls and stat, a transient /proc entry) used to fail the whole
listing with HTTP 500, blanking the file browser and discarding every
readable sibling. The frontend already expects /proc/sys to not fail
spuriously, but the backend did the opposite.

ListContainerDir now returns a ContainerDirListing{Files, Failures}: per-
entry stat errors no longer abort the batch. GetFlowContainerFiles serves
the readable entries as HTTP 200, carries the failures back in a new
ContainerFiles.Failures field, and logs each skipped entry (capped) plus a
degradation summary. Directory-level faults (not a dir, ls failed, container
gone) still return 500 — there is no partial to show. statContainerEntries
returns successes + failures instead of the lowest-index error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 13:32:48 +07:00
Sergey KozyrenkoandClaude Opus 4.8 8212802c84 fix(docker): clamp non-positive worker count in the stat fan-out
The deleted queue normalized workers<=0 to a default; the errgroup helper
passed the count straight to SetLimit, where 0 deadlocks the first Go() on an
unbuffered semaphore and a negative value disables the bound entirely. The
sole caller hardcodes 20 so this was latent, but clamp it back so a future
config-driven count can't hang the listing or flood the Docker daemon.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 03:03:57 +07:00
Sergey KozyrenkoandClaude Opus 4.8 40eeea93c0 refactor(docker): replace the custom queue with a bounded errgroup fan-out
ListContainerDir was the only consumer of pkg/queue and discarded its ordered
delivery (the caller dedups and re-sorts). Replace the fan-out with a small
errgroup helper — SetLimit keeps the same 20-way bound on concurrent stats
against the Docker daemon, the caller's context propagates into each call,
results stay in input order, and the lowest-index failure is surfaced
deterministically — then delete pkg/queue. The helper was verified byte-
identical to the previous queue implementation across 600 randomized directory
shapes (sizes, failure positions, out-of-order completion) before removal.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 02:05:24 +07:00
Sergey KozyrenkoandClaude Opus 4.8 d44a33f5dd test(observability): cover the telemetry success path
Adds a fake OTLP collector so NewTelemetryClient can be exercised against a
reachable endpoint: it builds all three providers and both ForceFlush and
Shutdown export and tear down without error. Previously only the
not-configured and unreachable-dial paths were covered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 10:02:58 +07:00
Sergey KozyrenkoandClaude Opus 4.8 745c8ae6d6 test(queue): cover the default-workers, nil-process, and Instance paths
Adds scenarios for a non-positive worker count (must fall back to the
default, else nothing drains the queue and delivery hangs), a nil process
function (logs and drops each item without panicking), and a stable
instance id. Brings queue.go to full statement coverage and kills the
mutants that previously survived on the workers<=0 fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 10:02:56 +07:00
Sergey KozyrenkoandClaude Opus 4.8 d49e740821 test(observability): cover dial-timeout + ErrNotConfigured; close conn on exporter error
Unit coverage for NewTelemetryClient: an unset OTEL_HOST yields ErrNotConfigured,
and an unreachable collector returns within the dial timeout instead of hanging
(a TCP-accepting, silent listener drives the WithBlock path).

Also close the grpc.ClientConn on the exporter-creation error paths — a
successful dial followed by a failed exporter New() previously leaked it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 09:09:38 +07:00
Sergey KozyrenkoandClaude Opus 4.8 62a1445d32 refactor(queue): drop the running-context for a flag; reject restart-without-stop
The two-context design in ddd3916 was correct but fragile: the field named
`ctx` was the liveness signal goroutines must NOT bail on, while the real
cancellation signal was `stopCtx`. That inversion is the exact trap that
produced the earlier happy-path regression (bailing on `ctx`, which a normal
input-close cancels, dropped in-flight results). It also left a latent
restart-without-Stop hazard: a second Start() after input-close orphaned the old
stopCtx and shared wg, deadlocking a later Stop() (with a data race).

Collapse `ctx` into a plain `running` bool (it was only ever read via .Err(),
never awaited), keep `stopCtx` as the single cancellation signal, and guard
Start() on stopCtx so a restart requires a prior Stop() — removing both the
naming trap and the restart hazard.

Comprehensive tests (queue_scenarios_test.go): boundary-N full delivery,
contiguous-prefix-on-abandon, a randomized prefix-invariant fuzz, goroutine-leak,
restart-reject, process-error, double-stop, Running() transitions. The happy-path
tests hang on the pre-rework code and pass here (side-by-side under -race). Live:
a flow's non-empty container dirs of 1/40/100/500 files each deliver every entry;
the error path returns 500 without hanging.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 09:09:36 +07:00
Sergey KozyrenkoandClaude Opus 4.8 ddd3916e9c fix(queue): only hard-stop workers on Stop(), not on normal input-close
The previous fix (97e5730) had workers and the reader bail on q.ctx, but the
reader also cancels q.ctx on a normal input-close — so the ListContainerDir
happy path regressed: after input closed (buffer names, close, read all N),
workers dropped still-undelivered results and the consumer hung waiting for the
last one. Any non-empty container directory that didn't error early hung the
file-manager request.

Split the signals: q.ctx still tracks "running" (cancelled by input-close or
Stop), and a new q.stopCtx is cancelled only by Stop(). Workers and the reader
bail on q.stopCtx, so a normal input-close drains and delivers every result
while a hard Stop() still unblocks a consumer that abandoned output. Stop()'s
already-stopped guard now checks q.stopCtx (input-close alone must not
short-circuit it, or blocked workers leak).

Tests: DeliversEveryResultAfterInputClose (hangs on the old fix, verified
side-by-side under -race) + StopHardStopsAfterInputClose; deadlock + ordering
still green under -race.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 00:02:51 +07:00
Sergey KozyrenkoandClaude Opus 4.8 d6fa58e5d4 fix(observability): don't let an unreachable telemetry collector down the app
A set-but-unreachable OTEL collector hung startup: NewTelemetryClient dialed
with grpc.WithBlock() and no timeout on the deadline-free signal context, so
the process blocked forever before it ever served. And a non-ErrNotConfigured
init error from either observability client was log.Fatalf, killing the
process. Both let an OPTIONAL integration take the whole app down.

Bound the dial with a 10s timeout, and on init failure degrade to a no-op
observer with a logged warning instead of exiting. Verified live: a bad
OTEL_HOST that hung startup indefinitely now boots in ~14s with a warning and
runs a full flow; a reachable collector still boots in ~4s with no warning.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 23:20:43 +07:00
Sergey KozyrenkoandClaude Opus 4.8 97e5730a53 fix(queue): don't deadlock Stop() when the consumer stops reading output
ListContainerDir aborts on the first stat error and stops reading the queue's
output channel, leaving workers blocked on the unbuffered `q.output <- result`
send and the reader blocked on a full `q.queue`; Stop() -> wg.Wait() then hung
forever. Select every pipeline send/wait on q.ctx so a stopped queue unwinds.
On stop a worker returns without msg.cancel(), so later workers also bail via
q.ctx and output ends at a contiguous prefix instead of developing gaps.

Regression test drives the exact hang (unread output -> Stop must return),
verified with -race.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 21:29:07 +07:00
Sergey KozyrenkoandClaude Opus 4.8 9876c13ea8 docs(providers): restore the DeepSeek legacy reasoning-format contract
The openai-compat consolidation dropped the comment warning that DeepSeek needs
the legacy top-level "reasoning_effort" string form; without it a maintainer
could add openai.WithModernReasoningFormat() to the shared opts and silently
break DeepSeek thinking mode. Restore it at the shared opts choke point.

Also fix two stale pointers: glm/qwen config.yml cited
WithPreserveReasoningContent() "in glm.go/qwen.go", but it moved to
openaicompat.go during that same consolidation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 17:02:21 +07:00
Sergey KozyrenkoandClaude Opus 4.8 f5f7974826 test(validation): cover REST prompt validation and knowledge length limits
- prompts_test.go: PatchPrompt rejects a syntax error, an undeclared variable,
  and a whitespace-only template over REST (the path a raw client hits when the
  UI is bypassed) and does not persist them; valid templates still create/update.
- validation_test.go: validateKnowledgeFieldLengths accepts each field at its
  max length and rejects one character over.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 16:34:04 +07:00
Sergey KozyrenkoandClaude Opus 4.8 72da224033 feat(prompts): validate template syntax on the REST update endpoint
The GraphQL createPrompt/updatePrompt mutations run validator.ValidatePrompt
(Go text/template parse + declared-variable check + trial render), but the
REST PUT /prompts/:type handler only checked the field was present, so a
prompt with a syntax error or an undeclared variable could be stored over
REST and later break rendering. Mirror the GraphQL check in PatchPrompt.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 16:01:36 +07:00
Sergey KozyrenkoandClaude Opus 4.8 0cf85ef029 feat(graphql): validate knowledge and API-token mutation inputs
The REST handlers reject empty or oversized knowledge fields and over-long
API-token names through their request-model validate tags, but the GraphQL
mutations — the path the web UI uses — accepted them unchecked, so the same
entity could be stored past its documented limits depending on the caller.

Mirror the REST caps at the resolver boundary:
- createKnowledgeDocument / updateKnowledgeDocument: require content (and
  question on create), and cap content/question/description/codeLang lengths.
- createAPIToken / updateAPIToken: cap the token-name length.

Limits are kept in sync with server/models via mirror comments.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 15:44:19 +07:00
Sergey KozyrenkoandClaude Opus 4.8 bcef80de99 fix(graph): reject empty flow title, flow input, and assistant input
The renameFlow / putUserInput / callAssistant GraphQL resolvers passed the
title/input straight to the controller with no non-empty check, while the
equivalent REST handlers reject them with 400. An empty flow title in
particular then fails the Flow model's `required` invariant and breaks the
REST GET /flows listing for that user. Mirror the REST guard at the resolver
boundary, matching the existing createFlow "... is required" checks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 15:16:33 +07:00