Commit the ZIP response status and headers lazily, on the first byte written, via a small zipStreamWriter. A build failure before any output now returns the normal structured error response instead of a committed 200 with a truncated body; mid-stream failures still abort. Also check the reader Close error in the helper test and cover the pre-stream failure path.
DownloadResource and DownloadFlowFile built the entire ZIP archive in a bytes.Buffer before sending it, so heap usage scaled with archive size and a few concurrent large-directory downloads could exhaust process memory.
Stream the archive straight to the response writer via a shared streamZipArchive helper; the existing ZipResources/ZipDirectory/ZipRelativePaths helpers already accept an io.Writer. Responses are now chunked (no Content-Length) and memory stays proportional to one file's copy buffer.
Rename a knowledge document via a dedicated mutation that rewrites only the
question in cmetadata — no re-embedding, no embedder required — mirroring the
flows renameFlow pattern instead of round-tripping the full document through
updateKnowledgeDocument.
Backend:
- renameKnowledgeDocument(id, question) mutation + resolver (admin/user split;
ownership enforced at GetUserDocument, like the update pair)
- metadata-only query UpdateKnowledgeDocumentMetadata (no migration)
- unit + edge tests: metadata-only, missing-doc error, non-owner rejection
Frontend:
- renameKnowledge provider method; wire list and detail inline-rename to it
- drop the content "Preview" column and request the list with withContent:false
so it no longer pulls full document bodies
Verified end to end against a local Docker backend (rename works; content and
embedding preserved) and against the remote backend (graceful failure where the
mutation is not yet deployed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ChangeNameCurrentUser issued a bare UPDATE and ignored RowsAffected, so a stale
session for a deleted user got 200 — inconsistent with the email/password handlers
(which 404) and leaving the frontend's Users.NotFound mapping unreachable.
- check RowsAffected and return ErrUsersNotFound (404) when no row matched
- document the 404 in the swagger annotation
- add TestChangeNameCurrentUser (success, missing user, invalid name)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The vmail validator rejected addresses the frontend's z.string().email() accepts
(uppercase, TLDs longer than 4 chars like .cloud), so users saw a confusing 400.
- relax the vmail regex to allow uppercase and TLDs of 2+ chars
- lowercase + trim the new address in ChangeEmailCurrentUser and in the form schema
so storage, the duplicate check, and login lookups stay case-stable
- validator + change-email tests for mixed case and long TLDs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- authLoginCallback matches users by email alone so an OAuth login links into an
existing (incl. local) account instead of 500-ing on users_mail_unique
- relink on a create-branch unique-violation race instead of returning 500
- issue the session with the linked account's actual role privileges
- clear the stale OAuth provider link when a user changes their email
- map the email-change unique-violation race to 409 instead of 500
- isUniqueViolation matches Postgres and SQLite case-insensitively
- tests for link/create/blocked/role-inheritance/race, email 409, provider reset
- update auth form test selectors after the form refactor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- OAuth login callback now stores the provider (google/github) on the
user record and backfills it for pre-existing OAuth users on login,
so the UI can show which provider an account is linked to.
- Add PUT /user/name for any authenticated user (including OAuth): a
NameChange model + ChangeNameCurrentUser handler that lets users edit
their display name. Placed in a /user group without localUserRequired.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two crashes seen in production, both reproduced on the live app:
- "Failed to fetch dynamically imported module": after a redeploy rotates the
hashed chunk filenames, an open tab imports a deleted one. The server answered a
missing /assets/* with 301 -> index.html (HTML for a JS module -> a MIME
failure); it now returns 404 + no-store. The client listens for Vite's
vite:preloadError and reloads once (debounced) to pull the current build. Hashed
assets are served immutable; index.html and SPA routes no-cache.
- "Failed to execute 'removeChild' ... not a child of this node": an external
agent (a browser extension or auto-translation) mutates the DOM React owns,
desyncing reconciliation. A root react-router errorElement catches this
commit-phase crash and self-heals with a debounced reload, instead of React
Router's dead default error screen. translate="no" opts the English-only UI out
of the one trigger it can prevent (browser translation); the errorElement covers
the rest regardless of source.
Verified by reproducing both on the live old build (missing-chunk 301->HTML;
extension/translation DOM mutation -> the exact removeChild crash) and confirming
the fixed build recovers from each. Adds chunk-reload + RouteErrorBoundary unit
tests and a static-serving integration test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove/Purge/FactoryReset failed with "file does not exist" for stacks
that were never extracted (no docker-compose-*.yml). Now such stacks are
skipped instead of aborting the whole operation.
- Add composeFileExists check before destructive compose commands
- Add isDestructiveComposeOperation helper
Problem:
In the installer processor, post-operation state refresh failures from
checker.GatherUpdatesInfo were silently discarded, so apply/install/
update/remove/purge could report success even when the refresh failed.
Root cause:
Two distinct shadowing/fall-through patterns wrapped the error into a
variable that was never returned:
- In the deferred closures of applyChanges/install/update/purge, the
inner `if err := GatherUpdatesInfo(ctx); err != nil` shadowed the
named return `err`, so `err = fmt.Errorf(...)` assigned the shadow and
the wrap never reached the `sendCompletion(stack, err)` defer.
- In the ProductStackWorker cases of remove/purge, the block-local err
was wrapped but the case fell through without returning, while every
sibling check in the same switch returns on error.
Solution:
- Deferred closures: wrap into a `gatherErr` local and assign the named
return `err`, so the failure propagates to sendCompletion.
- Switch cases: `return` the wrapped error, matching the sibling checks.
Testing:
go build ./cmd/installer/... -> OK
go vet ./cmd/installer/processor/... -> OK
go test ./cmd/installer/processor/... -> ok
golangci-lint (ineffassign) on logic.go -> clean
Fixes#312
Root cause:
During flow creation the first LLM call selects the primary Docker
image. When the configured LLM backend is unsupported or returns 404,
prv.Call fails, but the wrapped error read "failed to get primary
docker image". Users repeatedly interpreted this as a Docker problem
(see #312, #309, #203) and debugged Docker instead of their LLM
provider configuration.
Solution:
Reword the wrapped error to "failed to select primary docker image
via llm call" so the LLM provider is identified as the failing
component while still mentioning the image selection step.
Testing:
go build ./pkg/providers/ -> OK
Added new test cases to validate containment barriers in the `ResolvePulledStagedTarget` and `ZipRelativePaths` functions, ensuring that paths escaping the designated directories are properly rejected. Updated the `knowledge_test.go` to include scenarios for handling nil embedder and error propagation during document creation, improving overall test coverage and robustness.
Enhanced the flowWorker's task processing by introducing a cancellable context for task execution. This change ensures that tasks can be properly cancelled without reporting false success states. The `runTask` method has been refactored to utilize a new `execTask` method, which centralizes task execution logic and maintains context integrity. This update improves flow control and error handling during task creation and execution.
Updated the Qwen agent configuration to include `extra_body` parameters for thinking control across various models. Added `enable_thinking` and `preserve_thinking` options for reasoning agents, while utility agents have `enable_thinking` set to false. Adjusted the Qwen client initialization to support these new configurations. Updated test report to reflect changes in success rates and latencies.
Issue #310 asks how to provide a Google Vertex AI API key in .env for
Anthropic Claude. PentAGI currently has no dedicated Vertex AI provider
path in code: backend/pkg/config and backend/cmd/installer do not read
VERTEX_API_KEY, GOOGLE_APPLICATION_CREDENTIALS, or any vertex_ai
variable. The supported routes for Claude today are direct Anthropic
(ANTHROPIC_API_KEY / ANTHROPIC_SERVER_URL) and AWS Bedrock (BEDROCK_*).
Document this explicitly so users do not assume a hidden Vertex AI
configuration path exists:
- README.md: add a NOTE callout inside the Anthropic Provider
Configuration section listing the supported routes and pointing
users who need Vertex AI today at the OpenAI-compatible custom LLM
provider path (LLM_SERVER_URL / LLM_SERVER_KEY / LLM_SERVER_MODEL)
fronted by a translating gateway, with a caveat that reliability
depends on the gateway.
- backend/docs/config.md: add a matching Note paragraph under the
Anthropic section that points at the AWS Bedrock and custom LLM
provider sections, and states that no VERTEX_API_KEY or
GOOGLE_APPLICATION_CREDENTIALS variable is wired into provider
initialization today.
Docs-only change. No runtime Go code, no installer behavior, no
generated files, no new environment variables. All env var names cited
in the new text already exist in the current PentAGI .env.example,
backend/pkg/config, and backend/cmd/installer.
DeepSeek V4 thinking mode defaults to enabled on both deepseek-v4-flash
and deepseek-v4-pro; non-thinking behavior requires an explicit toggle.
Per official docs, in thinking mode temperature/top_p/presence_penalty/
frequency_penalty are ignored, so the existing Flash role sampling knobs
would have been silently no-ops without the toggle.
Add extra_body.thinking.type=disabled to the five non-thinking Flash
roles so deepseek-v4-flash actually runs in non-thinking mode and
honors the role's temperature/top_p settings:
- simple, simple_json, adviser, searcher, enricher
Pro roles (primary_agent, assistant, generator, refiner, reflector,
coder, installer, pentester) intentionally keep thinking enabled (the
V4 default) for reasoning, tool-use, and security analysis.
PentAGI provider config already supports extra_body as a first-class
yaml field on AgentConfig and forwards it through openai.WithExtraBody,
which the vxcontrol langchaingo fork serializes at the top level of the
Chat Completions request - the same pattern Kimi uses for tool_choice.
No code, schema, or LiteLLM prefix changes required.
Touches:
- backend/pkg/providers/deepseek/config.yml (embedded production config)
- examples/configs/deepseek.provider.yml (user-facing example)
No change to role-to-model mapping, model metadata, pricing, README
wording, LiteLLM prefix, unrelated providers, lifecycle, queues, or
installer flow.
- Update model descriptions to reflect V4 1M context window (up to 384K output)
instead of legacy 128K wording in models.yml and README.
- Split Flash and Pro pricing per official DeepSeek API docs:
- deepseek-v4-flash: input 0.14 / output 0.28 / cache_hit 0.0028 per 1M tokens
- deepseek-v4-pro: input 0.435 / output 0.87 / cache_hit 0.003625 per 1M tokens
- Apply per-role price split across all 13 role configs in both the embedded
config.yml and the user-facing examples/configs/deepseek.provider.yml.
- Replace stale "cache pricing is 10% of input cost" claim in the README,
which no longer holds for either V4 model.
- No change to LiteLLM prefix behavior, role-to-model mapping, lifecycle,
queues, GraphQL schema, migrations, frontend, or installer flow.
- README: align supported-models intro with the local convention used
by every other provider section ("Models marked with `*` are used
in default configuration"), so the asterisk on each model ID has a
near-by explanation.
- Installer help (`LLMFormDeepSeekHelp`): swap the legacy
"DeepSeek-Chat" / "DeepSeek-Reasoner" bullets in "Default PentAGI
Models" for the current `deepseek-v4-flash` / `deepseek-v4-pro`
defaults so the wizard guidance matches the bundled config.
No code, schema, or LiteLLM prefix behavior changes.
The DeepSeek provider config still defaulted to the legacy
`deepseek-chat` and `deepseek-reasoner` model names, which the
upstream DeepSeek API has announced for deprecation on 2026-07-24.
A first-run install therefore breaks once the legacy names are
removed.
Swap the defaults to the current DeepSeek V4 family:
- non-thinking roles use `deepseek-v4-flash`
- reasoning-heavy roles use `deepseek-v4-pro`
The change is limited to the embedded `config.yml` / `models.yml`
inside `backend/pkg/providers/deepseek`, the matching example at
`examples/configs/deepseek.provider.yml`, the `DeepSeekAgentModel`
fallback constant in `deepseek.go`, and three doc references
(README.md, backend/docs/config.md, backend/docs/llms_how_to.md)
plus one installer help string in
`backend/cmd/installer/wizard/locale/locale.go`. LiteLLM prefix
behavior is untouched.
- Introduced `EMBEDDING_MAX_TEXT_BYTES` to limit the maximum byte size of text sent to the embedding model.
- Renamed database connection pool settings: `DATABASE_MAX_OPEN_CONNS`, `DATABASE_MAX_IDLE_CONNS`, and `DATABASE_VECTOR_MAX_CONNS` for improved PostgreSQL connection management.
- Updated relevant documentation to reflect these new configuration options and their usage.
- Adjusted various components to utilize the new settings for enhanced performance and resource management.
- Added ToolCallLogProvider interface with methods for logging tool calls, updating success and failure statuses.
- Introduced proxyToolCallLogProvider to handle ToolCall logging operations.
- Updated flow execution components to integrate ToolCall logging, including flow workers and controllers.
- Enhanced GraphQL schema to support ToolCall logs, including queries and subscriptions for real-time updates.
- Updated documentation to reflect the new ToolCall logging features and their usage.
- Introduced shared connection pooling for PostgreSQL using `*sql.DB` for sqlc and GORM, optimizing resource usage.
- Added new environment variables: `DB_MAX_OPEN_CONNS`, `DB_MAX_IDLE_CONNS`, and `DB_VECTOR_MAX_CONNS` for configurable connection limits.
- Updated documentation to reflect new connection pooling strategy and provide operational commands for monitoring.
- Implemented shared `pgxpool` for pgvector stores to reduce connection overhead and improve performance.
- Adjusted various components to utilize the new connection pooling setup, ensuring efficient database interactions.
- Introduced WaitTaskCompletion method in FlowWorker interface to block until the current task completes or the context expires.
- Implemented signalTaskComplete to manage task completion signaling across goroutines.
- Added waitFlowCompletion tool to handle waiting for task completion with configurable timeout.
- Updated assistant provider to include wait functionality for flow completion.
- Enhanced templates and tool registry to support new wait functionality.
When an LLM emits a truncated or malformed arguments field (e.g. "{") instead of valid JSON, downstream consumers such as LiteLLM reject the entire request with a 400 Bad Request, causing the chain to degrade into an infinite retry loop.
- SanitizeToolCallArguments now falls back to "{}" after control-char escaping if the result is still not valid JSON, fixing already-stored chains on restore
- callWithRetries logs a warning and substitutes "{}" at the moment the bad arguments arrive from the LLM, preventing them from ever reaching the database
- Added test cases for truncated JSON, partial objects, and empty strings
- Added a new SQL migration to insert toolcall privileges into the privileges table.
- Introduced the `ToolcallService` for managing toolcall data, including retrieval of toolcalls and flow-specific toolcalls.
- Implemented API endpoints for fetching toolcalls and toolcall details, with appropriate permission checks.
- Enhanced Swagger documentation to include new toolcall endpoints and their specifications.
- Created a new model for toolcalls, defining their structure and validation rules.
- Added error handling for invalid toolcall requests and not found scenarios.
- Added a new SQL migration to insert the 'anonymize.call' privilege into the privileges table.
- Introduced the `anonymizeText` mutation in the GraphQL schema, allowing users to anonymize sensitive text.
- Implemented the `AnonymizerService` to handle text anonymization requests via a REST API endpoint.
- Updated the GraphQL resolver to integrate the new mutation and ensure proper permission checks.
- Enhanced documentation with Swagger and OpenAPI specifications for the new endpoint.
- Added error handling for invalid requests and unavailable anonymizer configurations.
Eliminate the message length truncation from both `putMsg` methods in `aslog.go` and `msglog.go`. This change simplifies the message handling process by allowing messages to be processed without arbitrary length restrictions.
- Fix empty ID bug: langchaingo SimilaritySearch discarded document UUIDs; new SearchKnowledgeDocuments/SearchUserKnowledgeDocuments return them directly
- Remove unsafe fmt.Sprintf SQL filter interpolation, use parameterised queries
- Exclude memory documents from search results at SQL level
- Add FlowID support to passesSearchFilter
- Convert all $N positional params to sqlc.arg(name) across knowledge queries
- Update tests: replace TestBuildSearchFilters with TestPassesSearchFilter, add TestSearchDocuments and TestSearchUserDocuments
- Introduced a new `docType` field in the `UpdateKnowledgeDocumentInput` to manage document type changes.
- Updated the `doUpdate` method to clear subtype fields (GuideType, AnswerType, CodeLang) when the document type changes.
- Added comprehensive tests for various document type transitions to ensure correct behavior and state preservation.
- Updated GraphQL schema and generated models to accommodate the new `docType` field.
- Introduced "/resources" and "/knowledges" routes to the frontendRoutes array in the router configuration, expanding the application's routing capabilities.
- Removed the requirement for the "destination" field in MoveResourceRequest, allowing it to be an empty string, which signifies moving to the root directory.
- Updated related documentation in swagger.json, swagger.yaml, and docs.go to reflect the new behavior.
- Adjusted the MoveResource function to handle cases where the destination is empty, ensuring proper path sanitization and resource movement semantics.
- Enhanced test cases to cover scenarios involving moving resources to the root directory and handling conflicts appropriately.
- Allow POST /flows/0/assistants/ to create a new flow together with the assistant, mirroring the existing GraphQL createAssistant(flowID: 0) behavior
- Require both assistants.create and flows.create permissions when flowID=0
- Add explicit flow ownership check for non-zero flowID using flows.admin scope
- Load flow data in the response by fetching it via assistant.FlowID after creation, ensuring AssistantFlow is fully populated in all cases
- Introduced new fields `version` and `isDevelopMode` in the Settings model to provide application versioning and development mode status.
- Updated GraphQL schema and resolvers to support the new fields, ensuring they are accessible via the Settings query.
- Enhanced Swagger documentation to reflect the changes in the Settings API endpoint.
- Added necessary validation and response handling for the new fields in the Settings service.
Address Copilot review feedback on PR #305: the 'Current Limitations' bullet for the Graphiti integration mixed PentAGI's user-facing .env variables with the container env vars defined in docker-compose-graphiti.yml.
Reword both README.md and backend/docs/config.md to lead with the user-facing OPEN_AI_KEY and OPEN_AI_SERVER_URL .env variables and explicitly note that docker-compose-graphiti.yml maps them into the bundled vxcontrol/graphiti container as OPENAI_API_KEY and OPENAI_BASE_URL. Operators set the .env variables; the container variables are an implementation detail.
Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
The Graphiti container shipped with docker-compose-graphiti.yml only
takes OPENAI_API_KEY and OPEN_AI_SERVER_URL for entity extraction.
PentAGI configures many other LLM providers (Anthropic, Google AI,
AWS Bedrock, DeepSeek, GLM, Kimi, Qwen) for the main flow, but those
credentials are not consumed by Graphiti today. Until that changes,
operators need to plan around an OpenAI-compatible endpoint just for
the knowledge graph.
This commit makes the limitation visible in two surfaces without
changing runtime behavior:
- README.md: Adds a beta callout at the top of the Knowledge Graph
Integration section and a new 'Current Limitations' subsection
covering provider scope, fixed model, independent billing, and the
lack of an in-app graph explorer.
- backend/docs/config.md: Mirrors the beta callout under Graphiti
Knowledge Graph Settings and adds a 'Current Limitations (Beta)'
subsection with the same constraints, so config-focused readers
see the same message.
Both notes explicitly point at the simple fallback: leave
GRAPHITI_ENABLED=false if the deployment cannot reach an
OpenAI-compatible endpoint.
Refs #187
Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
Address two reviewer concerns on the issue #300 follow-up:
* Replace the `DumpTemplates()` -> `json.Unmarshal` round-trip in
buildUserPrompter with a new `templates.LoadDefaultPromptsMap()`
helper that returns the embedded defaults as a `PromptsMap`
directly. `defaultPrompter.DumpTemplates()` now delegates to the
same helper, so the JSON output for that API is unchanged.
* Add an optional `prompter` field to `assistantWorkerCtx`. When
set, `LoadAssistantWorker` reuses it instead of re-querying
`GetUserPrompts` and re-merging defaults. `LoadFlowWorker`
populates it once per flow load so multi-assistant flows pay the
DB+merge cost a single time.
Tests updated to match the new pure-merge `buildUserPrompter`
signature; behavior for users with no overrides is unchanged.
Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
Address review on PR #303.
- helpers.go: remove the concrete model examples (llama3.1, qwen2.5, mistral-nemo) from the Ollama tools-unsupported hint. PentAGI does not verify Ollama model capabilities from upstream metadata, so a hard-coded list of supposedly tool-capable tags is risky and ages poorly. The hint now points users to select an Ollama model whose own metadata advertises tool/function calling support.
- helpers.go: rewrite the comment and error wording so it is no longer flow-only. The helper is shared by NewFlowProvider and NewAssistantProvider, so the message now refers to tool execution in flows and assistant sessions, and the doc-comment notes that wording must stay context-neutral.
- helpers_test.go: tighten TestWrapToolCallIDTemplateError to match the new wording, ban the previous concrete model tags, and ban flow-only language ('flow execution', 'flow creation'). The test still verifies that the original underlying error is wrapped via errors.Is.
Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
## Summary
Surface an actionable hint when an Ollama model that does not support
tool/function calling is selected for a flow or assistant session, so
users do not have to parse a five-wrap-deep "failed to determine tool
call ID template" stack trace to figure out why flow creation fails.
## Problem
A user on an Ollama deployment of `gemma3:27b-it-q4_K_M` reports that
the Provider/LLM test passes the basic completion checks but flow
creation fails with:
failed to create flow worker: failed to get flow provider: failed
to determine tool call ID template: failed to collect tool call ID
samples: all sample collection attempts failed: failed to call LLM:
400 Bad Request: registry.ollama.ai/library/gemma3:27b-it-q4_K_M
does not support tools
The underlying signal "does not support tools" is emitted by the
Ollama API itself; PentAGI buries it five wraps deep, so the user is
left guessing whether the issue is configuration, networking, or
something internal to PentAGI.
## Solution
Add an unexported helper `wrapToolCallIDTemplateError` in
`pkg/providers/helpers.go` that detects the upstream "does not support
tools" substring and prepends a one-sentence guidance message naming
the constraint and a few known-good Ollama tags
(`llama3.1`, `qwen2.5`, `mistral-nemo`). All other errors keep the
existing wording. The original error is preserved via `%w` so log
spans, langfuse traces, and any future `errors.Is/As` callers continue
to work.
Both call sites in `pkg/providers/providers.go`
(`NewFlowProvider`, `NewAssistantProvider`) now route through the
helper.
No schema, API, or DB changes. No new lifecycle state. No background
work. The behavior change is the wording of one error string in one
upstream-defined failure case.
## User Impact
- Users who pick an Ollama model without tool support get a clear,
actionable error during flow / assistant creation instead of a deep
stack trace.
- Users on tool-capable models (Anthropic, OpenAI, Bedrock, and
tool-capable Ollama tags) see no behavioral or message change.
- No restart, migration, or configuration step required.
## Test Plan
- New table-driven test `TestWrapToolCallIDTemplateError` covers:
- nil error -> nil
- upstream "does not support tools" chain (matching the real five-
wrap shape) -> actionable message + `errors.Is` round-trips to the
original error
- generic non-tools error -> existing wrap text preserved, no
new guidance appended
- `go test ./pkg/providers/...` passes locally.
- `go build ./...` and `go vet ./pkg/providers/...` clean.
Closes#280
Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
Custom prompts saved via the Settings -> Prompts UI are persisted to the
database, but every assistant and flow session creation path was using
templates.NewDefaultPrompter() with a leftover TODO, so user overrides
never reached the agents and Langfuse traces always showed the defaults.
Add a controller-side helper that loads the user's saved prompts and
overlays them on the compiled defaults. Prompt types the user has not
customized continue to use the defaults; an empty body row is treated
as no override (the UI uses delete to reset). A database error fails
session creation explicitly instead of silently falling back.
Wire the helper into the four affected call sites in NewAssistantWorker,
LoadAssistantWorker, NewFlowWorker, and LoadFlowWorker.
Closes#300
Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
mason5052