- Added configuration options for an optional internal analytics engine in `.env.example`, `docker-compose.yml`, and related files.
- Updated documentation to include details about the new internal analytics engine, which scrapes and summarizes pages as a fallback for the web_search tool.
- Enhanced the `ftester` to support testing the new internal engine alongside existing search engines.
- Adjusted various components to integrate the new engine, including configuration forms and validation logic.
This feature allows users to enable a browser-based fallback for analytic queries, enhancing the flexibility of the web search capabilities.
- Added `edit_file` to the file tool; diffs are parsed and applied in-memory via `github.com/sergi/go-diff`, exported as `tools.ApplyUnifiedDiff`.
- Made the hunk header lenient (position numbers optional) for models that omit them, with fuzzy content-based matching as fallback.
- Tuned `FileAction` field descriptions (layered, minimal-token) to steer models toward including context lines, based on live model testing via `ctester`.
- Added a `file_edit` multi-turn read_file → edit_file test case in `pkg/providers/tester`, using PentAGI's real tool schema, wired into the `advanced` test group.
- Fixed a bug where a single stateful test-case instance was shared across agent types, leaking conversation history and failures between independent runs.
- Extended the mock provider with sequential responses for multi-turn scenarios; added ctester integration tests.
Graphiti transport errors, an empty file path, and an omitted 'action' or
double-encoded 'questions' arg from the LLM all hard-failed the tool chain
and burned retries instead of degrading gracefully. Also: stopTaskTimeout
5s->60s (flow kept running after a false 500), and the routine "cookie
claim invalid" case now logs at Warn instead of Error.
Add Firecrawl as a network search provider alongside Tavily, Perplexity,
Traversaal, SearXNG and the others. Firecrawl's /v2/search endpoint combines
web search with page scraping, returning full-page markdown content for each
result, which the searcher agent summarizes like the other content-rich
engines.
- New firecrawl tool mirrors the Tavily provider layer-for-layer: registry
definition, tool-type mapping, factory wiring in both tool builders, and
ftester executor/args/mocks support.
- Configurable endpoint: FIRECRAWL_API_KEY plus an optional FIRECRAWL_API_URL
(default https://api.firecrawl.dev) so self-hosted Firecrawl instances work.
- Adds 'firecrawl' to the searchengine_type enum (goose migration + models)
and the search-log server model so searches are recorded per engine.
- Installer wizard, .env.example, docker-compose, prompts and docs updated.
The tool is only registered when FIRECRAWL_API_KEY is set, so existing
deployments are unaffected.
- demuxExecStdout errors on a header cut short mid-frame (was a silent EOF that
dropped the tail) and surfaces a docker systemerr frame instead of discarding it.
- ftester drains telemetry on exit through the bounded observer.Drain instead of
two unbounded ForceFlush calls, so an unreachable collector can't hang it at exit.
- container-listing failures are sorted for a deterministic skipped-entries preview,
logged per-entry at Debug (the detail is already in the response and the endpoint
is hit on every navigation) with names quoted so control bytes in a hostile
filename can't inject into a log line; corrected the stat-failure comment
(a dangling symlink lstats fine, it doesn't fail).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three defects on the shutdown and container-listing paths:
- Telemetry drain ran the langfuse and otel sinks serially under one deadline.
langfuse's ForceFlush/Shutdown ignore the caller ctx and block on their own
timeout, so a langfuse outage burned the whole budget and the healthy otel
sink's final batch was never flushed. Add observer.Drain, which flushes then
shuts down each sink in its own goroutine bounded by the deadline; main uses
it, retiring the inline flush-then-shutdown dance.
- The partial-success listing put the raw stat/list error into the 200 body's
Failures[].Message, exposing docker-layer detail (container ids, the daemon
address) that response.Error only reveals under develop mode. Send a generic
message to the client and keep the raw text dev-gated, logged server-side.
- A path read successfully by one query could also appear in Failures when
another query failed on it, order-dependent. Reconcile at assembly: drop any
failure whose path was read successfully, so no path is in both arrays.
Also bound the exec stdout buffered before parsing so a compromised sandbox
can't stream unbounded output into memory.
Tests: Drain delivers a healthy otel sink while a langfuse sink blocks and
returns within its deadline; the listing failure message is dev-gated; a path
never lands in both Files and Failures. Live: unreachable collector -> shutdown
stays bounded (~5s) and exits cleanly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The graceful-shutdown branch only logged "cleaning up resources" and exited
without flushing telemetry, so the last buffered batch was dropped on every
SIGTERM. Drain it, with three corrections:
- main: flush then shut down the observer on shutdown, bounded by a 5s
deadline in a goroutine so an unreachable collector can't stall exit.
Flush must precede Shutdown because the langfuse observer's Shutdown
cancels its context without flushing; its final batch drains only via
ForceFlush while the context is still live.
- observer Flush/Shutdown: drain both the langfuse and otel sinks and join
their errors instead of returning after the first, which skipped the
second sink entirely when both were configured.
- telemetryClient Flush/Shutdown: attempt all three providers and always
close the grpc connection, joining errors, so a stalled provider flush
no longer skips the rest or leaks the connection.
Tests: the langfuse observer test proves Shutdown drops the buffered batch
while ForceFlush drains it to the sink; observer-level tests prove both
sinks drain and errors aggregate. Live: with an unreachable collector,
shutdown stays bounded (~5s) and exits cleanly instead of stalling ~10s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A set-but-unreachable OTEL collector hung startup: NewTelemetryClient dialed
with grpc.WithBlock() and no timeout on the deadline-free signal context, so
the process blocked forever before it ever served. And a non-ErrNotConfigured
init error from either observability client was log.Fatalf, killing the
process. Both let an OPTIONAL integration take the whole app down.
Bound the dial with a 10s timeout, and on init failure degrade to a no-op
observer with a logged warning instead of exiting. Verified live: a bad
OTEL_HOST that hung startup indefinitely now boots in ~14s with a warning and
runs a full flow; a reachable collector still boots in ~4s with no warning.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reworks external PR #328 (octo-patch). The PR added only the provider core;
this brings MiniMax to full parity with the other providers (qwen) so it is
selectable and configurable in the UI and installer.
Applied from the PR (verified against MiniMax's official API docs — M3/M2.7/
M2.7-highspeed are real current models; corrected the M3 description from the
PR's "512K" to the documented ~1M context):
- minimax provider package (OpenAI-compatible https://api.minimax.io/v1),
config.yml, models.yml, tests; MINIMAX_API_KEY/SERVER_URL/PROVIDER env vars;
ProviderMiniMax type + DefaultProviderNameMiniMax; providers.go wiring;
Valid() whitelist.
Added for completeness:
- goose migration adding 'minimax' to the PROVIDER_TYPE enum + database
ProviderTypeMinimax const.
- GraphQL: minimax in ProviderType enum, ProvidersModelsList,
ProvidersReadinessStatus, DefaultProvidersConfig; resolvers wire default
config/models + enabled status; gqlgen regenerated.
- Frontend: MiniMax icon (lobehub), provider-icon + settings-providers
registration + provider type list; regenerated GraphQL types.
- Installer wizard: provider form, screen, list, registry, env-var mappings,
locale strings + help text.
- ctester/ftester: -type/-provider minimax support.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reworks PR #233 to follow the existing config-path pattern and make an
externally-added Bedrock model fully usable from the web UI.
- BEDROCK_CONFIG_PATH: replace the embedded per-agent config (model
assignments + prices), mirroring OLLAMA_SERVER_CONFIG_PATH /
LLM_SERVER_CONFIG_PATH. DefaultProviderConfig now takes *config.Config
and reads the external file when set, else the embedded config.yml.
- BEDROCK_MODELS_PATH: merge an external model catalog onto the embedded
models.yml so new ids appear and are selectable under Settings ->
Providers. DefaultModels now takes *config.Config and the resolver +
provider construction pass it through; new ids are added, a matching
name overrides the embedded entry.
Wires both through docker-compose, .env.example, config.md and the README,
and ships examples/configs/bedrock-glm-flash.{provider,models}.yml using the
real AWS Bedrock id zai.glm-4.7-flash (In-Region, no inference-profile
prefix, 4K max output).
Dropped from #233: the ProviderConfig.Name field and the SeedDefaultProviders
mechanism (a per-user DB write fired on startup and inside the read-only
SettingsProviders resolver) -- the config already reaches the UI through
DefaultProvidersConfig without any DB rows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove/Purge/FactoryReset failed with "file does not exist" for stacks
that were never extracted (no docker-compose-*.yml). Now such stacks are
skipped instead of aborting the whole operation.
- Add composeFileExists check before destructive compose commands
- Add isDestructiveComposeOperation helper
Problem:
In the installer processor, post-operation state refresh failures from
checker.GatherUpdatesInfo were silently discarded, so apply/install/
update/remove/purge could report success even when the refresh failed.
Root cause:
Two distinct shadowing/fall-through patterns wrapped the error into a
variable that was never returned:
- In the deferred closures of applyChanges/install/update/purge, the
inner `if err := GatherUpdatesInfo(ctx); err != nil` shadowed the
named return `err`, so `err = fmt.Errorf(...)` assigned the shadow and
the wrap never reached the `sendCompletion(stack, err)` defer.
- In the ProductStackWorker cases of remove/purge, the block-local err
was wrapped but the case fell through without returning, while every
sibling check in the same switch returns on error.
Solution:
- Deferred closures: wrap into a `gatherErr` local and assign the named
return `err`, so the failure propagates to sendCompletion.
- Switch cases: `return` the wrapped error, matching the sibling checks.
Testing:
go build ./cmd/installer/... -> OK
go vet ./cmd/installer/processor/... -> OK
go test ./cmd/installer/processor/... -> ok
golangci-lint (ineffassign) on logic.go -> clean
- README: align supported-models intro with the local convention used
by every other provider section ("Models marked with `*` are used
in default configuration"), so the asterisk on each model ID has a
near-by explanation.
- Installer help (`LLMFormDeepSeekHelp`): swap the legacy
"DeepSeek-Chat" / "DeepSeek-Reasoner" bullets in "Default PentAGI
Models" for the current `deepseek-v4-flash` / `deepseek-v4-pro`
defaults so the wizard guidance matches the bundled config.
No code, schema, or LiteLLM prefix behavior changes.
The DeepSeek provider config still defaulted to the legacy
`deepseek-chat` and `deepseek-reasoner` model names, which the
upstream DeepSeek API has announced for deprecation on 2026-07-24.
A first-run install therefore breaks once the legacy names are
removed.
Swap the defaults to the current DeepSeek V4 family:
- non-thinking roles use `deepseek-v4-flash`
- reasoning-heavy roles use `deepseek-v4-pro`
The change is limited to the embedded `config.yml` / `models.yml`
inside `backend/pkg/providers/deepseek`, the matching example at
`examples/configs/deepseek.provider.yml`, the `DeepSeekAgentModel`
fallback constant in `deepseek.go`, and three doc references
(README.md, backend/docs/config.md, backend/docs/llms_how_to.md)
plus one installer help string in
`backend/cmd/installer/wizard/locale/locale.go`. LiteLLM prefix
behavior is untouched.
- Introduced `EMBEDDING_MAX_TEXT_BYTES` to limit the maximum byte size of text sent to the embedding model.
- Renamed database connection pool settings: `DATABASE_MAX_OPEN_CONNS`, `DATABASE_MAX_IDLE_CONNS`, and `DATABASE_VECTOR_MAX_CONNS` for improved PostgreSQL connection management.
- Updated relevant documentation to reflect these new configuration options and their usage.
- Adjusted various components to utilize the new settings for enhanced performance and resource management.
- Added ToolCallLogProvider interface with methods for logging tool calls, updating success and failure statuses.
- Introduced proxyToolCallLogProvider to handle ToolCall logging operations.
- Updated flow execution components to integrate ToolCall logging, including flow workers and controllers.
- Enhanced GraphQL schema to support ToolCall logs, including queries and subscriptions for real-time updates.
- Updated documentation to reflect the new ToolCall logging features and their usage.
- Introduced shared connection pooling for PostgreSQL using `*sql.DB` for sqlc and GORM, optimizing resource usage.
- Added new environment variables: `DB_MAX_OPEN_CONNS`, `DB_MAX_IDLE_CONNS`, and `DB_VECTOR_MAX_CONNS` for configurable connection limits.
- Updated documentation to reflect new connection pooling strategy and provide operational commands for monitoring.
- Implemented shared `pgxpool` for pgvector stores to reduce connection overhead and improve performance.
- Adjusted various components to utilize the new connection pooling setup, ensuring efficient database interactions.
- GraphQL/REST CRUD + semantic search for knowledge documents
- KnowledgeStore with admin/user-scoped filtering, re-embedding on update
- Real-time subscriptions (created/updated/deleted) per user and admin
- user_id tracking in all agent-stored documents (guide/answer/code/memory)
- sqlc queries, goose migrations, privilege grants, user_id backfill
- Memory cleanup on flow deletion; stale orphan purge via migration
- Unit tests for all KnowledgeStore operations including security cases
- Frontend GraphQL schema and TypeScript types regenerated
- Added new endpoints for managing flow files, including listing, uploading, and deleting files within flow workspaces.
- Introduced FlowFile model to represent file metadata.
- Enhanced GraphQL schema to support flow file operations and subscriptions for real-time updates.
- Updated API documentation to reflect new flow file functionalities.
- Changed default terminal tool timeout from 600 to 1200 seconds.
- Updated related documentation across .env.example, README.md, and config files to reflect the new timeout settings and their constraints.
- Enhanced descriptions in code comments and documentation to clarify timeout behavior, including clamping rules for values outside the accepted range.
Adds conditional chain normalization in processChain to preserve reasoning cache when provider unchanged while fixing incomplete tool_calls and converting IDs when switching providers. Extends GraphQL API with modelProvider parameter for seamless provider changes without restart.
Mirrors the existing LLM_SERVER_CONFIG_PATH / OLLAMA_SERVER_CONFIG_PATH
pattern so operators can supply a YAML file that overrides the built-in
embedded bedrock/config.yml without modifying source code.
Changes:
- config.go: add BedrockConfig field mapped to BEDROCK_CONFIG_PATH env var
- bedrock/bedrock.go: DefaultProviderConfig now accepts *config.Config;
reads from BedrockConfig path when set, falls back to embedded config.yml
- providers.go, cmd/ctester/main.go: pass cfg to updated DefaultProviderConfig
- docker-compose.yml: wire BEDROCK_CONFIG_PATH env var and mount
${PENTAGI_BEDROCK_CONFIG_PATH:-./glm_flash_bedrock.yml} into the container
- .env.example: document the new BEDROCK_CONFIG_PATH variable
- glm_flash_bedrock.yml: example config exporting the glm_flash DB provider
(zai.glm-4.7-flash via Bedrock) as a portable YAML file
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Dmitry Ng