28 Commits
Author SHA1 Message Date
Dmitry Ng db68cde21c feat(flows): implement provider renaming and reset functionality
- Added `RenameFlowsProvider` and `ResetFlowsProviderToDefault` methods to the `FlowController` to handle renaming of user-defined providers and resetting to built-in defaults.
- Introduced SQL queries for bulk updating flow and assistant provider names based on user actions, ensuring idempotency and safe retries.
- Enhanced error handling and logging for provider updates, ensuring that flows and assistants remain valid after provider changes.
- Added unit tests to verify the correct behavior of provider renaming and resetting functionalities.
2026-08-04 01:06:58 +03:00
Dmitry Ng ca482686d7 feat(database): enhance PostgreSQL schema handling for multi-tenant support
- Added new environment variables `DATABASE_EXTENSIONS_SCHEMA` and `DATABASE_SEARCH_PATH_VIA_OPTIONS` to `.env.example` and `docker-compose.yml` for better management of PostgreSQL schemas in multi-tenant deployments.
- Updated the README and configuration documentation to explain the purpose and usage of the new variables, particularly for setups using Supabase.
- Implemented schema verification and initialization logic in the backend to ensure proper handling of tenant-specific schemas during database connections.
- Enhanced the installer and server settings forms to include the new configuration options, improving user experience and clarity.
2026-07-30 00:14:30 +03:00
Dmitry NgandSergey Kozyrenko 45c84c500f test(frontend): make the e2e suite falsifiable, and fix what it found (#380)
The e2e suite and CI gates could report green without checking anything: the
lint/codegen gate never ran on fork PRs, the codegen freshness check diffed the
wrong range, the sticky comment claimed passes it never read, and several
visual/palette/contrast/a11y gates were written so they could not fail. Closing
those gates then exposed real defects.

**E2E and CI.** New coverage for the assistant lifecycle, flow interrupt, live
panels, tab deep links, the report route and its exports, session expiry, and
uploads/downloads (59 → 125 spec cases). Gates now fail on real regressions:
absolute pixel budget, per-tab panel sweeps, scoped a11y waivers, page-error
assertions. CI runs on `pull_request`, ranges from the PR base, routes comments
by PR identity, and redacts stand secrets from the public artifact.

**Markdown editor.** ~20 round-trip and toolbar fixes: table pipe escaping
(nested lists, Go template pipelines, headerless tables), fence validity,
setext and line-leading escapes, inline code containing a backtick, block
toggles under select-all, controls disabled where the command is unavailable or
a table cell cannot hold the block, theme-aware code surface. Two perf fixes cut
1 MB document load from 69s to 4s.

**Accessibility and UI.** Focus returns to whatever opened a dialog, sheet or
menu; icon-only buttons named; `aria-sort` on table columns; syntax tokens
retuned to clear AA in both themes; a dedicated link token. Detail and settings
pages now distinguish authz denials and partial errors from missing records,
survive background refetches without blanking or discarding unsaved edits, and
stop naming exported PDFs `*.pdf.pdf`.

**Backend.** Passwords capped at the 72 bytes bcrypt can hash (over-long ones
previously 500'd after passing validation); four agent options the GraphQL round
trip silently dropped are carried through; a flow whose worker fails to start is
no longer left in the listing; the Graphiti health check retries before
permanently disabling the client.

---------

Co-authored-by: Sergey Kozyrenko <sirozha_blr@tut.by>
2026-07-29 16:34:29 +04:00
Dmitry Ng 5b48fd9f8e Merge branch 'feature/frontend' into feature/next-release 2026-07-22 01:38:05 +03:00
Dmitry Ng 74c2663114 Merge branch 'integrate/open-prs-v2' into feature/next-release 2026-07-22 01:25:25 +03:00
Sergey KozyrenkoandClaude Opus 4.8 c635a9fe34 test(converter): cover per-agent extra_body round-trip
Guards the two-way mapping added with the extra_body field: a JSON object
survives GraphQL→pconfig→GraphQL intact, and an absent one stays nil rather
than becoming an empty map.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 04:03:42 +07:00
Sergey KozyrenkoandClaude Opus 4.8 cae0295e8b feat(providers): expose per-agent extra_body in the provider config
The backend already threaded a per-agent `extra_body` map into the LLM request
(pconfig.AgentConfig, openai.WithExtraBody), but it was reachable only from YAML
provisioning — the GraphQL type, the converter, and the settings form all
dropped it. Editing a YAML-provisioned provider through the UI therefore round-
tripped its config and silently stripped extra_body.

Add `extraBody: Map` to AgentConfig / AgentConfigInput, carry it through the
converter in both directions, and give each agent an "Extra Body" JSON field in
the provider form (validated as an object, serialized on load, parsed on save).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 03:57:23 +07:00
rramprakash 3f68960d98 feat(tools): add Firecrawl web search provider
Add Firecrawl as a network search provider alongside Tavily, Perplexity,
Traversaal, SearXNG and the others. Firecrawl's /v2/search endpoint combines
web search with page scraping, returning full-page markdown content for each
result, which the searcher agent summarizes like the other content-rich
engines.

- New firecrawl tool mirrors the Tavily provider layer-for-layer: registry
  definition, tool-type mapping, factory wiring in both tool builders, and
  ftester executor/args/mocks support.
- Configurable endpoint: FIRECRAWL_API_KEY plus an optional FIRECRAWL_API_URL
  (default https://api.firecrawl.dev) so self-hosted Firecrawl instances work.
- Adds 'firecrawl' to the searchengine_type enum (goose migration + models)
  and the search-log server model so searches are recorded per engine.
- Installer wizard, .env.example, docker-compose, prompts and docs updated.

The tool is only registered when FIRECRAWL_API_KEY is set, so existing
deployments are unaffected.
2026-07-16 12:53:11 +05:30
Sergey KozyrenkoandClaude Opus 4.8 2853c97549 feat(reasoning): add three-state On/Off/Default reasoning control
Add an explicit reasoning Off to the per-agent config, backed by the langchaingo
tri-state API. Off emits llms.WithReasoningDisabled() (the provider disable wire)
via BuildOptions, and UsesAdaptiveThinking is guarded so Off overrides the
adaptive-only auto-adaptive.

Per-model capability is derived at runtime from llms.ReasoningSupportFor and
surfaced through GraphQL ModelReasoningInfo, so the UI only offers Off where it
actually disables: cannotDisable reports when Off would be rejected (always-on
models) OR a silent no-op (an unclassified default-on model whose disable wire is
omitted), and capability is surfaced for any thinking-capable model (e.g. Gemini,
which declares thinking without a reasoning block). A new
ProviderType.ReasoningProvider() supplies the provider to the resolver.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 19:50:43 +07:00
Sergey KozyrenkoandClaude Opus 4.8 86b1329c43 fix(providers): surface bedrock models error, make provider type list exhaustive, trim comments (refs #288, #328)
Review nits:
- schema.resolvers.go: log (not swallow) bedrock.DefaultModels errors so an
  invalid BEDROCK_MODELS_PATH no longer yields a silently-empty model list.
- settings-providers.tsx: derive the create-provider type list from an
  exhaustive Record<ProviderType,string> so a future ProviderType is a compile
  error here instead of silently missing from the menu.
- trim two doc comments (convertModelReasoningMode, ReasoningFields) to lead
  with the load-bearing contract instead of restating the name.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 18:05:26 +07:00
Sergey KozyrenkoandClaude Opus 4.8 55335feb17 feat(minimax): add MiniMax provider with full app/UI/installer parity (refs #328)
Reworks external PR #328 (octo-patch). The PR added only the provider core;
this brings MiniMax to full parity with the other providers (qwen) so it is
selectable and configurable in the UI and installer.

Applied from the PR (verified against MiniMax's official API docs — M3/M2.7/
M2.7-highspeed are real current models; corrected the M3 description from the
PR's "512K" to the documented ~1M context):
- minimax provider package (OpenAI-compatible https://api.minimax.io/v1),
  config.yml, models.yml, tests; MINIMAX_API_KEY/SERVER_URL/PROVIDER env vars;
  ProviderMiniMax type + DefaultProviderNameMiniMax; providers.go wiring;
  Valid() whitelist.

Added for completeness:
- goose migration adding 'minimax' to the PROVIDER_TYPE enum + database
  ProviderTypeMinimax const.
- GraphQL: minimax in ProviderType enum, ProvidersModelsList,
  ProvidersReadinessStatus, DefaultProvidersConfig; resolvers wire default
  config/models + enabled status; gqlgen regenerated.
- Frontend: MiniMax icon (lobehub), provider-icon + settings-providers
  registration + provider type list; regenerated GraphQL types.
- Installer wizard: provider form, screen, list, registry, env-var mappings,
  locale strings + help text.
- ctester/ftester: -type/-provider minimax support.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 15:24:04 +07:00
Sergey KozyrenkoandClaude Opus 4.8 001566c11a test(bedrock,pconfig,converter): cover adaptive-thinking backstop + reasoning (refs #288)
Closes audit-found coverage gaps:
- bedrock: TestBackstopForcesAdaptiveForAdaptiveOnlyModel (adaptive-only model
  forces adaptive with no agent reasoning block) + a negative case for a
  non-reasoning model. The Anthropic backstop already had an httptest test;
  Bedrock had none.
- pconfig: ReasoningConfig.EffectiveMode (incl. max_tokens->budget inference)
  and IsZero.
- converter: ConvertModels reasoning mapping (pconfig adaptive-only ->
  GraphQL adaptive_only, efforts passthrough, nil for no descriptor).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 14:35:34 +07:00
Sergey KozyrenkoandClaude Opus 4.8 31780e5262 feat(graph): expose model reasoning capability via GraphQL (refs #288)
Adds ModelReasoningInfo (mode + allowed efforts) to the GraphQL ModelConfig so
the frontend can gate the reasoning mode/effort UI by each model's declared
capability instead of a model-name regex. New ModelReasoningMode enum
(budget | adaptive | adaptive_only) mirrors the models.yml descriptor; the
converter maps pconfig adaptive-only -> GraphQL adaptive_only.

Backend groundwork for the Layer 2 frontend adaptive-thinking UI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 10:26:16 +07:00
Sergey KozyrenkoandClaude Opus 4.8 ca9283b93e feat(bedrock): integrate adaptive thinking mechanism backend (refs #288)
Integrates the backend half of PR #288 onto our branch (the frontend is
reworked separately in our codegen style), 3-way merged so the #233
config/models-path changes are preserved:
- adaptive_thinking.go: smithy Build middleware that rewrites the langchaingo
  Converse body from thinking{type:enabled,budget_tokens} to
  thinking{type:adaptive} + output_config.effort, wired via WithAPIOptions and
  prepareCallOptions in Call/CallEx/CallWithTools.
- pconfig: ReasoningConfig.Mode (adaptive|budget) + EffectiveMode/IsZero.
- GraphQL: ReasoningMode enum, reasoning.mode field, xhigh/max effort levels.

Drops the dead llms.WithMetadata adaptive branch from AgentConfig.BuildOptions
(nothing reads opts.Metadata on the Bedrock path; adaptive is applied per-call
by the provider) and updates the unit test accordingly.

Hardening still pending (next commit): strip temperature/top_p/top_k and set
display:summarized for adaptive requests, add Opus 4.7/4.8 catalog entries with
a model reasoning-capability descriptor + force-adaptive backstop, and a live
Bedrock repro. Until then Opus 4.7 should not be selected for an agent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 09:59:48 +07:00
Sergey KozyrenkoandClaude Fable 5 942fb45c16 feat(knowledge): dedicated renameKnowledgeDocument mutation, lighter list
Rename a knowledge document via a dedicated mutation that rewrites only the
question in cmetadata — no re-embedding, no embedder required — mirroring the
flows renameFlow pattern instead of round-tripping the full document through
updateKnowledgeDocument.

Backend:
- renameKnowledgeDocument(id, question) mutation + resolver (admin/user split;
  ownership enforced at GetUserDocument, like the update pair)
- metadata-only query UpdateKnowledgeDocumentMetadata (no migration)
- unit + edge tests: metadata-only, missing-doc error, non-owner rejection

Frontend:
- renameKnowledge provider method; wire list and detail inline-rename to it
- drop the content "Preview" column and request the list with withContent:false
  so it no longer pulls full document bodies

Verified end to end against a local Docker backend (rename works; content and
embedding preserved) and against the remote backend (graceful failure where the
mutation is not yet deployed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 13:15:57 +07:00
Dmitry Ng 8feb7bf311 feat(tests): enhance knowledge and flowfile tests with containment barriers
Added new test cases to validate containment barriers in the `ResolvePulledStagedTarget` and `ZipRelativePaths` functions, ensuring that paths escaping the designated directories are properly rejected. Updated the `knowledge_test.go` to include scenarios for handling nil embedder and error propagation during document creation, improving overall test coverage and robustness.
2026-05-29 15:42:11 +03:00
Dmitry Ng 39f122467d feat(config): add new embedding and rename database connection pool settings
- Introduced `EMBEDDING_MAX_TEXT_BYTES` to limit the maximum byte size of text sent to the embedding model.
- Renamed database connection pool settings: `DATABASE_MAX_OPEN_CONNS`, `DATABASE_MAX_IDLE_CONNS`, and `DATABASE_VECTOR_MAX_CONNS` for improved PostgreSQL connection management.
- Updated relevant documentation to reflect these new configuration options and their usage.
- Adjusted various components to utilize the new settings for enhanced performance and resource management.
2026-05-18 18:26:52 +03:00
Dmitry Ng 2ce863ec1a feat(toolcall): implement ToolCall logging functionality
- Added ToolCallLogProvider interface with methods for logging tool calls, updating success and failure statuses.
- Introduced proxyToolCallLogProvider to handle ToolCall logging operations.
- Updated flow execution components to integrate ToolCall logging, including flow workers and controllers.
- Enhanced GraphQL schema to support ToolCall logs, including queries and subscriptions for real-time updates.
- Updated documentation to reflect the new ToolCall logging features and their usage.
2026-05-18 11:21:56 +03:00
Dmitry Ng 077ddce476 feat(database): enhance PostgreSQL connection pooling and configuration
- Introduced shared connection pooling for PostgreSQL using `*sql.DB` for sqlc and GORM, optimizing resource usage.
- Added new environment variables: `DB_MAX_OPEN_CONNS`, `DB_MAX_IDLE_CONNS`, and `DB_VECTOR_MAX_CONNS` for configurable connection limits.
- Updated documentation to reflect new connection pooling strategy and provide operational commands for monitoring.
- Implemented shared `pgxpool` for pgvector stores to reduce connection overhead and improve performance.
- Adjusted various components to utilize the new connection pooling setup, ensuring efficient database interactions.
2026-05-18 11:13:43 +03:00
Dmitry Ng 0f0a7bd2d0 fix(database/knowledge): replace SimilaritySearch with direct SQLC vector queries
- Fix empty ID bug: langchaingo SimilaritySearch discarded document UUIDs; new SearchKnowledgeDocuments/SearchUserKnowledgeDocuments return them directly
- Remove unsafe fmt.Sprintf SQL filter interpolation, use parameterised queries
- Exclude memory documents from search results at SQL level
- Add FlowID support to passesSearchFilter
- Convert all $N positional params to sqlc.arg(name) across knowledge queries
- Update tests: replace TestBuildSearchFilters with TestPassesSearchFilter, add TestSearchDocuments and TestSearchUserDocuments
2026-05-12 20:47:46 +03:00
Dmitry Ng ca1dfa138c feat(knowledge): enhance document update functionality with docType handling
- Introduced a new `docType` field in the `UpdateKnowledgeDocumentInput` to manage document type changes.
- Updated the `doUpdate` method to clear subtype fields (GuideType, AnswerType, CodeLang) when the document type changes.
- Added comprehensive tests for various document type transitions to ensure correct behavior and state preservation.
- Updated GraphQL schema and generated models to accommodate the new `docType` field.
2026-05-11 10:29:13 +03:00
Dmitry Ng 3a52079278 feat(knowledge): add pgvector knowledge base management
- GraphQL/REST CRUD + semantic search for knowledge documents
- KnowledgeStore with admin/user-scoped filtering, re-embedding on update
- Real-time subscriptions (created/updated/deleted) per user and admin
- user_id tracking in all agent-stored documents (guide/answer/code/memory)
- sqlc queries, goose migrations, privilege grants, user_id backfill
- Memory cleanup on flow deletion; stale orphan purge via migration
- Unit tests for all KnowledgeStore operations including security cases
- Frontend GraphQL schema and TypeScript types regenerated
2026-05-05 01:09:20 +03:00
Dmitry Ng 629c018a68 fix: bug with retrieving resources recursive via graphql 2026-05-01 16:09:06 +03:00
Dmitry Ng aa4b70eaba feat: add user resources system and flow integration
- UserResource model with MD5-deduplicated blob storage and virtual path filesystem
- REST API for resource CRUD (upload, mkdir, move, copy, delete, download)
- GraphQL query/mutations with resourceIds support on createFlow, putUserInput, createAssistant, callAssistant
- Resource → flow copy with hierarchy restore; incremental container sync (find missing, copy once)
- FlowWorker.PutResources delegates copy, docker push and flowFileAdded events
- Agent prompts updated with {{.UserFiles}} XML listing of /work/uploads and /work/resources
- Resource subscriptions: resourceAdded/Updated/Deleted
2026-04-28 17:00:17 +03:00
Dmitry Ng 239dd31c91 feat: add GetUsageStatsByModelAgentsForFlow query and related GraphQL types
- Implemented a new SQL query to retrieve usage statistics by model and agents for a specific flow.
- Added corresponding Go types and methods to handle the new query in the database layer.
- Updated GraphQL schema to include the new ModelAgentsUsageStats type and the usageStatsByModelAgentsForFlow query.
- Enhanced frontend components to display the new usage statistics in the flow dashboard.
2026-04-22 19:51:20 +03:00
Dmitry Ng 7c25f356ed feat: enable runtime provider switching
Adds conditional chain normalization in processChain to preserve reasoning cache when provider unchanged while fixing incomplete tool_calls and converting IDs when switching providers. Extends GraphQL API with modelProvider parameter for seamless provider changes without restart.
2026-04-08 23:58:56 +03:00
Dmitry Ng 089dcb36b4 feat: implement flow templates management 2026-04-08 03:34:32 +03:00
Dmitry Ng b90ea4711e repo final state 2026-03-26 06:16:07 +03:00