Commit Graph
65 Commits
Author SHA1 Message Date
Dmitry Ng c257dabc57 feat(config): update hcnsec provider configuration and test report
- Replaced all instances of `Qwen3-Coder-Next-FP8` with `MiniMax-M2.7` in the hcnsec provider configuration, adjusting model settings and pricing accordingly.
- Updated the test report to reflect the new model configurations, including success rates and average latencies for various agents using `MiniMax-M2.7`.
- Enhanced the overall structure of the configuration file for clarity and consistency.
2026-07-25 15:02:49 +03:00
Dmitry Ng d9b28ce8bf chore(dependencies): update langchaingo version and adjust model configurations
- Updated the `langchaingo` dependency in `go.mod` from version `v0.1.15-0.20260723091023-35da5c0f0620` to `v0.1.14-update.6`.
- Modified the model configurations in `config.yml` and `models.yml` to replace `claude-opus-4-8` with `claude-opus-5` for both the generator and refiner sections.
- Updated the model descriptions and pricing in `models.yml` to reflect the new `claude-opus-5` model and added new models `claude-sonnet-4-6` and `claude-haiku-4-5`.
- Adjusted the test report in `anthropic-report.md` to reflect the updated model names and improved average latencies across various tests.
2026-07-25 02:52:04 +03:00
Dmitry Ng deaff9eafe chore(config): update launch.json and add new provider configurations
- Added new configuration options in `.vscode/launch.json` for vllm and hcnsec providers.
- Updated `deepinfra.provider.yml` to reflect changes in model configurations and pricing.
- Introduced `hcnsec.provider.yml` with detailed model settings and pricing for HCNSec.
- Adjusted various test reports to reflect updated success rates and latencies for different models.
2026-07-23 17:37:44 +03:00
Dmitry Ng 74c2663114 Merge branch 'integrate/open-prs-v2' into feature/next-release 2026-07-22 01:25:25 +03:00
Dmitry Ng c95cc778c0 Merge pull request #362 from jinhaosong-source/feat/orcarouter-provider-config
Add OrcaRouter pre-configured provider
2026-07-18 18:41:38 +04:00
Dmitry Ng d93d45bb4c Merge pull request #350 from mason5052/codex/issue-336-headless-crawler-rfc
docs: add headless crawler integration RFC
2026-07-18 10:14:57 +04:00
Dmitry Ng 6ca92ea40c Merge pull request #347 from mason5052/codex/issue-344-qwen-long-flow-troubleshooting
docs: add Qwen long-flow troubleshooting
2026-07-18 10:11:50 +04:00
Dmitry Ng f8b05690b1 Merge pull request #346 from mason5052/codex/issue-341-fallback-rfc
docs: add tool and model fallback RFC
2026-07-18 09:46:07 +04:00
Dmitry Ng 5e8b9c25e4 Merge pull request #345 from mason5052/codex/issue-342-browseros-mcp-rfc
docs: add BrowserOS MCP browser backend RFC
2026-07-17 17:33:19 +04:00
Dmitry Ng 2ef7076bb4 Merge pull request #331 from mason5052/codex/issue-321-vertex-provider-rfc
docs: add native Google Vertex AI provider RFC
2026-07-17 16:05:57 +04:00
Dmitry Ng 9e0c93d0e9 Merge pull request #329 from mason5052/codex/issue-61-install-config-guide
docs: add installing and configuring guide for first deployment
2026-07-17 15:41:09 +04:00
jinhaosong-source 2abeecbc63 Add OrcaRouter pre-configured provider config 2026-07-01 19:29:48 +08:00
Sergey KozyrenkoandClaude Opus 4.8 06e37d8026 feat(bedrock): support external config and model catalog paths (refs #233)
Reworks PR #233 to follow the existing config-path pattern and make an
externally-added Bedrock model fully usable from the web UI.

- BEDROCK_CONFIG_PATH: replace the embedded per-agent config (model
  assignments + prices), mirroring OLLAMA_SERVER_CONFIG_PATH /
  LLM_SERVER_CONFIG_PATH. DefaultProviderConfig now takes *config.Config
  and reads the external file when set, else the embedded config.yml.
- BEDROCK_MODELS_PATH: merge an external model catalog onto the embedded
  models.yml so new ids appear and are selectable under Settings ->
  Providers. DefaultModels now takes *config.Config and the resolver +
  provider construction pass it through; new ids are added, a matching
  name overrides the embedded entry.

Wires both through docker-compose, .env.example, config.md and the README,
and ships examples/configs/bedrock-glm-flash.{provider,models}.yml using the
real AWS Bedrock id zai.glm-4.7-flash (In-Region, no inference-profile
prefix, 4K max output).

Dropped from #233: the ProviderConfig.Name field and the SeedDefaultProviders
mechanism (a per-user DB write fired on startup and inside the read-only
SettingsProviders resolver) -- the config already reaches the UI through
DefaultProvidersConfig without any DB rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 08:30:10 +07:00
Sergey KozyrenkoandClaude Opus 4.8 ebe0618dee docs: add Atlas Cloud as a custom OpenAI-compatible provider (refs #332)
Reworks PR #332 to follow the repo's provider-doc convention instead of
its README-marketing form:
- add examples/configs/atlas.provider.yml, modeled on the existing
  aggregator example configs (deepinfra/openrouter/novita)
- list Atlas Cloud in the README aggregators line
- LLM_SERVER_PROVIDER left EMPTY for direct access — it is a LiteLLM
  model-name prefix, and #332 incorrectly recommended `openai`, which
  would break /models discovery for Atlas's vendor-prefixed model ids

Dropped from #332 (not matching any existing aggregator's docs): vendor
logo/banner, UTM-tracked links, and the static 59-model table.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 07:43:20 +07:00
mason5052 3ef8b51f36 docs: address review feedback on install/config guide
- Use the package-path form for the tester utilities
  (go run ./cmd/ctester, go run ./cmd/etester test) instead of shell
  glob / single-file invocations, which are more portable and avoid the
  multi-file compile pitfall.
- Add -verbose to the container etester example so it matches the local
  example.
- Wrap the new README Quick Start pointer across multiple lines; rendered
  Markdown is unchanged.
2026-06-19 04:45:20 +07:00
mason5052 ebfd2bd9e9 docs: add installing and configuring guide for first deployment
Issue #61 asked for a proper guide on how to start using PentAGI. A
first-use guide (How to Use PentAGI After Login) already covers what to
do after login, but the maintainer kept the issue open until an article
about installing and configuring the product is published.

Add examples/guides/installation_configuration.md, a concise ordered
walkthrough that connects the existing detailed README sections into a
single path: choose installer vs manual Docker Compose, set core server
variables (PUBLIC_URL, CORS_ORIGINS, security vars), configure and test
an LLM provider with ctester, configure and test the embedding provider
with etester, optionally add search providers and the Graphiti/Langfuse/
observability stacks, then start and verify the stack and change the
default admin password. It ends with a quick checklist and links forward
to the existing post-login usage guide.

The guide links to the relevant README reference sections rather than
duplicating them. A single pointer link is added at the top of the
README Quick Start section for discoverability.

Docs only. No installer behavior, runtime code, env vars, or schema
changes. Every environment variable named in the guide already exists in
.env.example and backend/pkg/config/config.go.
2026-06-19 04:45:20 +07:00
mason5052 6f93856716 docs: add qwen long-flow troubleshooting 2026-06-19 04:45:05 +07:00
mason5052 e8a26c6d2c docs: clarify crawler RFC config sketch and scope_decision
Address Copilot review on PR #350: note that default_backend selects a crawl backend and passive extractors such as jsfinder are not selectable there; model scope_decision as a decision plus reason in both the field list and the JSON example; add an illustrative scope_entries example and clarify when allowed_hosts applies.

Refs #336
2026-06-19 04:12:40 +07:00
mason5052 7af46322ca docs: add headless crawler integration RFC
Add examples/proposals/headless_crawler_integration.md, a docs-only RFC proposing an optional, tool-agnostic crawler / URL discovery capability for PentAGI agents. Candidate backends are katana, crawlergo, rad, and jsfinder, framed as candidates with no mandatory default. The RFC keeps dictionary fuzzing (ffuf/dirsearch) unchanged, defines structured discovery artifacts (URLs, forms, parameters, JavaScript endpoints, status codes, source page, depth, scope decision), and keeps crawler URL discovery separate from the BrowserOS MCP interactive browser backend.

Refs #336
2026-06-19 04:12:39 +07:00
mason5052 5980c02854 docs: add recommendation and requested decision to Kubernetes RFC 2026-06-19 04:12:39 +07:00
mason5052 5bf41e0ea1 docs: fix grammar in Kubernetes RFC open question (#324)
Address review feedback on PR #326: rephrase the executor-model open
question so the final clause reads clearly.

Refs #324
2026-06-19 04:12:39 +07:00
mason5052 8dbf8889e0 docs: add Kubernetes deployment compatibility RFC (#324)
Add examples/proposals/kubernetes_deployment.md, an RFC-style design
surface responding to the Kubernetes deployment request in #324. The
document is docs-only: no Helm charts, manifests, operator, compose,
installer, or environment-variable changes, and it does not claim
Kubernetes is supported today.

It records the current Compose/installer deployment assumptions, maps
each one (secrets/config, volumes, service discovery, ingress/TLS,
health checks, network policies, the Docker-socket flow executor,
observability, image overrides, migrations) to candidate Kubernetes
equivalents, and proposes an incremental, docs-first path with open
questions, security/operational considerations, and a test/validation
strategy. The hardest item -- the Docker-socket worker executor -- is
laid out as candidate options without choosing one, and keeps flow
lifecycle explicit and inspectable per the #268 review lesson.

Refs #324
2026-06-19 04:12:39 +07:00
mason5052 760574677b docs: address review feedback on Vertex AI RFC
- Drop the hard-coded provider count to avoid doc drift as providers are
  added; list the current provider types instead.
- Use the ADC-specific command (gcloud auth application-default login)
  rather than the ambiguous 'gcloud login'.
- Reference the enum-swap migration pattern generically under
  backend/migrations/sql/ instead of a single timestamped filename that
  may be renamed or squashed.
2026-06-19 04:12:39 +07:00
mason5052 16669696b6 docs: add native Google Vertex AI provider RFC
Issue #321 requests a native Vertex AI provider with service-account /
ADC auth, opened after #310 clarified Vertex is not supported today. The
issue carries a full implementation outline and four open questions, and
the author asks for direction on adapter strategy and Gemini-vs-Claude
scope before implementing.

Add examples/proposals/vertex_ai_provider.md, a planning RFC that frames
the work before any code is written:

- Distinguishes the current options (AI Studio gemini, direct Anthropic,
  AWS Bedrock, and the custom OpenAI-compatible LiteLLM proxy workaround)
  from the proposed native vertex provider.
- Recommends a staged approach: v1 Gemini-on-Vertex with ADC /
  service-account auth; Claude-on-Vertex deferred to a maintainer
  decision, noting it likely belongs on the Anthropic adapter (Vertex
  auth/endpoint mode) rather than the Gemini-shaped path because the
  message schema differs.
- Models auth on the existing Bedrock multi-auth precedent (ADC default
  chain plus service-account file), and flags that service-account JSON
  is sensitive and must be file-mounted/secret-managed, never pasted into
  UI or logs.
- Captures config/migration touch points (provider checklist, REST
  Valid() whitelist, PROVIDER_TYPE enum-swap migration) so the eventual
  implementation size is clear, and restates the issue's open questions.

Docs/RFC only. No code, schema, migration, generated, frontend, or
provider runtime files are touched, and no new env vars or types are
added; every VERTEX_* key and type named is labeled as a candidate. No
overlap with the provider files in open PR #328.
2026-06-19 04:12:38 +07:00
mason5052 040c69ff28 docs: add recommendation and requested decision to BrowserOS MCP RFC 2026-06-19 04:12:38 +07:00
mason5052 b6f1e61dae docs: add BrowserOS MCP browser backend RFC (#342) 2026-06-19 04:12:38 +07:00
mason5052 92426b9b98 docs: add v1 recommendation and requested decision to fallback RFC 2026-06-19 04:12:38 +07:00
mason5052 44393b0f06 docs: add tool and model fallback RFC 2026-06-19 04:12:37 +07:00
mason5052 5dad31d90e docs: add recommendation and requested decision to Kubernetes RFC 2026-06-13 23:07:13 -04:00
mason5052 a03c29589d docs: add recommendation and requested decision to BrowserOS MCP RFC 2026-06-13 23:04:56 -04:00
mason5052 673671cd0c docs: add v1 recommendation and requested decision to fallback RFC 2026-06-13 23:03:21 -04:00
mason5052 1043084050 docs: clarify crawler RFC config sketch and scope_decision
Address Copilot review on PR #350: note that default_backend selects a crawl backend and passive extractors such as jsfinder are not selectable there; model scope_decision as a decision plus reason in both the field list and the JSON example; add an illustrative scope_entries example and clarify when allowed_hosts applies.

Refs #336
2026-06-13 22:37:48 -04:00
mason5052 364fbd4010 docs: add headless crawler integration RFC
Add examples/proposals/headless_crawler_integration.md, a docs-only RFC proposing an optional, tool-agnostic crawler / URL discovery capability for PentAGI agents. Candidate backends are katana, crawlergo, rad, and jsfinder, framed as candidates with no mandatory default. The RFC keeps dictionary fuzzing (ffuf/dirsearch) unchanged, defines structured discovery artifacts (URLs, forms, parameters, JavaScript endpoints, status codes, source page, depth, scope decision), and keeps crawler URL discovery separate from the BrowserOS MCP interactive browser backend.

Refs #336
2026-06-13 22:33:11 -04:00
mason5052 1e19b70988 docs: add qwen long-flow troubleshooting 2026-06-09 20:09:55 -04:00
mason5052 1a7012c1f5 docs: add tool and model fallback RFC 2026-06-09 20:08:46 -04:00
mason5052 6146bc6b44 docs: add BrowserOS MCP browser backend RFC (#342) 2026-06-09 20:05:37 -04:00
mason5052 620e699e71 docs: address review feedback on install/config guide
- Use the package-path form for the tester utilities
  (go run ./cmd/ctester, go run ./cmd/etester test) instead of shell
  glob / single-file invocations, which are more portable and avoid the
  multi-file compile pitfall.
- Add -verbose to the container etester example so it matches the local
  example.
- Wrap the new README Quick Start pointer across multiple lines; rendered
  Markdown is unchanged.
2026-06-03 23:06:11 -04:00
mason5052 5c968919ec docs: address review feedback on Vertex AI RFC
- Drop the hard-coded provider count to avoid doc drift as providers are
  added; list the current provider types instead.
- Use the ADC-specific command (gcloud auth application-default login)
  rather than the ambiguous 'gcloud login'.
- Reference the enum-swap migration pattern generically under
  backend/migrations/sql/ instead of a single timestamped filename that
  may be renamed or squashed.
2026-06-03 23:04:50 -04:00
mason5052 bd7c14f854 docs: add native Google Vertex AI provider RFC
Issue #321 requests a native Vertex AI provider with service-account /
ADC auth, opened after #310 clarified Vertex is not supported today. The
issue carries a full implementation outline and four open questions, and
the author asks for direction on adapter strategy and Gemini-vs-Claude
scope before implementing.

Add examples/proposals/vertex_ai_provider.md, a planning RFC that frames
the work before any code is written:

- Distinguishes the current options (AI Studio gemini, direct Anthropic,
  AWS Bedrock, and the custom OpenAI-compatible LiteLLM proxy workaround)
  from the proposed native vertex provider.
- Recommends a staged approach: v1 Gemini-on-Vertex with ADC /
  service-account auth; Claude-on-Vertex deferred to a maintainer
  decision, noting it likely belongs on the Anthropic adapter (Vertex
  auth/endpoint mode) rather than the Gemini-shaped path because the
  message schema differs.
- Models auth on the existing Bedrock multi-auth precedent (ADC default
  chain plus service-account file), and flags that service-account JSON
  is sensitive and must be file-mounted/secret-managed, never pasted into
  UI or logs.
- Captures config/migration touch points (provider checklist, REST
  Valid() whitelist, PROVIDER_TYPE enum-swap migration) so the eventual
  implementation size is clear, and restates the issue's open questions.

Docs/RFC only. No code, schema, migration, generated, frontend, or
provider runtime files are touched, and no new env vars or types are
added; every VERTEX_* key and type named is labeled as a candidate. No
overlap with the provider files in open PR #328.
2026-06-03 23:01:05 -04:00
mason5052 5b957bde25 docs: add installing and configuring guide for first deployment
Issue #61 asked for a proper guide on how to start using PentAGI. A
first-use guide (How to Use PentAGI After Login) already covers what to
do after login, but the maintainer kept the issue open until an article
about installing and configuring the product is published.

Add examples/guides/installation_configuration.md, a concise ordered
walkthrough that connects the existing detailed README sections into a
single path: choose installer vs manual Docker Compose, set core server
variables (PUBLIC_URL, CORS_ORIGINS, security vars), configure and test
an LLM provider with ctester, configure and test the embedding provider
with etester, optionally add search providers and the Graphiti/Langfuse/
observability stacks, then start and verify the stack and change the
default admin password. It ends with a quick checklist and links forward
to the existing post-login usage guide.

The guide links to the relevant README reference sections rather than
duplicating them. A single pointer link is added at the top of the
README Quick Start section for discoverability.

Docs only. No installer behavior, runtime code, env vars, or schema
changes. Every environment variable named in the guide already exists in
.env.example and backend/pkg/config/config.go.
2026-06-03 22:49:52 -04:00
mason5052 250824ae20 docs: fix grammar in Kubernetes RFC open question (#324)
Address review feedback on PR #326: rephrase the executor-model open
question so the final clause reads clearly.

Refs #324
2026-06-01 17:26:06 -04:00
mason5052 6141975338 docs: add Kubernetes deployment compatibility RFC (#324)
Add examples/proposals/kubernetes_deployment.md, an RFC-style design
surface responding to the Kubernetes deployment request in #324. The
document is docs-only: no Helm charts, manifests, operator, compose,
installer, or environment-variable changes, and it does not claim
Kubernetes is supported today.

It records the current Compose/installer deployment assumptions, maps
each one (secrets/config, volumes, service discovery, ingress/TLS,
health checks, network policies, the Docker-socket flow executor,
observability, image overrides, migrations) to candidate Kubernetes
equivalents, and proposes an incremental, docs-first path with open
questions, security/operational considerations, and a test/validation
strategy. The hardest item -- the Docker-socket worker executor -- is
laid out as candidate options without choosing one, and keeps flow
lifecycle explicit and inspectable per the #268 review lesson.

Refs #324
2026-06-01 17:16:17 -04:00
Dmitry Ng 83c263e98e feat(qwen): enhance agent configurations with thinking control parameters
Updated the Qwen agent configuration to include `extra_body` parameters for thinking control across various models. Added `enable_thinking` and `preserve_thinking` options for reasoning agents, while utility agents have `enable_thinking` set to false. Adjusted the Qwen client initialization to support these new configurations. Updated test report to reflect changes in success rates and latencies.
2026-05-29 00:06:47 +03:00
Dmitry Ng 643ff7d218 feat(providers): Update model configurations for various LLM providers: qwen, kimi, glm, deepseek, gemini 2026-05-28 15:41:17 +03:00
Dmitry Ng 2f827a54ef Merge pull request #307 from mason5052/codex/issue-296-mcp-client-rfc
docs(rfc): propose MCP client integration design
2026-05-28 00:11:02 +04:00
Dmitry Ng e674113745 Merge pull request #317 from mason5052/codex/issue-314-deepseek-v4-models
fix(deepseek): update default model names to DeepSeek V4
2026-05-28 00:09:25 +04:00
Dmitry Ng 084670dd97 Merge pull request #306 from mason5052/codex/issue-298-flow-concurrency-rfc
docs(rfc): propose persistent flow queue and completion webhooks
2026-05-23 16:06:23 +04:00
mason5052 24176c2805 fix(deepseek): explicitly disable thinking mode for non-thinking Flash roles
DeepSeek V4 thinking mode defaults to enabled on both deepseek-v4-flash
and deepseek-v4-pro; non-thinking behavior requires an explicit toggle.
Per official docs, in thinking mode temperature/top_p/presence_penalty/
frequency_penalty are ignored, so the existing Flash role sampling knobs
would have been silently no-ops without the toggle.

Add extra_body.thinking.type=disabled to the five non-thinking Flash
roles so deepseek-v4-flash actually runs in non-thinking mode and
honors the role's temperature/top_p settings:

- simple, simple_json, adviser, searcher, enricher

Pro roles (primary_agent, assistant, generator, refiner, reflector,
coder, installer, pentester) intentionally keep thinking enabled (the
V4 default) for reasoning, tool-use, and security analysis.

PentAGI provider config already supports extra_body as a first-class
yaml field on AgentConfig and forwards it through openai.WithExtraBody,
which the vxcontrol langchaingo fork serializes at the top level of the
Chat Completions request - the same pattern Kimi uses for tool_choice.
No code, schema, or LiteLLM prefix changes required.

Touches:
- backend/pkg/providers/deepseek/config.yml (embedded production config)
- examples/configs/deepseek.provider.yml (user-facing example)

No change to role-to-model mapping, model metadata, pricing, README
wording, LiteLLM prefix, unrelated providers, lifecycle, queues, or
installer flow.
2026-05-22 12:26:28 -04:00
mason5052 67bf76514b docs(deepseek): align V4 model metadata with official pricing and context
- Update model descriptions to reflect V4 1M context window (up to 384K output)
  instead of legacy 128K wording in models.yml and README.
- Split Flash and Pro pricing per official DeepSeek API docs:
  - deepseek-v4-flash: input 0.14 / output 0.28 / cache_hit 0.0028 per 1M tokens
  - deepseek-v4-pro:   input 0.435 / output 0.87 / cache_hit 0.003625 per 1M tokens
- Apply per-role price split across all 13 role configs in both the embedded
  config.yml and the user-facing examples/configs/deepseek.provider.yml.
- Replace stale "cache pricing is 10% of input cost" claim in the README,
  which no longer holds for either V4 model.
- No change to LiteLLM prefix behavior, role-to-model mapping, lifecycle,
  queues, GraphQL schema, migrations, frontend, or installer flow.
2026-05-22 12:04:14 -04:00
mason5052 3113ff3aa3 fix(deepseek): update default model names to DeepSeek V4
The DeepSeek provider config still defaulted to the legacy
`deepseek-chat` and `deepseek-reasoner` model names, which the
upstream DeepSeek API has announced for deprecation on 2026-07-24.
A first-run install therefore breaks once the legacy names are
removed.

Swap the defaults to the current DeepSeek V4 family:

- non-thinking roles use `deepseek-v4-flash`
- reasoning-heavy roles use `deepseek-v4-pro`

The change is limited to the embedded `config.yml` / `models.yml`
inside `backend/pkg/providers/deepseek`, the matching example at
`examples/configs/deepseek.provider.yml`, the `DeepSeekAgentModel`
fallback constant in `deepseek.go`, and three doc references
(README.md, backend/docs/config.md, backend/docs/llms_how_to.md)
plus one installer help string in
`backend/cmd/installer/wizard/locale/locale.go`. LiteLLM prefix
behavior is untouched.
2026-05-21 15:44:30 -04:00