78 Commits
Author SHA1 Message Date
Dmitry Ng a4dcfa8178 feat(provider): update OpenCode provider configuration and testing report
- Enhanced OpenCode provider configuration with detailed pricing for various models.
- Updated agent settings to reflect new model assignments and pricing structures.
- Revised testing report to include updated performance metrics and average latencies for agents.
- Adjusted overall success rates and latencies in the report for improved accuracy.
2026-08-04 23:57:58 +03:00
Dmitry Ng 454569a8eb feat(provider): add OpenCode provider configuration and testing report
- Added OpenCode provider configuration file with model settings and parameters.
- Updated README to include OpenCode as a supported provider and modified relevant sections for clarity.
- Created a testing report for OpenCode, summarizing agent performance and success rates.
- Enhanced VSCode launch configuration to include OpenCode provider options for testing.
2026-08-04 20:58:08 +03:00
Dmitry Ng 2da929d38d feat(providers): add xAI provider support and configuration details
- Updated the README to include xAI as a supported provider, along with pre-configured provider files for testing.
- Added a new configuration file for xAI with detailed model settings and pricing.
- Created a testing report for xAI, summarizing the performance and success rates of various agents.
- Enhanced the VSCode launch configuration to include xAI provider options for testing.
2026-08-04 00:42:25 +03:00
Dmitry Ng c19665d822 fix(docs): improve clarity and formatting in installation guides
- Consolidated sentences in the README and installation configuration guide for better readability.
- Removed unnecessary line breaks and ensured consistent formatting for installation instructions and requirements.
- Enhanced links to related documentation for a smoother user experience.
2026-08-03 17:04:12 +03:00
Dmitry Ng f6fe867a99 feat(worker_node): enhance monitoring setup and documentation for worker node
- Updated `METRICS_IP` documentation to reflect new metrics ports (8080, 9100) for cAdvisor and node-exporter.
- Added detailed instructions for deploying cAdvisor and node-exporter on the worker node's host Docker, including scripts for setup.
- Enhanced the security and firewall configuration section to include new metrics endpoints.
- Improved overall documentation clarity for metrics integration with PentAGI's observability stack.
2026-08-01 20:16:15 +03:00
Dmitry Ng 339aa958f0 feat(neo4j): enhance configuration and directory structure for Neo4j integration
- Added new environment variables to `.env.example` for Neo4j memory and transaction settings, including `NEO4J_HEAP_INITIAL_SIZE`, `NEO4J_HEAP_MAX_SIZE`, `NEO4J_PAGECACHE_SIZE`, and `NEO4J_TRANSACTION_MAX`.
- Updated `docker-compose-graphiti.yml` to mount Neo4j configuration and plugin directories, ensuring proper integration with the Graphiti stack.
- Introduced static configuration files for Neo4j and APOC in the `examples/neo4j/conf` directory, providing a clear structure for user-editable settings.
- Enhanced the README to document the new configuration options and directory structure for Neo4j, improving user guidance for setup and customization.
2026-08-01 20:03:02 +03:00
Dmitry Ng 1778290d15 feat(graphiti): update model versions in configuration files
- Updated the model references in `gemini.yaml` from `gemini/gemini-2.5-flash-lite` to `gemini-3.5-flash-lite` and `gemini/gemini-2.5-flash-lite` to `gemini-3.1-flash-lite` for improved performance.
- Changed the model references in `openai.yaml` from `openai/gpt-5-mini` to `gpt-5.6-luna` and `openai/gpt-5.4-nano` to `gpt-5-nano`, reflecting the latest versions for better capabilities.
2026-08-01 19:54:27 +03:00
Dmitry Ng cfa325228e feat(gemini): update model configurations and test report
- Increased temperature settings for `simple` and `simple_json` models from 0.7 to 1.0 for enhanced variability.
- Changed model references from `gemini-3.5-flash` to `gemini-3.5-flash-lite` for `reflector`, `searcher`, and `enricher`, optimizing for cost and performance.
- Updated `coder`, `installer`, and `pentester` models to `gemini-3.6-flash`, reflecting the latest version with improved pricing and capabilities.
- Revised test report to reflect new model configurations, including updated success rates and average latencies, ensuring accurate performance metrics.
- Enhanced documentation to clarify model descriptions and pricing adjustments for better user understanding.
2026-08-01 19:53:43 +03:00
Dmitry Ng 341a9def05 feat(worker_node): add browser scraper deployment instructions for worker node
- Introduced a new section in the worker node guide detailing the deployment of a browser scraper on the worker node's host Docker, including setup instructions and security considerations.
- Added a script for launching the scraper with appropriate configurations, including environment variables for credentials and resource limits.
- Updated firewall requirements to include access for the scraper service, ensuring clarity on network configurations.
- Enhanced documentation to guide users on verifying the scraper's readiness and integrating it with the main PentAGI node.
2026-08-01 19:44:55 +03:00
Dmitry Ng c383086afe refactor(worker_node): improve worker node guide and policy test script
- Revised the worker node guide to clarify the use of TCP over TLS for worker connections, addressing potential failure modes associated with socket mounts.
- Updated environment variable configurations to include `METRICS_IP`, ensuring proper binding for Docker metrics.
- Enhanced the `policy-tests.sh` script to improve transport resolution for the DinD API, allowing for better handling of both TCP and Unix socket connections.
- Improved documentation for the metrics endpoint and security model, ensuring clearer instructions for users.
2026-07-30 20:46:01 +03:00
Dmitry Ng ebb228b19a feat(graphiti): enhance configuration and integration settings
- Added new environment variables to `.env.example` and `docker-compose-graphiti.yml` for Graphiti, including `GRAPHITI_CPUS`, `GRAPHITI_MEMORY`, and various ingestion and logging settings.
- Updated the README to clarify the optional nature of the Graphiti integration and its configuration requirements.
- Enhanced the backend installer to support new Graphiti configuration options, ensuring proper handling of the Graphiti stack.
- Improved the user interface for Graphiti settings in the installer wizard, allowing for better configuration management.
- Revised documentation to reflect changes in Graphiti's deployment modes and integration capabilities.
2026-07-30 16:26:16 +03:00
Dmitry Ng dfb762f420 fix(config): update model versions in Anthropic configuration and test report
- Changed the model version from `claude-opus-5` to `claude-opus-4-8` in the configuration file for both generator and refiner.
- Updated the test report to reflect the new model version, including adjustments to success rates and average latencies for the generator and refiner agents.
- Revised overall average latency in the test report to account for the updated model performance.
2026-07-29 22:05:27 +03:00
Dmitry Ng 464dca8903 feat(worker_node): enhance Docker-in-Docker setup with security and cleanup features
- Updated the worker node guide to include a hardened Docker-in-Docker (dind) configuration, emphasizing security measures such as OPA authorization and seccomp profiles.
- Introduced new scripts for managing dind containers, including `dind-cleanup.sh` for removing stale containers and `run-dind.sh` for starting the dind container with authorization.
- Added configuration files for the dind setup, including `authz.rego`, `seccomp.json`, and `daemon.json`, to enforce strict security policies.
- Implemented a systemd timer and service for regular cleanup of nested containers, ensuring efficient resource management.
- Expanded documentation to clarify the purpose and usage of new scripts and configuration files, enhancing user understanding of the setup process.
2026-07-29 16:01:42 +03:00
Dmitry Ng c257dabc57 feat(config): update hcnsec provider configuration and test report
- Replaced all instances of `Qwen3-Coder-Next-FP8` with `MiniMax-M2.7` in the hcnsec provider configuration, adjusting model settings and pricing accordingly.
- Updated the test report to reflect the new model configurations, including success rates and average latencies for various agents using `MiniMax-M2.7`.
- Enhanced the overall structure of the configuration file for clarity and consistency.
2026-07-25 15:02:49 +03:00
Dmitry Ng d9b28ce8bf chore(dependencies): update langchaingo version and adjust model configurations
- Updated the `langchaingo` dependency in `go.mod` from version `v0.1.15-0.20260723091023-35da5c0f0620` to `v0.1.14-update.6`.
- Modified the model configurations in `config.yml` and `models.yml` to replace `claude-opus-4-8` with `claude-opus-5` for both the generator and refiner sections.
- Updated the model descriptions and pricing in `models.yml` to reflect the new `claude-opus-5` model and added new models `claude-sonnet-4-6` and `claude-haiku-4-5`.
- Adjusted the test report in `anthropic-report.md` to reflect the updated model names and improved average latencies across various tests.
2026-07-25 02:52:04 +03:00
Dmitry Ng deaff9eafe chore(config): update launch.json and add new provider configurations
- Added new configuration options in `.vscode/launch.json` for vllm and hcnsec providers.
- Updated `deepinfra.provider.yml` to reflect changes in model configurations and pricing.
- Introduced `hcnsec.provider.yml` with detailed model settings and pricing for HCNSec.
- Adjusted various test reports to reflect updated success rates and latencies for different models.
2026-07-23 17:37:44 +03:00
Dmitry Ng 74c2663114 Merge branch 'integrate/open-prs-v2' into feature/next-release 2026-07-22 01:25:25 +03:00
Dmitry Ng c95cc778c0 Merge pull request #362 from jinhaosong-source/feat/orcarouter-provider-config
Add OrcaRouter pre-configured provider
2026-07-18 18:41:38 +04:00
Dmitry Ng d93d45bb4c Merge pull request #350 from mason5052/codex/issue-336-headless-crawler-rfc
docs: add headless crawler integration RFC
2026-07-18 10:14:57 +04:00
Dmitry Ng 6ca92ea40c Merge pull request #347 from mason5052/codex/issue-344-qwen-long-flow-troubleshooting
docs: add Qwen long-flow troubleshooting
2026-07-18 10:11:50 +04:00
Dmitry Ng f8b05690b1 Merge pull request #346 from mason5052/codex/issue-341-fallback-rfc
docs: add tool and model fallback RFC
2026-07-18 09:46:07 +04:00
Dmitry Ng 5e8b9c25e4 Merge pull request #345 from mason5052/codex/issue-342-browseros-mcp-rfc
docs: add BrowserOS MCP browser backend RFC
2026-07-17 17:33:19 +04:00
Dmitry Ng 2ef7076bb4 Merge pull request #331 from mason5052/codex/issue-321-vertex-provider-rfc
docs: add native Google Vertex AI provider RFC
2026-07-17 16:05:57 +04:00
Dmitry Ng 9e0c93d0e9 Merge pull request #329 from mason5052/codex/issue-61-install-config-guide
docs: add installing and configuring guide for first deployment
2026-07-17 15:41:09 +04:00
jinhaosong-source 2abeecbc63 Add OrcaRouter pre-configured provider config 2026-07-01 19:29:48 +08:00
Sergey KozyrenkoandClaude Opus 4.8 06e37d8026 feat(bedrock): support external config and model catalog paths (refs #233)
Reworks PR #233 to follow the existing config-path pattern and make an
externally-added Bedrock model fully usable from the web UI.

- BEDROCK_CONFIG_PATH: replace the embedded per-agent config (model
  assignments + prices), mirroring OLLAMA_SERVER_CONFIG_PATH /
  LLM_SERVER_CONFIG_PATH. DefaultProviderConfig now takes *config.Config
  and reads the external file when set, else the embedded config.yml.
- BEDROCK_MODELS_PATH: merge an external model catalog onto the embedded
  models.yml so new ids appear and are selectable under Settings ->
  Providers. DefaultModels now takes *config.Config and the resolver +
  provider construction pass it through; new ids are added, a matching
  name overrides the embedded entry.

Wires both through docker-compose, .env.example, config.md and the README,
and ships examples/configs/bedrock-glm-flash.{provider,models}.yml using the
real AWS Bedrock id zai.glm-4.7-flash (In-Region, no inference-profile
prefix, 4K max output).

Dropped from #233: the ProviderConfig.Name field and the SeedDefaultProviders
mechanism (a per-user DB write fired on startup and inside the read-only
SettingsProviders resolver) -- the config already reaches the UI through
DefaultProvidersConfig without any DB rows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 08:30:10 +07:00
Sergey KozyrenkoandClaude Opus 4.8 ebe0618dee docs: add Atlas Cloud as a custom OpenAI-compatible provider (refs #332)
Reworks PR #332 to follow the repo's provider-doc convention instead of
its README-marketing form:
- add examples/configs/atlas.provider.yml, modeled on the existing
  aggregator example configs (deepinfra/openrouter/novita)
- list Atlas Cloud in the README aggregators line
- LLM_SERVER_PROVIDER left EMPTY for direct access — it is a LiteLLM
  model-name prefix, and #332 incorrectly recommended `openai`, which
  would break /models discovery for Atlas's vendor-prefixed model ids

Dropped from #332 (not matching any existing aggregator's docs): vendor
logo/banner, UTM-tracked links, and the static 59-model table.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 07:43:20 +07:00
mason5052 3ef8b51f36 docs: address review feedback on install/config guide
- Use the package-path form for the tester utilities
  (go run ./cmd/ctester, go run ./cmd/etester test) instead of shell
  glob / single-file invocations, which are more portable and avoid the
  multi-file compile pitfall.
- Add -verbose to the container etester example so it matches the local
  example.
- Wrap the new README Quick Start pointer across multiple lines; rendered
  Markdown is unchanged.
2026-06-19 04:45:20 +07:00
mason5052 ebfd2bd9e9 docs: add installing and configuring guide for first deployment
Issue #61 asked for a proper guide on how to start using PentAGI. A
first-use guide (How to Use PentAGI After Login) already covers what to
do after login, but the maintainer kept the issue open until an article
about installing and configuring the product is published.

Add examples/guides/installation_configuration.md, a concise ordered
walkthrough that connects the existing detailed README sections into a
single path: choose installer vs manual Docker Compose, set core server
variables (PUBLIC_URL, CORS_ORIGINS, security vars), configure and test
an LLM provider with ctester, configure and test the embedding provider
with etester, optionally add search providers and the Graphiti/Langfuse/
observability stacks, then start and verify the stack and change the
default admin password. It ends with a quick checklist and links forward
to the existing post-login usage guide.

The guide links to the relevant README reference sections rather than
duplicating them. A single pointer link is added at the top of the
README Quick Start section for discoverability.

Docs only. No installer behavior, runtime code, env vars, or schema
changes. Every environment variable named in the guide already exists in
.env.example and backend/pkg/config/config.go.
2026-06-19 04:45:20 +07:00
mason5052 6f93856716 docs: add qwen long-flow troubleshooting 2026-06-19 04:45:05 +07:00
mason5052 e8a26c6d2c docs: clarify crawler RFC config sketch and scope_decision
Address Copilot review on PR #350: note that default_backend selects a crawl backend and passive extractors such as jsfinder are not selectable there; model scope_decision as a decision plus reason in both the field list and the JSON example; add an illustrative scope_entries example and clarify when allowed_hosts applies.

Refs #336
2026-06-19 04:12:40 +07:00
mason5052 7af46322ca docs: add headless crawler integration RFC
Add examples/proposals/headless_crawler_integration.md, a docs-only RFC proposing an optional, tool-agnostic crawler / URL discovery capability for PentAGI agents. Candidate backends are katana, crawlergo, rad, and jsfinder, framed as candidates with no mandatory default. The RFC keeps dictionary fuzzing (ffuf/dirsearch) unchanged, defines structured discovery artifacts (URLs, forms, parameters, JavaScript endpoints, status codes, source page, depth, scope decision), and keeps crawler URL discovery separate from the BrowserOS MCP interactive browser backend.

Refs #336
2026-06-19 04:12:39 +07:00
mason5052 5980c02854 docs: add recommendation and requested decision to Kubernetes RFC 2026-06-19 04:12:39 +07:00
mason5052 5bf41e0ea1 docs: fix grammar in Kubernetes RFC open question (#324)
Address review feedback on PR #326: rephrase the executor-model open
question so the final clause reads clearly.

Refs #324
2026-06-19 04:12:39 +07:00
mason5052 8dbf8889e0 docs: add Kubernetes deployment compatibility RFC (#324)
Add examples/proposals/kubernetes_deployment.md, an RFC-style design
surface responding to the Kubernetes deployment request in #324. The
document is docs-only: no Helm charts, manifests, operator, compose,
installer, or environment-variable changes, and it does not claim
Kubernetes is supported today.

It records the current Compose/installer deployment assumptions, maps
each one (secrets/config, volumes, service discovery, ingress/TLS,
health checks, network policies, the Docker-socket flow executor,
observability, image overrides, migrations) to candidate Kubernetes
equivalents, and proposes an incremental, docs-first path with open
questions, security/operational considerations, and a test/validation
strategy. The hardest item -- the Docker-socket worker executor -- is
laid out as candidate options without choosing one, and keeps flow
lifecycle explicit and inspectable per the #268 review lesson.

Refs #324
2026-06-19 04:12:39 +07:00
mason5052 760574677b docs: address review feedback on Vertex AI RFC
- Drop the hard-coded provider count to avoid doc drift as providers are
  added; list the current provider types instead.
- Use the ADC-specific command (gcloud auth application-default login)
  rather than the ambiguous 'gcloud login'.
- Reference the enum-swap migration pattern generically under
  backend/migrations/sql/ instead of a single timestamped filename that
  may be renamed or squashed.
2026-06-19 04:12:39 +07:00
mason5052 16669696b6 docs: add native Google Vertex AI provider RFC
Issue #321 requests a native Vertex AI provider with service-account /
ADC auth, opened after #310 clarified Vertex is not supported today. The
issue carries a full implementation outline and four open questions, and
the author asks for direction on adapter strategy and Gemini-vs-Claude
scope before implementing.

Add examples/proposals/vertex_ai_provider.md, a planning RFC that frames
the work before any code is written:

- Distinguishes the current options (AI Studio gemini, direct Anthropic,
  AWS Bedrock, and the custom OpenAI-compatible LiteLLM proxy workaround)
  from the proposed native vertex provider.
- Recommends a staged approach: v1 Gemini-on-Vertex with ADC /
  service-account auth; Claude-on-Vertex deferred to a maintainer
  decision, noting it likely belongs on the Anthropic adapter (Vertex
  auth/endpoint mode) rather than the Gemini-shaped path because the
  message schema differs.
- Models auth on the existing Bedrock multi-auth precedent (ADC default
  chain plus service-account file), and flags that service-account JSON
  is sensitive and must be file-mounted/secret-managed, never pasted into
  UI or logs.
- Captures config/migration touch points (provider checklist, REST
  Valid() whitelist, PROVIDER_TYPE enum-swap migration) so the eventual
  implementation size is clear, and restates the issue's open questions.

Docs/RFC only. No code, schema, migration, generated, frontend, or
provider runtime files are touched, and no new env vars or types are
added; every VERTEX_* key and type named is labeled as a candidate. No
overlap with the provider files in open PR #328.
2026-06-19 04:12:38 +07:00
mason5052 040c69ff28 docs: add recommendation and requested decision to BrowserOS MCP RFC 2026-06-19 04:12:38 +07:00
mason5052 b6f1e61dae docs: add BrowserOS MCP browser backend RFC (#342) 2026-06-19 04:12:38 +07:00
mason5052 92426b9b98 docs: add v1 recommendation and requested decision to fallback RFC 2026-06-19 04:12:38 +07:00
mason5052 44393b0f06 docs: add tool and model fallback RFC 2026-06-19 04:12:37 +07:00
mason5052 5dad31d90e docs: add recommendation and requested decision to Kubernetes RFC 2026-06-13 23:07:13 -04:00
mason5052 a03c29589d docs: add recommendation and requested decision to BrowserOS MCP RFC 2026-06-13 23:04:56 -04:00
mason5052 673671cd0c docs: add v1 recommendation and requested decision to fallback RFC 2026-06-13 23:03:21 -04:00
mason5052 1043084050 docs: clarify crawler RFC config sketch and scope_decision
Address Copilot review on PR #350: note that default_backend selects a crawl backend and passive extractors such as jsfinder are not selectable there; model scope_decision as a decision plus reason in both the field list and the JSON example; add an illustrative scope_entries example and clarify when allowed_hosts applies.

Refs #336
2026-06-13 22:37:48 -04:00
mason5052 364fbd4010 docs: add headless crawler integration RFC
Add examples/proposals/headless_crawler_integration.md, a docs-only RFC proposing an optional, tool-agnostic crawler / URL discovery capability for PentAGI agents. Candidate backends are katana, crawlergo, rad, and jsfinder, framed as candidates with no mandatory default. The RFC keeps dictionary fuzzing (ffuf/dirsearch) unchanged, defines structured discovery artifacts (URLs, forms, parameters, JavaScript endpoints, status codes, source page, depth, scope decision), and keeps crawler URL discovery separate from the BrowserOS MCP interactive browser backend.

Refs #336
2026-06-13 22:33:11 -04:00
mason5052 1e19b70988 docs: add qwen long-flow troubleshooting 2026-06-09 20:09:55 -04:00
mason5052 1a7012c1f5 docs: add tool and model fallback RFC 2026-06-09 20:08:46 -04:00
mason5052 6146bc6b44 docs: add BrowserOS MCP browser backend RFC (#342) 2026-06-09 20:05:37 -04:00
mason5052 620e699e71 docs: address review feedback on install/config guide
- Use the package-path form for the tester utilities
  (go run ./cmd/ctester, go run ./cmd/etester test) instead of shell
  glob / single-file invocations, which are more portable and avoid the
  multi-file compile pitfall.
- Add -verbose to the container etester example so it matches the local
  example.
- Wrap the new README Quick Start pointer across multiple lines; rendered
  Markdown is unchanged.
2026-06-03 23:06:11 -04:00