Commit the ZIP response status and headers lazily, on the first byte written, via a small zipStreamWriter. A build failure before any output now returns the normal structured error response instead of a committed 200 with a truncated body; mid-stream failures still abort. Also check the reader Close error in the helper test and cover the pre-stream failure path.
DownloadResource and DownloadFlowFile built the entire ZIP archive in a bytes.Buffer before sending it, so heap usage scaled with archive size and a few concurrent large-directory downloads could exhaust process memory.
Stream the archive straight to the response writer via a shared streamZipArchive helper; the existing ZipResources/ZipDirectory/ZipRelativePaths helpers already accept an io.Writer. Responses are now chunked (no Content-Length) and memory stays proportional to one file's copy buffer.
Address Copilot review on PR #350: note that default_backend selects a crawl backend and passive extractors such as jsfinder are not selectable there; model scope_decision as a decision plus reason in both the field list and the JSON example; add an illustrative scope_entries example and clarify when allowed_hosts applies.
Refs #336
Add examples/proposals/headless_crawler_integration.md, a docs-only RFC proposing an optional, tool-agnostic crawler / URL discovery capability for PentAGI agents. Candidate backends are katana, crawlergo, rad, and jsfinder, framed as candidates with no mandatory default. The RFC keeps dictionary fuzzing (ffuf/dirsearch) unchanged, defines structured discovery artifacts (URLs, forms, parameters, JavaScript endpoints, status codes, source page, depth, scope decision), and keeps crawler URL discovery separate from the BrowserOS MCP interactive browser backend.
Refs #336
Add examples/proposals/kubernetes_deployment.md, an RFC-style design
surface responding to the Kubernetes deployment request in #324. The
document is docs-only: no Helm charts, manifests, operator, compose,
installer, or environment-variable changes, and it does not claim
Kubernetes is supported today.
It records the current Compose/installer deployment assumptions, maps
each one (secrets/config, volumes, service discovery, ingress/TLS,
health checks, network policies, the Docker-socket flow executor,
observability, image overrides, migrations) to candidate Kubernetes
equivalents, and proposes an incremental, docs-first path with open
questions, security/operational considerations, and a test/validation
strategy. The hardest item -- the Docker-socket worker executor -- is
laid out as candidate options without choosing one, and keeps flow
lifecycle explicit and inspectable per the #268 review lesson.
Refs #324
- Drop the hard-coded provider count to avoid doc drift as providers are
added; list the current provider types instead.
- Use the ADC-specific command (gcloud auth application-default login)
rather than the ambiguous 'gcloud login'.
- Reference the enum-swap migration pattern generically under
backend/migrations/sql/ instead of a single timestamped filename that
may be renamed or squashed.
Issue #321 requests a native Vertex AI provider with service-account /
ADC auth, opened after #310 clarified Vertex is not supported today. The
issue carries a full implementation outline and four open questions, and
the author asks for direction on adapter strategy and Gemini-vs-Claude
scope before implementing.
Add examples/proposals/vertex_ai_provider.md, a planning RFC that frames
the work before any code is written:
- Distinguishes the current options (AI Studio gemini, direct Anthropic,
AWS Bedrock, and the custom OpenAI-compatible LiteLLM proxy workaround)
from the proposed native vertex provider.
- Recommends a staged approach: v1 Gemini-on-Vertex with ADC /
service-account auth; Claude-on-Vertex deferred to a maintainer
decision, noting it likely belongs on the Anthropic adapter (Vertex
auth/endpoint mode) rather than the Gemini-shaped path because the
message schema differs.
- Models auth on the existing Bedrock multi-auth precedent (ADC default
chain plus service-account file), and flags that service-account JSON
is sensitive and must be file-mounted/secret-managed, never pasted into
UI or logs.
- Captures config/migration touch points (provider checklist, REST
Valid() whitelist, PROVIDER_TYPE enum-swap migration) so the eventual
implementation size is clear, and restates the issue's open questions.
Docs/RFC only. No code, schema, migration, generated, frontend, or
provider runtime files are touched, and no new env vars or types are
added; every VERTEX_* key and type named is labeled as a candidate. No
overlap with the provider files in open PR #328.
commitlint had no config anywhere (no config file, no husky commit-msg
hook), so its rules were never loaded; the "commit" script also pointed
at an uninstalled binary. Remove @commitlint/cli and
@commitlint/config-conventional, the dead "commit"/"commitlint" scripts,
and the stale commitlint entry in README Development Requirements.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Confirmed unused via depcheck + manual verification:
- anser, js-cookie, @types/js-cookie — no imports anywhere
- rehype-raw — only referenced in a vite manualChunks regex, never
imported as a markdown plugin (markdown.tsx uses rehype-highlight/slug
and remark-gfm); also dropped from that regex
- @graphql-codegen/{client-preset,near-operation-file-preset,typescript}
— codegen config uses explicit plugins (typescript-operations,
typed-document-node), not these presets/base plugin
Also drop the dead package.json "eslintConfig" block — it is ignored by
the flat config (eslint.config.mjs) and referenced an uninstalled
storybook plugin.
Verified: graphql:generate reproduces src/graphql/types.ts unchanged,
plus build, lint, 604 tests, and --frozen-lockfile all pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The plugin was a direct devDependency but never enabled in
eslint.config.mjs (0 active jsx-a11y rules via --print-config). Dropping
it removes dead weight and one stale eslint-9 peer constraint. Lint still
passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- add frontend/.nvmrc (24.17.0) as the single source of truth for Node;
GitHub CI reads it via node-version-file, Dockerfile uses node:24.17.0-slim
- bump packageManager to pnpm@11.8.0; drop "corepack prepare pnpm@latest"
so the pnpm version derives from packageManager everywhere
- migrate pnpm onlyBuiltDependencies -> allowBuilds in pnpm-workspace.yaml
(the package.json "pnpm" field is no longer read by pnpm 11)
- add a CI step that fails if the Dockerfile Node tag drifts from .nvmrc
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The directory expand/collapse control now shows the folder icon by
default and crossfades to a chevron on hover (motion, reduced-motion
aware); the single element toggles expansion on click. Nesting indent
gains the icon->text gap (22px/level) so a child's icon lines up under
its parent's label, and the header expand-all control moves to the
shared Button. Folder and chevron share the text-blue-400 accent set on
the parent (header chevron picks it up on hover).
Also prunes ~46 restatement/justification comments and a dead
collectAllFilePaths export (plus its tests) to match the house style.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Prettier / Lint / Test were `continue-on-error: true` (advisory — failures did not fail CI), the same gap that let type errors pile up. Now Prettier, Lint, Type check and Test all block the lint-and-test job (which runs on every branch push).
Prerequisite: `prettier --write` on 5 pre-existing non-conformant files (pages/login.tsx, lib/report/report-pdf.tsx, 3 *.test.tsx) so the now-blocking Prettier check passes — pure formatting, no logic change. Install stays advisory (setup step); backend checks unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vite build strips types and never type-checks, and the old build`s bare tsc (solution config, files:[]) checked nothing — so type errors reached main unnoticed (which is how 76 had accumulated). Now they fail fast everywhere.
- build: `tsc -b && vite build` — `pnpm build` (and the Docker image build, which runs `pnpm run build`) fails on any type error before bundling.
- add `typescript` script (`tsc -b`, checks both app + node project configs).
- ci.yml: blocking "Frontend - Type check" step in lint-and-test (runs on every branch push; docker-build only runs on main/tags).
Verified: a type error makes both `pnpm run typescript` and `pnpm build` exit non-zero (vite never runs); removing it restores a clean build.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The provider form had 23 errors rooted in Zod v4: `z.preprocess` gives an `unknown` input type, which @hookform/resolvers v5 surfaces as the form field-values type, mismatching `useForm<output>` and cascading to every Control<FieldValues> site.
- Replace `z.preprocess` with explicit helper schemas (proper input/output types) and use `useForm<FormInput, unknown, FormData>`; make the reusable field components generic `<T extends FieldValues>` (Control<T> + FieldPath<T>).
- Fetch `thinking` on the model-config fragment (regenerated types.ts; backend ModelConfig.thinking exists).
- Validate the provider `type` into the ProviderType enum via `z.nativeEnum(...).parse` at the GraphQL boundary instead of an `as` cast (the form intentionally uses watch() to keep disabled fields in the payload, so its values stay input-typed).
tsc: 23 → 0; the frontend now has zero TypeScript errors. 606 tests pass; eslint + prettier clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hack-free fixes (no `as any`/`@ts-ignore`/`as unknown as`):
- tsconfig lib es2023 for Array.findLast; tighten DocumentTitle TitleResolver to ApolloTitleComponent (the broad ComponentType was masked by findLast returning any).
- NodeJS.Timeout → ReturnType<typeof setTimeout> in browser code; models import casing (./User → ./user).
- settings prompt/api-token forms: type against the real generated fragment types and the Zod input/output split (useForm<Input, ctx, Output>); generic FormTextareaItem<T>.
- data-table / detail-nav: noUncheckedIndexedAccess guards; React vs DOM KeyboardEvent disambiguation; drop invalid user-event delay option.
tsc: 76 → 23 (the remaining 23 are one settings-provider Zod/RHF cluster). 606 tests pass; eslint + prettier clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TypeScript 6.0 deprecated `baseUrl` (removed in 7.0). The `@/*` alias resolves relative to each tsconfig without it, and Vite resolves `@` via its own resolve.alias, so baseUrl was vestigial. Dropping it removes the need for `ignoreDeprecations: "6.0"`, which editor-bundled TypeScript (5.x) flagged as an invalid value (TS5103).
Also drop the dead `@env`/`./env.ts` entry from tsconfig.node.json: the file does not exist and the alias is imported nowhere.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the deprecated typescript-react-apollo codegen plugin (unmaintained for Apollo Client v4) with typed-document-node. All ~105 useXxxQuery/Mutation/Subscription call sites are rewritten to the generic useQuery/useMutation/useSubscription(XxxDocument, ...) form, with skipToken replacing skip + conditional-variables.
graphql-codegen.ts: plugins typescript-operations + typed-document-node; add scalars { Time: string } (was unknown); drop withHooks/apolloReact* options. tsconfig.app/node.json: ignoreDeprecations "6.0" for the baseUrl TS5101 deprecation. Regenerate src/graphql/types.ts.
Behavior-preserving: variables, fetchPolicy, error handling, and subscription onData/refetchQueries are unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The /knowledges list query fetches documents with `withContent: false` to save bandwidth, writing an empty `content` to the shared normalized KnowledgeDocument cache entity. That clobbered the full body loaded by the detail query, so opening a document by direct URL (or reload) showed an empty editor.
Add a `content` field merge policy so an empty incoming value never blanks out a body already loaded.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rename a knowledge document via a dedicated mutation that rewrites only the
question in cmetadata — no re-embedding, no embedder required — mirroring the
flows renameFlow pattern instead of round-tripping the full document through
updateKnowledgeDocument.
Backend:
- renameKnowledgeDocument(id, question) mutation + resolver (admin/user split;
ownership enforced at GetUserDocument, like the update pair)
- metadata-only query UpdateKnowledgeDocumentMetadata (no migration)
- unit + edge tests: metadata-only, missing-doc error, non-owner rejection
Frontend:
- renameKnowledge provider method; wire list and detail inline-rename to it
- drop the content "Preview" column and request the list with withContent:false
so it no longer pulls full document bodies
Verified end to end against a local Docker backend (rename works; content and
embedding preserved) and against the remote backend (graceful failure where the
mutation is not yet deployed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace 13 `any` usages with real shapes: ProviderTest / ProviderTestResults
for the test-mutation payload, Control<FieldValues> for the form control, and
inferred element types in the result maps. Collapse the two duplicated
3-level-nested error-formatting blocks into one recursive formatFormErrors
helper — validation error output is byte-identical (verified against the same
inputs).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
react-hooks v6 ships this rule, but it only carries signal once the React
Compiler is enabled; until then it flags every RHF watch() and useReactTable
as unactionable noise (12 hits). Turn it off project-wide and drop the
now-redundant inline disable in use-element-virtual-list.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The CJK-detection regexes embedded a literal U+3000 ideographic space inside
their character class, which ESLint flagged as no-irregular-whitespace (error).
Switch the fullwidth ranges to \u escapes ( -〿-) — identical
match behaviour (verified: Han / U+3000 / fullwidth match, ASCII does not), no
irregular whitespace in source.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
No consumer reads the returned `virtualizer` (data-table destructures only the
spacer / measure fields), and exposing the raw instance leaks the abstraction —
the same dead, over-broad return just removed from useElementVirtualList. Drop it.
Also correct a stale doc claim: react-virtual does have an `enabled` option (it
gates the scroll listeners), the hook just doesn't surface it — so the
conditional-mount guidance stands without the false "no enabled option" note.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The sheet rendered every filtered option (1797 on a busy flows list), so the
filter re-render and the DOM weight (~1800 nodes) made filtering jank. Above a
100-item threshold the listbox now virtualizes via a new element-scroll
useElementVirtualList hook (the inner-container analogue of useWindowVirtualList):
only the ~30-node visible window is mounted. Small lists — and JSDOM tests —
keep rendering in full.
Roving focus onto an off-screen option scrolls it into view and focuses it once
its row mounts (pendingFocusId), so arrows/Home/End and open-time focus on the
current item work across the virtualized window. Options carry aria-setsize /
aria-posinset so screen readers still see the full count and position.
Measured live (chrome-devtools, test.pentagi.net, 1797-flow sheet): DOM nodes
~1800 → ~32; typing INP 258ms → ~200ms. The remaining cost is the flow.tsx page
re-render when the controller's filtered list changes (debounce flush) — a
separate concern. Open / End / Home / typing focus all verified; full suite green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Typing in the sheet's search lagged badly (INP ~600ms on a 1797-item list): the
input bound to the controller's searchQuery, whose state lives in the page-level
hook (flow.tsx), so every keystroke re-rendered the whole detail page AND rebuilt
all ~1800 option rows before the caret could echo — characters appeared in batches.
Mirror the input value in local sheet state (instant caret echo, re-renders only
the sheet) and push to the controller in a transition (filtering + prev/next stay
correct; the heavy re-render no longer blocks the keystroke). Memoize the option
rows so a keystroke that only changes the local mirror does not rebuild them.
Measured live (chrome-devtools, test.pentagi.net, 1797-flow sheet): typing INP
606ms -> 258ms; all characters land, focus stays on the input. The residual is the
DOM reconciliation when the filter changes — addressed next by virtualizing the list.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Focus was driven off `focusedId` through an effect, so a reconciliation-driven
change (the filter shrinking and re-pinning to the first survivor) also moved
DOM focus, yanking it off the search input mid-type. Two document.activeElement
guards (the second added in 5a7bdca) papered over the timing; the rAF re-check
only ever fired under JSDOM and was dead in real browsers — verified live.
Move focus imperatively only where the user navigates: opening the sheet now
uses Radix's onOpenAutoFocus (preventDefault + focus the current option, with no
rAF race against the focus trap), arrow keys focus synchronously in the key
handler, and render-phase reconciliation only updates the roving tabindex. Both
guards, the effect and the rAF are gone, and the previously-flaky focus-steal
test is deterministic (10/10 isolated).
Verified live (chrome-devtools, test.pentagi.net, 1797-flow sheet): open focuses
the current option, typing keeps focus on the input with no dropped chars, and
arrows / ArrowDown-from-input move focus.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The account page tracked a single `editing` section, so opening one editor
unmounted any other open form and silently discarded its unsaved input. Track
an open-section set instead: opening a section no longer closes the others, so
a half-typed draft survives switching between name, email and password.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The forced password-change screen reused the compact account-card footer, so it
rendered small right-aligned buttons that clashed with the full-width "Sign in"
CTA on the same login page. Add two orthogonal props: layout
('horizontal' default | 'vertical') drives footer direction and button width;
buttonSize ('default' default, mirroring Button) drives size. The forced screen
passes layout="vertical" (full-width stacked buttons, submit on top, matching the
login button language); the account cards pass buttonSize="sm" to stay compact.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
isModal (gated the Cancel button) and showSkip (gated the Skip button) were
redundant with the onCancel/onSkip callbacks they always paired with: every
call site that set a flag also passed the matching handler, and none passed a
handler while wanting the button hidden. Render each button from its callback's
presence instead. Removes two props and a name that misdescribed its job
(isModal gated a button, not a modal).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first migration pass missed call sites where the path is nested inside a
helper rather than passed straight to navigate/to: detail-navigation getHref
helpers, mergeHrefWithSearchParams row-open handlers, the knowledge create
redirect, the flow report window.open targets (wiring up the unused
routes.flowReport builder), and the OAuth return_uri. All now build from routes.
Add routes.test.ts: every builder output is asserted to match the app.tsx route
pattern that serves it via matchPath, guarding URL<->pattern sync, plus exact
checks for query encoding and the login return-url guard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Step 3 of the route-registry migration: every /login redirect and post-login landing
fallback builds from lib/routes instead of hardcoded strings.
- protected-route, axios 401/403 interceptor, user-provider (logout, session-expiry,
/login + public-route checks) -> routes.login(...) / routes.login()
- oauth-result error redirect -> routes.login()
- login-form / public-route / login page post-login fallback -> routes.newFlow
getReturnUrlParam now has a single in-app caller (the routes.login builder) plus the
logout suffix-override; dropped its now-unused imports from axios and protected-route.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Step 2 of the route-registry migration: the provider/prompt settings pages build their
navigate() targets from lib/routes instead of hardcoded strings.
- settings-providers: provider(id), newProvider({ id | type }), newProvider()
- settings-provider: routes.settings.providers (x5)
- settings-prompts: routes.settings.prompt(name)
- settings-prompt: routes.settings.prompts (x2)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Introduce lib/routes.ts as the single source of truth for client route paths, with typed
builders for parameterised routes (flow, provider, prompt, login return-url, ...). First
migration step covers the settings shell we own:
- app.tsx settings index/catch-all redirects -> routes.settings.account
- main-sidebar Settings/Profile links -> routes.settings.root / .account
- settings-layout menu items + the "Back to App" fallback -> routes.*
The rest (settings provider/prompt pages, auth redirects, main nav) migrate in follow-ups.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Account was added as the first settings sidebar item, but the index and catch-all
routes still redirected to /settings/providers, so the gear "Settings" link dropped
the user on the second item. Point both redirects at /settings/account.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(displayName).charAt(0) returns the first UTF-16 unit, so a non-BMP first character
(emoji, rare CJK) rendered as a lone surrogate (�). Spread to iterate by code point.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The settings shell's only exit was a "Back to App" link hardcoded to /flows, so opening
Settings or Profile from anywhere dropped the user at the flows list on the way out.
- the Settings and Profile entry links pass the current path as location.state.from
- SettingsLayout captures it once on entry (surviving sub-tab navigation, which drops
state) and points "Back to App" there, falling back to /flows
- tests: SettingsLayout honors the origin / falls back / preserves it across sub-tabs;
MainSidebar's Settings and Profile links carry the origin
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The three account forms duplicated the getApiErrorCode + ERROR_BY_CODE lookup pattern
verbatim, and each carried a dead `AuthRequired` entry that the axios interceptor
already handles (hard redirect) before the form's catch can render it.
- add resolveApiErrorMessage(err, map, fallback) next to the axios error helpers
- email/name/password forms use it and drop the unreachable AuthRequired entry
- unit-test the helper's precedence (mapped code > server msg > fallback)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A network blip or 5xx on an /info refresh — after an email/name change or on plain
navigation — wrongly cleared local auth and bounced the user to /login, even though the
session cookie was still valid (the axios interceptor already handles real 401/403).
- refreshAuthInfo and the navigation refresh now keep the current session on a transient
/info failure instead of clearing auth and redirecting to /login
- add patchUser so the email/name forms reflect the change immediately, dropping the
dependency on a successful /info round-trip for the visible field
- tests: UserProvider keeps the session on transient failure (refresh and navigation
paths); forms call patchUser with the new value
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- settings-account: local accounts expose name/email/password; OAuth accounts hide
the password card and email editing, keep the name editable, and label the provider
(known label, raw fallback, then generic); renders nothing without a user
- name-change-form (previously untested): seeds the current name, submits the trimmed
value and refreshes auth, blocks an empty name, maps Users.NotFound to friendly copy
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ChangeNameCurrentUser issued a bare UPDATE and ignored RowsAffected, so a stale
session for a deleted user got 200 — inconsistent with the email/password handlers
(which 404) and leaving the frontend's Users.NotFound mapping unreachable.
- check RowsAffected and return ErrUsersNotFound (404) when no row matched
- document the 404 in the swagger annotation
- add TestChangeNameCurrentUser (success, missing user, invalid name)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The vmail validator rejected addresses the frontend's z.string().email() accepts
(uppercase, TLDs longer than 4 chars like .cloud), so users saw a confusing 400.
- relax the vmail regex to allow uppercase and TLDs of 2+ chars
- lowercase + trim the new address in ChangeEmailCurrentUser and in the form schema
so storage, the duplicate check, and login lookups stay case-stable
- validator + change-email tests for mixed case and long TLDs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
