- Failover: when the primary provider is down, flow and assistant creation now falls back for the tool call ID template.
- Prompts: a stored custom prompt that no longer validates is ignored in favor of the default, with a warning, instead of rendering "<no value>".
- Security: PUT /users/{hash} refuses a password change without users.edit, so a stolen session cookie can no longer reset the account password.
- Langfuse: minio/minio is gone from Docker Hub, so the stack runs cgr.dev/chainguard/minio:latest as root to keep existing volumes readable.
- Installer: the main menu highlights Maintenance while a worker, stack or installer update is waiting.
- Versions: the build revision is the first 7 characters of the commit SHA instead of git rev-parse --short, which depends on the size of the clone.
- Tests: the docker pids-limit test sets its limit after the probes start, so the setup execs no longer exhaust it on slow CI runners.
- Docs: README covers Azure OpenAI through the custom provider and the Neo4j/APOC files a manual Graphiti install needs.
- Reports: added the Azure OpenAI ctester report and regenerated vllm-mixed for Qwen3.8.
- Updated Go version from 1.24 to 1.26.5 in Dockerfile and CI configuration.
- Upgraded various dependencies in go.mod, including AWS SDK and OpenTelemetry packages, to their latest versions for enhanced features and security.
- Refactored Docker client imports from the Docker library to the Moby library for better compatibility with the updated API.
- Adjusted Docker client usage throughout the codebase to align with the new Moby client structure.
The e2e suite and CI gates could report green without checking anything: the
lint/codegen gate never ran on fork PRs, the codegen freshness check diffed the
wrong range, the sticky comment claimed passes it never read, and several
visual/palette/contrast/a11y gates were written so they could not fail. Closing
those gates then exposed real defects.
**E2E and CI.** New coverage for the assistant lifecycle, flow interrupt, live
panels, tab deep links, the report route and its exports, session expiry, and
uploads/downloads (59 → 125 spec cases). Gates now fail on real regressions:
absolute pixel budget, per-tab panel sweeps, scoped a11y waivers, page-error
assertions. CI runs on `pull_request`, ranges from the PR base, routes comments
by PR identity, and redacts stand secrets from the public artifact.
**Markdown editor.** ~20 round-trip and toolbar fixes: table pipe escaping
(nested lists, Go template pipelines, headerless tables), fence validity,
setext and line-leading escapes, inline code containing a backtick, block
toggles under select-all, controls disabled where the command is unavailable or
a table cell cannot hold the block, theme-aware code surface. Two perf fixes cut
1 MB document load from 69s to 4s.
**Accessibility and UI.** Focus returns to whatever opened a dialog, sheet or
menu; icon-only buttons named; `aria-sort` on table columns; syntax tokens
retuned to clear AA in both themes; a dedicated link token. Detail and settings
pages now distinguish authz denials and partial errors from missing records,
survive background refetches without blanking or discarding unsaved edits, and
stop naming exported PDFs `*.pdf.pdf`.
**Backend.** Passwords capped at the 72 bytes bcrypt can hash (over-long ones
previously 500'd after passing validation); four agent options the GraphQL round
trip silently dropped are carried through; a flow whose worker fails to start is
no longer left in the listing; the Graphiti health check retries before
permanently disabling the client.
---------
Co-authored-by: Sergey Kozyrenko <sirozha_blr@tut.by>
The check ran on every push and failed with a bare diff, so a backend schema
change surfaced as an unexplained red frontend job.
Run it only when a codegen input moved — the backend schema, the operations
document, the codegen config, or the lockfile (a codegen bump can change the
output, and skipping it there would let types.ts go stale and fail someone
else's later push). When the compare range can't be resolved (new branch,
force-push, tag) it still runs. On failure it now says which command to run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Reject a non-object JSON body in the mock-LLM (JSON.parse('null') parsed but
reading .tools then threw and killed the process).
- Pin serve-dist's PORT from the config so an ambient PORT can't move it off the
port Playwright waits on.
- Fail lint on any warning, so a Playwright test with no assertion (expect-expect
is a warning) can no longer lint clean.
- Add a CI check that regenerates src/graphql/types.ts and diffs it, so the
compiled operations can't drift from the codegen input the stand validates.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Drop the screenshot threshold to 0.02 (baselines and CI both render in the
pinned container, so there is no host rasterisation noise to absorb) so a
palette change now fails the visual gate; verified 20/20 still match.
- Gate the visual guard on a container marker set by run-visual.sh and the CI
job, not the host OS, so an --update on a Linux workstation cannot overwrite
baselines with host-font pixels.
- retries:0 on the hermetic mock tier so a retry-recovered race fails instead of
merging green; wire @quarantine via grepInvert as the escape hatch.
- Scope globalTimeout to the mock tier so a real-tier run is not aborted mid-retry.
- Verify the visual container against @playwright/test's actual version instead
of a hardcoded literal.
- Key the report's "snapshots differ" advisory on the diffs artifact (uploaded
only on a real snapshot-step failure), not the visual job conclusion; and stop
a transient jobs-API error from silently leaving a stale sticky comment.
- Drop the unconsumed blob report that doubled every uploaded trace.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The gate subscribed to `labeled` so applying `e2e:stand` could start the stand
job, but every gate job carried `if: github.event.action != 'labeled'`. A label
event then ran with e2e-mock skipped, and GitHub counts a skipped required check
as satisfied on the same head SHA — so adding any label to a PR whose e2e-mock
was red published a merge-permitting check over the red one.
Move the stand tier into its own workflow (e2e-stand.yml) that owns the
`labeled` trigger; e2e.yml now fires only on real code events (opened /
synchronize / reopened) and its gate jobs never skip. The stand workflow keeps
the protected environment + reviewer gate, and uploads results.json only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Four independent ways the signal misled:
Adding a label to a PR skips every gate job, yet the report still ran, found no
results file and overwrote a correct green comment with a red "no results". It
now returns early when the run carries no completed mock job.
A run aborted by the global timeout still uploads a results file whose failure
count is zero, so the header could go green over a red job; the verdict now
requires the mock job's own conclusion too.
The report's concurrency key was the branch name alone, so a fork PR opened from
a branch named `main` lost its comment to upstream `main` activity. The gate's
key split `opened` from `synchronize`, leaving both runs live so the older one
could finish last.
Stand failure screenshots carried the stand URL and the login into a public
artifact — screenshots are now tier-aware like traces and videos already were.
Also guard the Tier-2 teardown: the trap is the script's last statement, so a
teardown hiccup turned a passing run red.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The sticky PR comment took its header from the whole-workflow conclusion, so a
visual-snapshot diff — advisory, never a required check — rendered a red ❌ over a
"62 passed, 0 failed" mock-tier stats line, reading as a gate failure it isn't.
Track the header on the mock gate instead (✅ pass / ❌ real failure) and surface a
differing visual job as a separate ⚠️ line pointing at the diff artifact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Real-tier (Tier 2/3) fixes, the blocking set:
- stand-smoke asserts a per-route anchor (breadcrumb title span) instead
of a shared sidebar link that 0-matches on /settings/* and double-matches
on /templates — the smoke was structurally red on 3 of 5 routes.
- stand-tier trace/video forced off (they embedded the session cookie and
the stand password in a public-repo artifact) and the stand project
selects @stand via config grep (the CI step selected zero tests).
- run-local-tier's sandbox filter is anchored to the seeded 9xxxx range
(`^/?pentagi-terminal-9[0-9]{4,}$`) so it can't force-remove a dev's
flow-9 sandbox; sandbox cleanup moved behind the keep-stack gate; compose
logs are captured before teardown.
- mock-LLM wraps the request handler (a non-JSON body crashed the process);
its transcript now issues a real terminal exec via a delegate chain and
flow-run asserts the terminal output; OpenAI-shaped error envelope.
Mock-gate rigor + coverage:
- order-insensitive deep-subset variable matching; unmatched WS subscribes
are now accounted at teardown; REST/GraphQL entries can pin a body/vars
subset; favorite-star asserts the persisted (not optimistic) value.
- new flows/create smoke (the primary journey had no mock coverage), a
graphql-ws protocol unit test against the real client, and a
route-manifest completeness test.
- affected-routes no longer under-scopes shared component dirs.
Hygiene: .mjs tools now linted/prettier'd; dead gitignore line; JSDoc/
sheet aria-label/schema-compat/trend/serve-dist/report-comment fixes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lays the Phase-3 substrate on top of the three tiers.
- schema-compat pre-flight (e2e/tools/schema-compat.mjs): introspects a
target backend's live GraphQL schema and validates every frontend
operation against it, so deploy skew (a renamed/removed field) fails
once, readably, instead of as dozens of red specs. Verified against the
live Tier-2 backend both ways: 105 operations pass, an injected bogus
field is caught with the exact location.
- trend aggregation (trend.mjs): turns a run's results.json into one JSONL
record (p50/p95 spec duration, slowest three, pass/flaky/fail) so slow
regressions are visible, not just green/red; CI appends it to a
90-day-retained artifact.
- diff-scoping (affected-routes.ts + affected.ts): maps a diff to the
manifest routes it touches via each route's owning sources — the
substrate for selective runs and for scoping the exploratory agent.
Pure mapping fn, unit-tested (backend-only → none; shared infra → all;
owned → that route).
- CI: an e2e-stand job (label-gated + a protected Environment whose
reviewers approve before secrets are exposed; schema-compat runs first),
a trend step, and an LLM-independent @stand smoke.
- docs: the stand tier, trend/affected tools, and the LLM advisory recipe
(playwright-mcp + init-agents + guardrails) — the recipe the
deterministic tiers plug into, not a bespoke bot.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
18 baselines (9 manifest routes × light/dark) compared only inside
mcr.microsoft.com/playwright:v<version>-noble, so pixels are identical on
every machine and CI; the wrapper derives the tag from the installed
@playwright/test version, making the pin drift-proof, and a CI guard
fails loudly if the workflow's container tag falls behind.
macOS hosts cannot run the visual project directly (parallel darwin
baselines) nor mount their node_modules into the container (native vite
binaries): the wrapper builds dist on the host and the container serves
it with a dependency-free static server — route mocks intercept API
calls before the network, so no proxy is needed. The xterm canvas is
masked (SwiftShader pixels are driver-dependent). Determinism proven by
back-to-back container runs. The e2e-visual CI job is advisory, never a
required check.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
specs/real/** now exercise the actual agent loop end to end: create flow
-> image/language/title -> tool-call-ID sampling -> subtask_list plan ->
done barrier -> subtask_patch refine -> report_result — with messages
streamed over the real GraphQL websocket and the flow settling in
Waiting. The mock LLM is ~150 lines of Node driven by a deterministic
first-match transcript; the custom provider env seam means zero backend
changes.
The stack is fully isolated from a developer machine:
- own compose project/network/ports (8444/5433), coexists with a dev stack
- --env-file /dev/null so the developer's .env (live keys, DOCKER_HOST,
config paths) never leaks into the e2e backend
- flow ids seeded from 90001: sandbox containers are named
pentagi-terminal-<flowId> on the shared docker daemon, so the range
keeps them clear of dev flows and makes runner cleanup unambiguous
(down -v cannot remove them — the backend spawns them outside compose)
- pentagi healthcheck via busybox wget (the alpine image has no curl),
since the base service has none and up --wait returns too early
Auth for live tiers moved to the canonical setup-project + storageState;
the localStorage seed is now mock-tier-only (a forged client session has
no cookie behind it and the first 401 wipes it). CI gains a nightly/
dispatch-only e2e-local job.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
setup-go's cache:true already caches the build and module caches keyed
on go.sum; the extra actions/cache step restored the same read-only
~/go/pkg/mod on top and failed with "Cannot open: File exists" whenever
both caches hit. Also sets the workflow's GITHUB_TOKEN to contents:read.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The default tier runs the production bundle (vite build + preview)
against a cassette-driven mock of the whole API surface — GraphQL over
HTTP, graphql-transport-ws subscriptions, and REST — so the suite needs
no backend, no keys, and no secrets, and fork PRs can run it.
- mock engine: (operationName, variables) matching with sequenced
entries and world flags (login flips /info guest->user without
call-order coupling); unmatched calls answer 501 and fail the test,
so nothing leaks through the vite preview proxy to a live backend
- ws mock follows the graphql-transport-ws contract the app's client
needs: immediate ack, nothing before ack, streams stay open, delta
cursors survive reconnects, drops use retryable close codes
- clock and timezone pinned on the mock tier: formatDate branches on
isToday/isThisYear, so unpinned cassette dates rot within a day
- cassettes are TS modules typed against the generated GraphQL types,
so schema/operation drift fails the existing tsc gate at compile time
- CI: fork-safe e2e.yml (read-only token, no secrets, no write steps)
plus e2e-report.yml posting a sticky PR comment via workflow_run,
resolving fork PRs by head SHA
- smoke specs (login redirect, form login, authenticated /flows render)
green on the mock tier; e2e/ wired into tsc, eslint, and prettier
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The "Frontend - Install dependencies" step ran pnpm install --frozen-lockfile
with continue-on-error: true, swallowing the ERR_PNPM_OUTDATED_LOCKFILE that the
flag exists to raise. The rest of the frontend pipeline (prettier, lint,
type-check, test) already gates strictly, so a stale pnpm-lock.yaml would only
surface later as a confusing downstream "command not found". Drop the flag so
lockfile drift fails loudly at the install step. Backend steps left as-is.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- add frontend/.nvmrc (24.17.0) as the single source of truth for Node;
GitHub CI reads it via node-version-file, Dockerfile uses node:24.17.0-slim
- bump packageManager to pnpm@11.8.0; drop "corepack prepare pnpm@latest"
so the pnpm version derives from packageManager everywhere
- migrate pnpm onlyBuiltDependencies -> allowBuilds in pnpm-workspace.yaml
(the package.json "pnpm" field is no longer read by pnpm 11)
- add a CI step that fails if the Dockerfile Node tag drifts from .nvmrc
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Prettier / Lint / Test were `continue-on-error: true` (advisory — failures did not fail CI), the same gap that let type errors pile up. Now Prettier, Lint, Type check and Test all block the lint-and-test job (which runs on every branch push).
Prerequisite: `prettier --write` on 5 pre-existing non-conformant files (pages/login.tsx, lib/report/report-pdf.tsx, 3 *.test.tsx) so the now-blocking Prettier check passes — pure formatting, no logic change. Install stays advisory (setup step); backend checks unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vite build strips types and never type-checks, and the old build`s bare tsc (solution config, files:[]) checked nothing — so type errors reached main unnoticed (which is how 76 had accumulated). Now they fail fast everywhere.
- build: `tsc -b && vite build` — `pnpm build` (and the Docker image build, which runs `pnpm run build`) fails on any type error before bundling.
- add `typescript` script (`tsc -b`, checks both app + node project configs).
- ci.yml: blocking "Frontend - Type check" step in lint-and-test (runs on every branch push; docker-build only runs on main/tags).
Verified: a type error makes both `pnpm run typescript` and `pnpm build` exit non-zero (vite never runs); removing it restores a clean build.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
