test(e2e): make the visual gate actually sensitive, and fix its masking race

The suite was carrying Playwright's 0.2 per-pixel default, a tolerance meant for
cross-machine rendering noise that the pinned container removes by construction.
It cost the gate its sensitivity: a real palette change passed while three
baselines silently went stale.

Compare strictly and bound the differing-pixel count instead. Measured over
repeated container runs, glyph antialiasing jitters by exactly 2 pixels, while the
palette change that slipped through moved 71 — so threshold 0 with a 20-pixel
allowance tolerates the noise and catches the class of change that got away.

Strict comparison also exposed a real defect the loose one hid: the flow detail's
ready locator was the header button, but the visual spec masks the terminal, which
mounts later. Capturing in that window compared live terminal pixels against a
masked baseline — a 261k-pixel diff, roughly a third of the frame, appearing at
random. The route now counts as rendered when its terminal exists.

Seven consecutive container runs are clean at the new settings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Sergey Kozyrenko
2026-07-21 15:14:25 +07:00
co-authored by Claude Fable 5
parent 1d98b73660
commit 01cecc2173
2 changed files with 8 additions and 1 deletions
+5
View File
@@ -36,6 +36,11 @@ if (tier === 'stand' && !TIERS.stand.baseURL) {
}
export default defineConfig<BackendOptions>({
// Playwright's 0.2 per-pixel default absorbs cross-machine rendering noise the
// pinned container removes by construction; it also let a palette change through
// while reporting green. Count differing pixels instead: glyph antialiasing
// jitters by 2, while the palette change that slipped moved 71.
expect: { toHaveScreenshot: { maxDiffPixels: 20, threshold: 0 } },
forbidOnly: isCI,
fullyParallel: true,
globalTimeout: isCI ? 10 * 60_000 : undefined,
+3 -1
View File
@@ -44,7 +44,9 @@ export const ROUTE_MANIFEST: RouteManifestEntry[] = [
{
cassette: flowsCassette,
path: routes.flow('5'),
ready: (page) => page.getByRole('button', { name: 'Flow actions' }),
// The terminal mounts after the header, and the visual spec masks it —
// capturing before it exists compares live pixels against a masked baseline.
ready: (page) => page.locator('.xterm').first(),
sources: [
'src/pages/flows',
'src/features/flows',