From fab990d9b96cdf5c8fa16e334a5736ba9dae157f Mon Sep 17 00:00:00 2001
From: Sergey Kozyrenko <sirozha_blr@tut.by>
Date: Sat, 25 Jul 2026 03:19:04 +0700
Subject: [PATCH] fix(ci): don't post a green e2e comment when the run's result
 is unconfirmed
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The sticky comment computed `mockPassed` by falling back to results.json alone
whenever the jobs API was unreachable. A run killed by the global timeout writes
its results.json before the abort is observed — the un-run tests land in
`skipped`, so `unexpected` stays 0 — so that fallback posted a ✅ on a run that
never finished.

Model three states instead of a boolean: pass (zero failures AND the job
confirmed success), fail, and unknown (jobs API unreachable, so the job cannot
be confirmed). Unknown renders ⚠️ with a note to check the run, never ✅.
Verified by driving the extracted script over a timeout-shaped results.json with
a rejecting jobs API: it now yields ⚠️ unknown where it used to yield ✅.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---
 .github/workflows/e2e-report.yml | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/.github/workflows/e2e-report.yml b/.github/workflows/e2e-report.yml
index 8ce581c7..b0acceef 100644
--- a/.github/workflows/e2e-report.yml
+++ b/.github/workflows/e2e-report.yml
@@ -112,19 +112,27 @@ jobs:
               core.info(`results.json unavailable: ${error}`);
             }
 
-            // Both halves are required: results.json is written before an abort can be
-            // observed, so a run killed by the global timeout still reports zero failures.
-            // When the jobs API was unreachable, fall back to results.json alone.
-            const mockPassed = noFailures && (jobsFetched ? mockJob?.conclusion === 'success' : true);
+            // The job conclusion is load-bearing, not a nicety: results.json is written before a
+            // global-timeout abort can be observed (its un-run tests land in `skipped`, so
+            // `unexpected` stays 0), and only the job's own conclusion reflects the abort. So a
+            // green needs BOTH zero failures AND a successful job. When the jobs API was
+            // unreachable we cannot confirm the job, so the verdict is unknown — never a green.
+            const mockStatus = !noFailures || (jobsFetched && mockJob?.conclusion !== 'success')
+              ? 'fail'
+              : jobsFetched
+                ? 'pass'
+                : 'unknown';
 
             // Key the advisory on the e2e-visual-diffs artifact (uploaded only when the snapshot
             // step itself failed), not on the visual job's conclusion — a tag-guard, install or
             // build failure would otherwise read as "snapshots differ".
             const visualDiffers = '${{ steps.visualdiffs.outcome }}' === 'success';
 
-            const conclusion = !mockPassed ? '❌' : visualDiffers ? '⚠️' : '✅';
+            const conclusion = mockStatus === 'fail' ? '❌' : mockStatus === 'unknown' || visualDiffers ? '⚠️' : '✅';
             const marker = '<!-- e2e-report -->';
-            const advisoryLine = visualDiffers
+            const advisoryLine = mockStatus === 'unknown'
+              ? '⚠️ **Result unconfirmed** — the workflow jobs API was unreachable, so this reflects `results.json` alone, which cannot distinguish a clean pass from a run killed mid-flight. Check the run before merging.'
+              : visualDiffers
               ? '⚠️ **Visual snapshots differ** — advisory only, does not block merge. Download the `e2e-visual-diffs` artifact to compare expected / actual / diff.'
               : null;
             const body = [