test(ts): de-flake the normalizer scaling test that reddened v8 CI

The #3359 scaling guard flaked on CI 3.13 (3.2x vs a 3.0x ceiling): at n=1000 the
measurement (~0.03s) is small enough that a GC pause landing in process_time
inflates the ratio. Not a regression — the normalizer path is unchanged; it was
green on 0.9.64 by luck. Harden it: disable GC around the measured section,
measure at 4k/8k where O(n) work dominates and linear converges to a clean ~2x,
and relax the ceiling to 3.5x (still separates linear from quadratic 4x).
Observed locally: 2.10x, stable across repeated runs on 3.10 and 3.13.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
safishamsiandClaude Opus 4.8 committed 2026-09-20 18:53:35 +01:00
1 parent 7ca736cf94
commit 20a20d30d8
1 file changed
+23 -9
+23 -9
View File
@@ -303,6 +303,7 @@ def test_ts_normalizer_scales_linearly_on_large_files():
flaking the ratio. CPU time counts only work actually done, so it isolates
the algorithmic scaling regardless of load.
"""
import gc
import time
def build(n: int) -> bytes:
@@ -314,17 +315,30 @@ def test_ts_normalizer_scales_linearly_on_large_files():
def timed(n: int) -> float:
source = build(n)
start = time.process_time()
_normalize_ts_import_types(source)
return time.process_time() - start
# A GC pause lands in process_time and, on a small measurement, inflates
# the ratio enough to flake (#3154 CI hit 3.2x on a noisy runner). Disable
# it for the measured section so we time only the normalizer's own work.
gc_was_enabled = gc.isenabled()
gc.disable()
try:
start = time.process_time()
_normalize_ts_import_types(source)
return time.process_time() - start
finally:
if gc_was_enabled:
gc.enable()
timed(200) # warm the grammar/parser import off the measured path
small = min(timed(1000) for _ in range(5))
large = min(timed(2000) for _ in range(5))
# Measure at larger sizes than the smoke case: at ~1k the per-call fixed
# overhead is a big fraction of the tiny (~0.03s) measurement, so the ratio
# is noisy; at 4k/8k the O(n) work dominates and linear scaling converges
# toward a clean ~2x. min() over repeats takes the least-preempted run.
timed(500) # warm the grammar/parser import off the measured path
small = min(timed(4000) for _ in range(5))
large = min(timed(8000) for _ in range(5))
# Linear work doubles (~2x). Quadratic work quadruples (~4x). A generous
# 3x ceiling separates the two without being flaky under load.
assert large < small * 3, (
# Linear work doubles (~2x). Quadratic work quadruples (~4x). A 3.5x ceiling
# separates the two with enough headroom to survive CI scheduling noise.
assert large < small * 3.5, (
f"scaling looks super-linear: {small:.4f}s -> {large:.4f}s "
f"({large / small:.1f}x for 2x input)"
)