mirror of
https://github.com/dgtlmoon/changedetection.io.git
synced 2026-09-18 19:36:04 +00:00
dgtlmoonandClaude Opus 5
7a29b5bc73
Page.stopLoading stops the network, not script execution. A page whose JavaScript has pegged the renderer's main thread keeps that thread indefinitely, and every CDP call that needs to run script then queues behind it and never returns - page.content, the xPath scraper, the favicon fetcher. The fetch dies at PUPPETEER_MAX_PROCESSING_TIMEOUT_SECONDS having extracted nothing, with a core spinning the entire time. Seen in production on a watch that failed every check for days: the renderer sat at 1.04-1.07 cores for the full 60s budget (sampled every 2s, flat), 33s of which was a single unanswered Runtime.evaluate, and the watch logged "xpath_data length returned empty" every time. Nothing else recovers this state. Runtime.evaluate's own `timeout` parameter bounds an evaluation once it starts, not time spent queued behind the running task - measured, it still hung past 15s. Wrapping the call in asyncio.wait_for is worse than useless: cancelling a pyppeteer request mid-flight leaves the connection unusable, with "Protocol error: Target closed" on everything after it. Runtime.terminateExecution is what releases the thread. Against a page that fires load and then spins forever, through the real fetcher: before: 60.6s, BrowserFetchTimedOut, 0 bytes content, no xpath_data, no screenshot after: 5.6s, no exception, content + payload, xpath_data present, screenshot 8415b and the renderer drops from 1.00 to 0.08 cores. Safe at this point in the fetch: stopLoading has already declared "give me what rendered", and the content-ready wait above has already had its chance to let late JS-rendered content appear. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>