mirror of
https://github.com/dgtlmoon/changedetection.io.git
synced 2026-10-26 00:28:42 +00:00
dgtlmoonandClaude Opus 5.5
44fd71679e
* Fetchers - Define raw_content on the base fetcher, give a clear error when a fetcher can't supply the PDF Fetchers that never set raw_content (puppeteer, playwright, selenium) crashed with "'fetcher' object has no attribute 'raw_content'" when the response turned out to be a PDF. Browsers render PDFs in their built-in viewer, so all they have is the viewer HTML; now the processor reports that plainly instead of crashing or feeding HTML to pdftohtml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Puppeteer - Capture the real PDF body when the page turns out to be a PDF Chrome hands a PDF straight to its built-in viewer, after which page.content and Network.getResponseBody only return the viewer's HTML. Pause main-frame document responses with the CDP Fetch domain and, when the Content-Type is application/pdf, copy the body out before it is continued - a single download, and nothing needs to be known in advance. content/raw_content then match the requests fetcher's binary handling (md5 + original bytes) so the existing PDF processing and checksum skip work unchanged. Normal HTML pages are continued immediately, no measurable difference in fetch time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fetchers - Add supports_pdf capability, only force html_requests for PDFs when the fetcher can't supply them PDF watches were always forced onto html_requests. Now that puppeteer can capture the original PDF (useful when the session needs a real browser, Cloudflare etc), only fetchers without supports_pdf fall back - both at fetch time (resolve_content_fetcher) and in Watch.get_fetch_backend. Also save the content-type when a fetcher couldn't supply the PDF (it was only saved on success), otherwise watch.is_pdf never became true for URLs not ending in .pdf and the watch failed on every check instead of falling back to html_requests on the next one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>