* LLM restock: parse string prices with price_parser instead of stripping separators
If the model returns the price as page text, e.g. "1.299,95 €", stripping
everything but digits and dots gave 1.29995. price_parser (already a
dependency) handles the locale formats the prompt lists.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* LLM restock: use Restock.parse_currency() like the other price extractors
Same parser c8afea1b (#4494) moved the structured-data extractors to, so
there's one place price text becomes a number. parse_currency() now lists
every caller, and the call sites point back to it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* LLM restock: expose prompt/cache/parsing helpers so other LLM backends can reuse them
Split the hook body into build_restock_messages(), restock_cache_key/get/put()
and parse_restock_reply(), so a plugin talking to a provider without litellm
only has to supply the HTTP call. accumulate_global_tokens() takes an optional
cost_usd, since the default estimate imports litellm's pricing (~200MB that's
never released). The built-in plugin behaves exactly as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test: give the no-litellm subprocess check a PYTHONPATH
CI runs pytest from inside the package dir without installing it, so the
child interpreter couldn't import changedetectionio.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Restock plugins: pass html_to_text() output to the restock hook, let third-party backends go first
get_itemprop_availability_override now also gets `text` (the fetched page through
html_to_text()), only computed when an implementation declares it. The hook is
firstresult and the built-in litellm plugin is trylast, so a plugin that talks to an
LLM with plain requests answers first and the built-in one is never billed as well.
Also fixes the built-in plugin never receiving llm_intent: pluggy silently doesn't
pass hookimpl arguments that have a default value, and it declared llm_intent=None.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Watch.history ran os.path.realpath() twice and os.path.exists() once for every
line in history.txt (~10 syscalls per snapshot), on every watch at startup and
on every history read after that.
basename() already reduces each entry to a plain filename, so the only way out
of the watch data dir is a symlink. One os.scandir() of the data dir now
answers "does it exist", and only symlinks pay for a realpath() containment
check. scandir() never lists '.' or '..', so those are still rejected.
~13x faster on 200 watches x 500 snapshots (1.18s -> 0.09s, warm cache),
identical output. Adds tests for symlinks escaping/inside the data dir,
dangling symlinks, and missing snapshots/blank lines.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Currently dark mode is only available to users that have opted-in by
selecting a cookie. Users that have set their device preference to
dark still see light mode until they've specifically clicked the button
to set the cookie.
Without a css_dark_mode cookie the page now renders data-darkmode="auto",
and the dark styles apply under @media (prefers-color-scheme: dark), so a
device in dark mode gets the dark theme without opting in from the menu.
Pure CSS - no flash of the wrong theme and no JS needed.
The cookie set by the menu toggle still wins either way, so light-mode
devices can choose dark and vice versa. In "auto" the toggle reads the
current effective theme via matchMedia so the first click always flips it.
A dark-mode SCSS mixin emits each dark block for both
html[data-darkmode="true"] and the auto+media-query case; the processor
badge and tag colour rules generated outside SCSS get the same pair.
* PDF - Keep the original PDF of the newest two versions
The history snapshot of a PDF watch is only the pdftohtml text, and the "last fetched" .html.br
for a PDF is just a checksum, so the actual documents were never kept. Store {timestamp}.pdf
alongside the snapshot (same as {timestamp}.html.br does for HTML) and prune to the newest two
history entries, so the latest two versions are always available for a visual PDF compare.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test - Browser PDF test, fix race with an immediate fallback recheck, check the original PDFs are kept
A duplicate queue entry (scheduler + import) can run the html_requests fallback check right
after the failed browser check, so by the time the test looks the error may already be cleared.
Accept both outcomes, the error message itself is covered in tests/test_pdf.py.
Also check both original PDF versions are stored, whichever fetcher got them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* PDF - Checksum the original document not the browser's PDF viewer HTML, browser PDF test
The "unchanged since last check" checksum was taken from fetcher.content, for a PDF in a browser
that is the viewer's HTML wrapper which is identical for every version. Prefer raw_content.
Also check that a PDF is actually available before the checksum skip; previously the viewer HTML
checksum was saved before the PDF error was raised, so the next check skipped as "unchanged",
cleared the error, and the watch silently never recorded anything.
Adds a playwright + puppeteer container test (PDF detected, unchanged PDF is not a change,
new PDF version is), puppeteer must fetch it itself, playwright must fall back to html_requests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PDF - Only use raw_content for the document checksum when it is a PDF
Every other content type keeps checksumming .content exactly as before, so existing
watches keep their checksum basis.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Revert the raw_content document checksum change
Choosing between .content and .raw_content belongs in the general binary content handling
(#1334), which will be done separately. The browser PDF test and the early "PDF not available"
check stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Fetchers - Define raw_content on the base fetcher, give a clear error when a fetcher can't supply the PDF
Fetchers that never set raw_content (puppeteer, playwright, selenium) crashed with
"'fetcher' object has no attribute 'raw_content'" when the response turned out to be a PDF.
Browsers render PDFs in their built-in viewer, so all they have is the viewer HTML; now the
processor reports that plainly instead of crashing or feeding HTML to pdftohtml.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Puppeteer - Capture the real PDF body when the page turns out to be a PDF
Chrome hands a PDF straight to its built-in viewer, after which page.content and
Network.getResponseBody only return the viewer's HTML. Pause main-frame document responses
with the CDP Fetch domain and, when the Content-Type is application/pdf, copy the body out
before it is continued - a single download, and nothing needs to be known in advance.
content/raw_content then match the requests fetcher's binary handling (md5 + original bytes)
so the existing PDF processing and checksum skip work unchanged. Normal HTML pages are
continued immediately, no measurable difference in fetch time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Fetchers - Add supports_pdf capability, only force html_requests for PDFs when the fetcher can't supply them
PDF watches were always forced onto html_requests. Now that puppeteer can capture the original
PDF (useful when the session needs a real browser, Cloudflare etc), only fetchers without
supports_pdf fall back - both at fetch time (resolve_content_fetcher) and in Watch.get_fetch_backend.
Also save the content-type when a fetcher couldn't supply the PDF (it was only saved on success),
otherwise watch.is_pdf never became true for URLs not ending in .pdf and the watch failed on
every check instead of falling back to html_requests on the next one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* LLM restock: parse string prices with price_parser instead of stripping separators
If the model returns the price as page text, e.g. "1.299,95 €", stripping
everything but digits and dots gave 1.29995. price_parser (already a
dependency) handles the locale formats the prompt lists.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* LLM restock: use Restock.parse_currency() like the other price extractors
Same parser c8afea1b (#4494) moved the structured-data extractors to, so
there's one place price text becomes a number. parse_currency() now lists
every caller, and the call sites point back to it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The extractors stripped everything but [\d.] before converting, so the
decimal comma was thrown away before Restock.parse_currency(), which
already understands it, ever saw the value. Route the values through
parse_currency() instead, and teach it that a lone "1,299" is a
thousands separator so that case does not regress.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Annotate fragmented gettext locations with marker comments (#4074)
* update_catalog
* Re-add translator marker comments after UI refactor
* Use `msgctxt` and correct ja translations for -ed forms
# Conflicts:
# changedetectionio/blueprint/ui/templates/edit.html
# changedetectionio/translations/messages.pot
* Set Cache-Control: no-store on dynamic responses by default
Pages with per-session content (settings, CSRF-token-bearing forms,
watch data) had no Cache-Control header, so a misconfigured CDN or
reverse proxy sitting in front of the app could cache and replay them
across requests/sessions — most commonly surfacing as "CSRF tokens do
not match" after the edge served a stale cached page. Routes that
already set their own Cache-Control (static assets, screenshots,
favicons) are left untouched.
* Adding tests and moving function
---------
Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>
