541 Commits

Include renames

Author	SHA1	Message	Date
dgtlmoon	318e07388e	Improvements to plugin architecture	2026-10-24 15:48:57 +02:00
dgtlmoonandClaude Opus 5.5	352b036217	API - Reject broken Jinja2 templates in url, link_to_open, body and headers (shared with the edit form) (#4587) ... * API - Reject broken Jinja2 in url, link_to_open, body and headers, shared validation with the edit form The edit form validated Jinja2 templates on save but the API did not, so a broken body or header template saved through the API only failed at fetch time, on every check. - New jinja2_custom/validate.py: validate_jinja2_fields() renders each value that matches JINJA2_MARKER_PATTERN and returns {field: translated error} - The edit form uses it instead of four copies of the same try/except - API create and update return 400 naming the field Translations: messages moved to the new file (msgids unchanged); also translated the header error for fr/it and fixed the cs/es wording. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * rebuild --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-23 18:15:52 +02:00
dgtlmoonandClaude Opus 5.5	2bd1a87403	Tests - Fix test_itemprop_percent_threshold for WTForms 3.3 (FloatField is now type="number") (#4588) ... * Tests - Don't assert the input type of the restock threshold field, WTForms 3.3 renders FloatField as type="number" wtforms~=3.2 now installs 3.3.0, which changed FloatField's widget from a text input to a number input, so test_itemprop_percent_threshold failed on every branch. The value is still saved and rendered correctly; only check that. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Tests - Add get_form_field() helper (BeautifulSoup), use it for the restock threshold value instead of a regex Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-23 17:51:24 +02:00
dgtlmoonandClaude Opus 5.5	ce91b85220	Favicon - Clear the saved favicon when the watch URL changes (#4569) (#4586) ... The old site's icon was kept until it aged past the 24h refetch threshold, or forever when the new site has no icon or the watch uses a fetcher that doesn't fetch favicons. Watch.update() now clears favicon.* and the filename cache when the URL changes, which covers both the UI edit form and the API. favicon_is_expired() then sees no file and the next check fetches the new icon. A change from an empty URL is ignored, since every fresh Watch object starts with url '' and is filled from storage via update(). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-23 15:33:26 +02:00
4 people	f20c82fb15	Fix Jinja2 false-positive when URL contains {% (#4208) ... * Add tests for Jinja2 template detection in URL validation (RED) * Fix Jinja2 false-positive when URL contains `{%` (GREEN) * Consolidate Jinja2 template detection across URL/body/headers validation * Propagate `FORCE_COLOR` to test containers * Apply suggestion from @dgtlmoon * Apply suggestion from @dgtlmoon * Only render request headers/body at fetch time when they contain Jinja2, drop FORCE_COLOR changes The edit form now skips Jinja2 validation for text like {%22 that is not a template, but processors/base.py still rendered every header and the body unconditionally, so a body such as q={%22key%22:1} was accepted on save and then failed every check with a TemplateSyntaxError. Use JINJA2_MARKER_PATTERN there too, so it is sent as-is. Also reset the CI workflow and run_*_tests.sh scripts to master, the FORCE_COLOR changes are unrelated to this fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: dgtlmoon <dgtlmoon@gmail.com> Co-authored-by: dgtlmoon <leigh@morresi.net> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-23 15:32:32 +02:00
dgtlmoonandClaude Opus 5.5	94d3404d88	Notifications - |regex_replace ReDoS timeout now works in any thread (use 'regex' package timeout=) (#4585) ... * Notifications - |regex_replace ReDoS timeout now works in any thread, use the 'regex' package timeout= instead of SIGALRM signal.alarm() can only be set from the main thread, but notifications are rendered in the notification runner threads and previews in the Flask request threads, so since #4577 the timeout was effectively never applied in production. Re #3501, follow-up to #4577 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * regex_replace - Allow the timeout to be set with the REGEX_TIMEOUT_SECONDS env var Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * regex_replace - Import 'regex' lazily inside the filter, saves ~2MB RSS per instance that never uses it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-23 15:31:51 +02:00
dgtlmoonandClaude Opus 5	1dd6291667	Judge favicon expiry on the icon the watch actually has, not whatever glob lists first (#4434) ... * Judge favicon expiry on the icon the watch actually has, not whatever glob lists first bump_favicon() names the saved file after the icon's type (favicon.png, favicon.ico, ...) and did not remove a previous save under a different extension, so a watch whose icon changed type ends up holding more than one favicon.*. favicon_is_expired() then picked next(iter(glob.glob("favicon.*"))). glob order is os.scandir order - stable, but decided by filename hash - so for some extension pairs the stale file wins every single time, its age always exceeds the 24h threshold, and the favicon is refetched on every check forever. Measured across realistic pairs, 3 of 10 lose that way: old=.png new=.ico -> glob lists favicon.png first -> always expired old=.ico new=.svg -> glob lists favicon.ico first -> always expired old=.png new=.webp -> glob lists favicon.png first -> always expired That is an in-page network fetch through the watch's proxy on every check, plus the favicon fetcher's own sequential per-icon timeouts, for a favicon that is already saved and current. Fixed at both ends: pick the newest candidate by mtime, and have bump_favicon() drop superseded files so the ambiguity stops arising. Also tidied two things in the same function: - os.path.isfile(None) raised a TypeError when the module-level filename cache said yes after the file had been removed, which the broad except turned into a logger.critical() for an entirely benign condition. The empty case is now handled explicitly. - the broad except stays (this runs inline on the check path as an argument to fetcher.run(), so it must never fail a watch) but logs at warning, since the only cost of getting this wrong is one extra favicon fetch. Tests fail on master (favicon.ico left behind; fresh icon reported expired) and pass here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Favicon - write the new icon before removing superseded ones, so a failed write keeps the old icon Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-10-23 12:04:17 +02:00
Alexandr Kazmin	04bcf5ea5a	Decode proxy credentials for browser fetchers and redact them in logs (#4502) ... Proxy usernames and passwords with special characters have to be URL-escaped (#2062). The requests fetcher decodes them, but the Playwright and Puppeteer fetchers and Browser Steps passed the escaped values to the browser as-is. The proxy URL, password included, was also logged at DEBUG, the default level. Add proxy_credentials() and redact_proxy_url() in changedetectionio/proxy_url.py and use them at those sites, with unit tests.	2026-10-23 12:01:23 +02:00
yi111andYi-111-a	b82750421b	Fix IndexError on empty/whitespace-only JSON content (#4534) ... extract_json_blob_from_html() and extract_json_as_string() both peeked at content_start[0] to decide whether the body looks like JSON. A blank body leaves content_start empty, so the peek raised IndexError instead of the watch reporting that no JSON was found. Guard the index in both places so empty and whitespace-only content takes the same JSONNotFound path as any other non-JSON body. Fixes #4531 Co-authored-by: Yi-111-a <yi-111-a@users.noreply.github.com>	2026-10-23 12:00:11 +02:00
James TannahillandClaude Opus 5.5	e3e51a3a33	Log puremagic "Could not identify file" at debug instead of warning (#4533) ... * Log puremagic 'Could not identify file' at debug, not warning puremagic raises PureError when no signature matches, which is the normal result for plain text and most HTML pages. guess_stream_type() caught it with every other exception and logged a warning on many checks, although the content-based detection that follows handles it as designed. Other puremagic failures still log a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Skip puremagic on empty content and catch PureError directly puremagic.magic_string() raises ValueError('Input was empty') for empty input, which is as normal as an unidentified signature, so skip the call then. Import puremagic before the try (it is a hard dependency) so the except clause can name PureError instead of comparing the class name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep the puremagic import inside the try so a missing library falls back ImportError is caught before puremagic.PureError, so that clause is only evaluated once the import has succeeded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-23 11:58:55 +02:00
Enes Uygur	d88f8d7b0b	Notifications - Fix |regex_replace doing nothing when rendered outside the main thread (#4577)	2026-10-23 11:47:47 +02:00
dgtlmoonandClaude Opus 5.5	be09f73c3a	Restock: let a plugin do LLM price/stock extraction without litellm (HTML + text hook, reusable helpers) (#4545) ... * LLM restock: parse string prices with price_parser instead of stripping separators If the model returns the price as page text, e.g. "1.299,95 €", stripping everything but digits and dots gave 1.29995. price_parser (already a dependency) handles the locale formats the prompt lists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * LLM restock: use Restock.parse_currency() like the other price extractors Same parser c8afea1b (#4494) moved the structured-data extractors to, so there's one place price text becomes a number. parse_currency() now lists every caller, and the call sites point back to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * LLM restock: expose prompt/cache/parsing helpers so other LLM backends can reuse them Split the hook body into build_restock_messages(), restock_cache_key/get/put() and parse_restock_reply(), so a plugin talking to a provider without litellm only has to supply the HTTP call. accumulate_global_tokens() takes an optional cost_usd, since the default estimate imports litellm's pricing (~200MB that's never released). The built-in plugin behaves exactly as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test: give the no-litellm subprocess check a PYTHONPATH CI runs pytest from inside the package dir without installing it, so the child interpreter couldn't import changedetectionio. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Restock plugins: pass html_to_text() output to the restock hook, let third-party backends go first get_itemprop_availability_override now also gets `text` (the fetched page through html_to_text()), only computed when an implementation declares it. The hook is firstresult and the built-in litellm plugin is trylast, so a plugin that talks to an LLM with plain requests answers first and the built-in one is never billed as well. Also fixes the built-in plugin never receiving llm_intent: pluggy silently doesn't pass hookimpl arguments that have a default value, and it declared llm_intent=None. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-19 12:32:32 +02:00
dgtlmoonandClaude Opus 5.5	9686c3e217	Watch history index - Replace per-snapshot realpath()/exists() with a single directory listing (#4562) ... Watch.history ran os.path.realpath() twice and os.path.exists() once for every line in history.txt (~10 syscalls per snapshot), on every watch at startup and on every history read after that. basename() already reduces each entry to a plain filename, so the only way out of the watch data dir is a symlink. One os.scandir() of the data dir now answers "does it exist", and only symlinks pay for a realpath() containment check. scandir() never lists '.' or '..', so those are still rejected. ~13x faster on 200 watches x 500 snapshots (1.18s -> 0.09s, warm cache), identical output. Adds tests for symlinks escaping/inside the data dir, dangling symlinks, and missing snapshots/blank lines. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-16 11:43:23 +02:00
Tim Bond	22369731e3	UI - Follow the device's light/dark setting by default (#4507) ... Currently dark mode is only available to users that have opted-in by selecting a cookie. Users that have set their device preference to dark still see light mode until they've specifically clicked the button to set the cookie. Without a css_dark_mode cookie the page now renders data-darkmode="auto", and the dark styles apply under @media (prefers-color-scheme: dark), so a device in dark mode gets the dark theme without opting in from the menu. Pure CSS - no flash of the wrong theme and no JS needed. The cookie set by the menu toggle still wins either way, so light-mode devices can choose dark and vice versa. In "auto" the toggle reads the current effective theme via matchMedia so the first click always flips it. A dark-mode SCSS mixin emits each dark block for both html[data-darkmode="true"] and the auto+media-query case; the processor badge and tag colour rules generated outside SCSS get the same pair.	2026-10-16 11:36:52 +02:00
dgtlmoon	c58d12245e	UI - Fixing notification settings still trying to access main settings form that caused a crash on save (#4561)	2026-10-16 11:05:23 +02:00
dgtlmoonandClaude Opus 5.5	ce777a6d60	PDF - Keep the original PDF of the newest two versions (#4549) ... * PDF - Keep the original PDF of the newest two versions The history snapshot of a PDF watch is only the pdftohtml text, and the "last fetched" .html.br for a PDF is just a checksum, so the actual documents were never kept. Store {timestamp}.pdf alongside the snapshot (same as {timestamp}.html.br does for HTML) and prune to the newest two history entries, so the latest two versions are always available for a visual PDF compare. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Test - Browser PDF test, fix race with an immediate fallback recheck, check the original PDFs are kept A duplicate queue entry (scheduler + import) can run the html_requests fallback check right after the failed browser check, so by the time the test looks the error may already be cleared. Accept both outcomes, the error message itself is covered in tests/test_pdf.py. Also check both original PDF versions are stored, whichever fetcher got them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-13 15:53:18 +02:00
dgtlmoonandClaude Opus 5.5	ff6a4c2fc0	Test - Puppeteer/Playwright PDF - Checksum the original document not the browser's PDF viewer HTML, browser PDF test (#4548) ... * PDF - Checksum the original document not the browser's PDF viewer HTML, browser PDF test The "unchanged since last check" checksum was taken from fetcher.content, for a PDF in a browser that is the viewer's HTML wrapper which is identical for every version. Prefer raw_content. Also check that a PDF is actually available before the checksum skip; previously the viewer HTML checksum was saved before the PDF error was raised, so the next check skipped as "unchanged", cleared the error, and the watch silently never recorded anything. Adds a playwright + puppeteer container test (PDF detected, unchanged PDF is not a change, new PDF version is), puppeteer must fetch it itself, playwright must fall back to html_requests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * PDF - Only use raw_content for the document checksum when it is a PDF Every other content type keeps checksumming .content exactly as before, so existing watches keep their checksum basis. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Revert the raw_content document checksum change Choosing between .content and .raw_content belongs in the general binary content handling (#1334), which will be done separately. The browser PDF test and the early "PDF not available" check stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-13 14:44:28 +02:00
dgtlmoonandClaude Opus 5.5	44fd71679e	Puppeteer fetcher - PDF improvements, Handle PDF fetching by Puppeteer directly, cleanup supports (#4547) ... * Fetchers - Define raw_content on the base fetcher, give a clear error when a fetcher can't supply the PDF Fetchers that never set raw_content (puppeteer, playwright, selenium) crashed with "'fetcher' object has no attribute 'raw_content'" when the response turned out to be a PDF. Browsers render PDFs in their built-in viewer, so all they have is the viewer HTML; now the processor reports that plainly instead of crashing or feeding HTML to pdftohtml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Puppeteer - Capture the real PDF body when the page turns out to be a PDF Chrome hands a PDF straight to its built-in viewer, after which page.content and Network.getResponseBody only return the viewer's HTML. Pause main-frame document responses with the CDP Fetch domain and, when the Content-Type is application/pdf, copy the body out before it is continued - a single download, and nothing needs to be known in advance. content/raw_content then match the requests fetcher's binary handling (md5 + original bytes) so the existing PDF processing and checksum skip work unchanged. Normal HTML pages are continued immediately, no measurable difference in fetch time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fetchers - Add supports_pdf capability, only force html_requests for PDFs when the fetcher can't supply them PDF watches were always forced onto html_requests. Now that puppeteer can capture the original PDF (useful when the session needs a real browser, Cloudflare etc), only fetchers without supports_pdf fall back - both at fetch time (resolve_content_fetcher) and in Watch.get_fetch_backend. Also save the content-type when a fetcher couldn't supply the PDF (it was only saved on success), otherwise watch.is_pdf never became true for URLs not ending in .pdf and the watch failed on every check instead of falling back to html_requests on the next one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-13 11:50:43 +02:00
dgtlmoonandClaude Opus 5.5	7ec79fc231	LLM restock: fix locale-formatted string prices (1.299,95 read as 1.29995) (#4544) ... * LLM restock: parse string prices with price_parser instead of stripping separators If the model returns the price as page text, e.g. "1.299,95 €", stripping everything but digits and dots gave 1.29995. price_parser (already a dependency) handles the locale formats the prompt lists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * LLM restock: use Restock.parse_currency() like the other price extractors Same parser c8afea1b (#4494) moved the structured-data extractors to, so there's one place price text becomes a number. parse_currency() now lists every caller, and the call sites point back to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-13 11:21:06 +02:00
DevNullanda-ahmed	6a23dcdca5	UI - Language selector should be available on the login page (#4490) (#4501) ... Co-authored-by: a-ahmed <8526328+a-ahmed@users.noreply.github.com>	2026-10-08 10:30:05 +02:00
contentforge-press	e8da5fc1f0	feat(jinja2): add unixtime filter to format Unix timestamps in notifications (#3641) (#4503)	2026-10-07 16:24:24 +02:00
Jaime KingandClaude Opus 5.5	c8afea1b05	Restock & Price detection - Fix decimal-comma prices ("39,99") being read as 3999 (#4493) (#4494), Warning: Could change previous values that were wrongly stored as 4999 -> 49.99 for example ... The extractors stripped everything but [\d.] before converting, so the decimal comma was thrown away before Restock.parse_currency(), which already understands it, ever saw the value. Route the values through parse_currency() instead, and teach it that a lone "1,299" is a thousands separator so that case does not regress. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-02 00:04:52 +02:00
dgtlmoon	6704fdb29e	Title extract - Limit search window and improve regex - reduce memory usage (#4487)	2026-09-25 11:26:21 +02:00
dgtlmoon	167597628d	LLM - History/difference page - CSRF fix for LLM summary fetching between versions	2026-09-20 20:29:36 +02:00
dgtlmoon	ed217644fa	Our first basic PWA mobile integration! Look for the QR-Code in Settings (#4472)	2026-09-20 19:45:11 +02:00
dgtlmoon	239a69dcc4	UI - Prevent accidental change of LLM Key (#4473)	2026-09-20 19:44:06 +02:00
dgtlmoon	de4637c0bd	LLM - Fixes for temperature/reasoning GPT-5 etc	2026-09-20 18:09:55 +02:00
dgtlmoon	4da220be62	LLM timeout improvements (frontend) (#4470)	2026-09-20 15:59:58 +02:00
dgtlmoon	a165df2da9	LLM UI - When LLM is disabled, hide it from the groups settings also (#4471)	2026-09-20 15:54:41 +02:00
dgtlmoon	dd588bf3e3	Make LLM summary from UI non blocking (#4467)	2026-09-20 13:01:14 +02:00
dgtlmoon	1b7219755f	Groups - UI fix for #4451 and tweaks (#4466)	2026-09-20 09:11:16 +02:00
dgtlmoon	bb37ea23d2	System - Ensure import+API update values are verified types (#4458)	2026-09-18 13:55:50 +02:00
dgtlmoon	5e5e0d8f36	Adding HTTP browser caching of UI elements (#4450)	2026-09-17 16:16:42 +02:00
Dennis Gaidaanddgtlmoon	b91082b38a	Set Cache-Control: no-store on dynamic responses by default (#4319) ... * Set Cache-Control: no-store on dynamic responses by default Pages with per-session content (settings, CSRF-token-bearing forms, watch data) had no Cache-Control header, so a misconfigured CDN or reverse proxy sitting in front of the app could cache and replay them across requests/sessions — most commonly surfacing as "CSRF tokens do not match" after the edge served a stale cached page. Routes that already set their own Cache-Control (static assets, screenshots, favicons) are left untouched. * Adding tests and moving function --------- Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>	2026-09-17 11:20:31 +02:00
dgtlmoon	36ca14df82	UI - Make the 'share watch' optional, reduces visual noise, not really used by non-power users (#4448)	2026-09-17 10:54:43 +02:00
4 people	77fb923de6	fix(http): send a single Date header on werkzeug built-in server (#4372) ... * fix(http): send a single Date header on werkzeug built-in server Static resources served via werkzeug send_from_directory/send_file get a Date header injected into the WSGI response by make_conditional() (werkzeug/wrappers/response.py:752-757). When the app runs on Werkzeug's built-in server -- the default path started through socketio.run(..., allow_unsafe_werkzeug=True) in changedetectionio/__init__.py:694 and used by the docker entrypoint -- BaseHTTPRequestHandler.send_response() (werkzeug/serving.py:271) emits its own Date header line as well, so the wire response carries two Date headers. RFC 9110 forbids this and nginx rejects the response with "upstream sent duplicate header line" (issue #4299, see also #4101). Fix: a global after_request hook pops the application-side Date copy so only the server's single header reaches the wire. Verified safe on gunicorn too, which also emits its own Date header. Test: new tests/test_duplicate_date_header.py hits the live_server over real HTTP with http.client (the Flask test client talks to the WSGI app directly and never sees the server-added header) and asserts the Date header appears exactly once, on the exact static resources named in the issue. Fails on unfixed code with two identical Date lines; passes with the fix. Fixes #4299 * Apply suggestion from @dgtlmoon * Tidy the #4299 Date header fix and its test flask_app.py: the applied suggestion landed with a 3-space indent and trailing whitespace - the latter was the only W291 in the file, which .ruff.toml selects. test_duplicate_date_header.py: - drop the 10s socket wait loop, pytest-flask's live_server already blocks until the port accepts connections - drop the unused `app` fixture argument (live_server depends on it) - stop hardcoding jquery-3.6.0.min.js: asserting 200 on a vendored filename turns a jQuery bump into a failure in a file about HTTP headers. Any send_from_directory() response exercises the same path, so styles.css alone is enough. Still red before the fix (two identical Date lines) and green after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: dgtlmoon <leigh@morresi.net> Co-authored-by: dgtlmoon <dgtlmoon@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 07:16:00 +02:00
5 people	a5f85da4af	UI - Keep history diff title and controls visible while scrolling (#4412) ... * Keep history diff title and controls visible while scrolling The watch label, version selectors, diff options and tabs move into a sticky #diff-header, and the top menu above it (watch URL, EDIT, theme and GitHub links) becomes sticky too, so the whole header stays on screen while the diff scrolls. The minimap and anchor jumps offset by the measured height of both bars instead of a fixed value. Heights are measured on load and kept current with a ResizeObserver where it is available, falling back to the window resize and hashchange events as the restock graph and queue sparkline already do, plus one deferred measure at load - the tab switch hides #settings and so resizes the header without a window resize, and opening a link already at #screenshot does the same with no hashchange to follow. The app header is stored before the diff header is measured, since the latter's cap is derived from the former. A sticky bar must be opaque to the content scrolling beneath it, but the page's own backdrop is a fixed full-viewport gradient layer over the page colour, so a flat --color-background panel reads as a white band cutting across it. Instead each bar repaints that same backdrop via a shared page-surface-gradient mixin: the three gradient stops pre-composited against --color-background-page with color-mix() at the layer's 0.91 opacity, with background-attachment: fixed so they stay registered with body::after. Children of #diff-header are centred with a flex column, mirroring the align-items: center that section.content applies to the non-sticky siblings, so the controls panel and tabs keep their intrinsic width and standard colours. The top menu is made sticky only under body.difference-page. Upstream already wants this globally (see the @todo in parts/_top_menu.scss) but held off because the bar has no background of its own; scoping it here keeps that decision separate. Sticky makes .header a stacking context, so its mobile drawer is capped at the bar's z-index of 30 - high enough to cover #diff-header, low enough that the action rail's hover flyout and toast notifications still paint over the bar as before. The activity strip is fixed on body in the root stacking context, so it would have painted through the capped drawer; on this page it drops just below the bar rather than raising the bar past the rail. Dropping the 40px section.content padding and tightening the title's own padding closes the gap between the two bars, and #diff-header's half-viewport cap now excludes the top menu so the sticky stack stays within the same budget on short viewports. .app-main's 0.55rem gap goes too on this page: it sits between the two sticky bars, so the diff header would otherwise start 8.8px below the top menu and slide up to meet it over the first 8.8px of scroll. That gap cannot survive sticking - it is outside both bars, so the diff would scroll through it - and the bars are flush at every scroll position instead. Co-authored-by: Engineer <engineer@agents.matrixsi.com> Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Condense the history diff header into a compact toolbar The sticky bar the previous commit introduced takes 297px of a 1000px desktop viewport and 422px of a 390x844 phone - at that width its own 50dvh cap is already clipping it, so the shipped page scrolls a header inside itself before the diff has moved at all. Compact it to three rows and that becomes 154px and 228px, without hiding any control. The watch title takes the full width of the bar on its own row above the controls, centred, 0.9rem, one line with an ellipsis. It truncates only when it is wider than the whole viewport - a 184-character title shows in full at 1440px and ellipsizes at 768 and 390 - and the bar keeps exactly the same height either way, which a two-line clamp would not. The full text goes in the title attribute so hover reveals what the ellipsis hides, the same bargain the watch URL above it already makes. The From/To labels shrink to chips rather than hiding. display: none takes a <label> out of the accessibility tree: verified through Chromium's accessibility tree, the selects report "From" and "To" with the chips and "" without them. Sighted users need them too, since two identically formatted datetime selects side by side have only their order to tell them apart, and on a phone they stack so even that stops helping. The chips cost horizontal space and no height at all. Dropping the shipped width: 4rem sizes them to their text, and "From" is wider than "To", so a min-width floor keeps the stacked selects flush. The seven diff options move behind a Filters button. The fieldset stays inline, inside the form, until diff-overview.js swaps it for the popover, so with scripting off the options are exactly as reachable as they are today. The panel is position: fixed, not absolute: #diff-header carries overflow: auto to enforce its 50dvh cap, which clips an absolutely positioned descendant at the bar's bottom edge. The bar is sticky with no transform, filter or contain, so it is not a containing block for fixed and the panel escapes the clip; the script parks it under the button and keeps it inside the viewport, with aria-expanded, Escape, outside-click and a reposition on resize. The keyboard-nav prompt and the two button words are hidden, leaving the arrows; both links carry the full wording as aria-label and title so the accessible name survives the CSS. The tabs' vertical padding halves on this page, 11px off every scroll position, leaving their horizontal padding and so their widths alone. Narrow viewports need one more thing, and font-size alone cannot give it: once the chip takes inline width the select clips its own displayed value, and two snapshots from the same day differ only by the meridiem, so they read identically while closed. #settings, #diff-form, .diff-fieldset and the span are all sized to max-content, which is the select's text, so the container shrinks by exactly as much as the text does - headroom measured 0.0px at every size from 0.9rem to 0.7rem. The chain is pinned to the row first and only then does 0.75rem buy room: +19.3px on the demo pair and +6.3px against the longest realistic English value, which still clips at 0.8rem. All of it hangs off the one 700px breakpoint, and every selector in it carries an ID because the rules it overrides are written with one and a class-only override would lose silently. Co-authored-by: Architect <architect@agents.matrixsi.com> Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Rebuild translation catalog for the new diff header strings The compact header adds three msgids - Filters, Previous version and Next version - and the lint-translations job re-runs extract_messages, update_catalog and compile_catalog and fails on any resulting diff, so the catalog has to travel with the strings that created it. It failed on 297bd365 and took the four test-application jobs down with it, since they depend on it. Purely mechanical: the extract adds exactly those three entries to messages.pot and to each of the seventeen catalogs, all with an empty msgstr, and changes nothing else. The .mo files are unchanged, because an empty msgstr is not written to the compiled catalog. POT-Creation-Date is deliberately left at its committed value rather than restamped. The job filters that line out of the diff it checks, so restamping buys nothing, and 0.60.4 rewrites the same header lines - a restamp here would conflict with the release commit over a timestamp. Kept out of the header commit so the change being reviewed stays the template, the stylesheet and the script. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Make the sticky diff header full-bleed and pin its cap to svh Two iOS-only faults in the sticky bar, both reported from a real iPhone and neither reproducible in Chromium, emulated or not. Both are properties of 43b07d5b rather than of the compact toolbar on top of it. section.content insets its children by $common-gap so they clear the left rail, while the top menu above stays flush against it, so the bar left an 8.8px strip of real page background down each side. That strip is invisible only while the bar's gradient stays aligned with the page's, which is what background-attachment: fixed buys - and iOS Safari does not honour it. There each box samples its own origin, so the bar paints the start of the ramp (#5ad8f7) while the strips paint whatever the document shows at that scroll offset (#9150bf at depth), and they read as mismatched borders that hold still while the diff scrolls past. Cancelling the inset removes the thing that has to stay matched, on every engine, instead of blending across it. The width has to come from align-self: stretch, not width: 100%. .content-main centres its children, so an item is fit-content unless it stretches, and a percentage resolves against the parent without the negative margins - it would leave the bar 17.6px short. Measured flush against the top menu at 1440, 768 and 390: 1358.4/1358.4, 768/768, 390/390. Note the desktop figure is not the 1440px viewport, because the rail takes 81.6px; the invariant is the top menu's box, not the window's. The height cap moves from dvh to svh. dvh tracks the dynamic viewport, which on iOS grows and shrinks as Safari's toolbar collapses and expands during a scroll: the cap breathed mid-scroll, changing the header's height and shifting the content below it, and each transition also fired resize and re-ran the sticky measurement behind it. svh is defined as the toolbar-expanded minimum and holds still for the life of the page. It is the smaller of the two, which is the safe side to be wrong on for a cap, and the plain vh line stays as the fallback for engines without the new units. This one has no Chromium instrument - there is no dynamic toolbar to simulate - so it is verified on the device, not in the harness. No control moves and no height changes: the bar grows outwards by exactly the margin it takes back, and the sticky stack still measures 153.9px at 1440, 230.7px at 768 and 227.5px at 390, the same as before this commit. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Stop the tab wrap verdict depending on which tab is selected base.html decides body.wrapped-tabs - the vertical tab stack - by flipping the tab list to flex-wrap and checking whether any tab lands on a second row. It re-runs on load, on resize and on every tab switch. Both sides of that comparison move, so in a band of widths the answer changes per tab: a phone showed the three tabs in a row in one screenshot and stacked in a video of the same page. The container moves because _tabs.scss says, in upstream's own comment, that .tabs must take the full width of the centred column "so the wrap-detector JS has the real available space to measure against" - and the two properties that would do it sit commented out directly beneath it. Inside the compact header, which centres its children, .tabs is therefore sized to its own content and the detector measures against a container that follows the thing it is judging. align-self: stretch here fixes that, scoped to this page; un-commenting the global lines would change every page in the app and this branch has no business doing that. The tab row is re-centred on the ul, which is a grid, so the visual is unchanged. The content moves because the active tab's label is bold. At 355px the Text and Extract Data states measure 332.9 and 330.5 against a 339.0 container and stay on one row, while Current screenshot - the widest label - goes over and wraps to three. Removing only the bold rule collapses all three states to 330.5 and the flipping stops, which is what identifies the weight rather than the container as the second variable. Dropping the bold on this page is the fix: the active tab already carries its own background-colour and its own text colour from _tabs.scss, so it loses the third of three signals and nothing else. Reserving room for the bold instead - a hidden bold copy behind every label - stabilises it too, but widens the row ~14px in every state, which pushes 360px from one row into the stack. Measured stable across 320, 344, 350, 355, 358, 360, 375, 390 and 412px, with the one-row layout now holding down to 350px where before it was unstable from 350 to 358. iOS put the same band at the phone's own width because its fonts are wider. Nothing here is engine-specific - the instability is arithmetic, and Chromium reproduces it once you look at the widths where the numbers cross. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Keep the Filters popover reachable and anchored on a small screen Two defects in the same placement code, both outside the viewport range the desktop frames covered. The panel is position: fixed, because #diff-header carries overflow: auto for its 50svh cap and would clip an absolutely positioned descendant. Fixed also means nothing the document does can scroll the panel: whatever hangs off the bottom edge at placement time stays unreachable. It was parked below the button at full height regardless, so in a landscape phone viewport - 844x390 - it spanned y=150.4 to 431.6 and the last three filters sat past the 390px edge. Replaced, at y=396.6..423.6, could not be clicked at all. place() now measures the room below the button and caps max-height to it, and the panel scrolls inside that cap. Room below is always the roomier side, so there is no flip to decide: the bar's own cap keeps its bottom edge inside 50svh, so below the button there is always at least half the viewport. The harness asserts that premise rather than trusting it. The cap is handed to max-height less the panel's own padding and borders, since the panel is content-box and switching it to border-box would fold the padding into min-width and narrow it by 24px. The bar is also its own scroll container, which the code did not account for - its comment claimed only a resize could move the button out from under the panel. On a short viewport the 50svh cap engages and #diff-header scrolls independently: 16px of scroll at 390x390 moved the button 16px while the fixed panel stayed at y=171, and 30px at 320x480. The popover visibly comes away from its trigger and can end up over the wrong control. It now re-places on the bar's scroll, and closes instead once the button has scrolled out of the bar altogether - reachable below 280x300, where the bar has 120px of travel. Verified in Chromium at 844x390, 740x360, 390x390, 320x480, 280x300, 390x844 and 1440x1000: the panel stays inside the viewport at every one, the last filter is clicked for real rather than asserted on by rect, and the gap between button and panel holds at 4px across the bar's whole scroll range. All seven fail on the tree without this change - twelve failures, the click timing out on the three short viewports. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Keep the Filters popover's height budget current with the visual viewport place() already budgets against the smaller of documentElement.clientHeight and visualViewport.height, because a fixed element is positioned against the layout viewport while iOS shrinks the visual one behind its chrome. But the open panel was re-placed only on window resize and on #diff-header scroll, and neither of those fires when the visual viewport alone changes - an expanding toolbar or the on-screen keyboard does exactly that. The cap then stays at the height it was computed with and the bottom filters go back behind the chrome, which is the failure the cap was added to prevent. That half of place() was recorded as unverifiable, on the grounds that Chromium has no dynamic toolbar and the two viewports are always equal here. They are equal only at page scale 1. CDP Emulation.setPageScaleFactor is pinch-zoom: it shrinks the visual viewport against a layout viewport that does not move, and fires VisualViewport resize - the same split iOS produces. So both halves are measurable, and the harness now measures them at 390x844@2, 844x390@1.5 and 390x390@1.5: * placing while the visual viewport is already the smaller one caps the panel to it - 85.6px against a layout-budget 215.6px at 844x390 - which is what shows the Math.min binds rather than sitting inert; * on the tree without this commit, shrinking the viewport under an open panel left the cap untouched and the panel overhung the visible area by 122px at both short viewports and 46.8px at 390x844. Three failures, one per viewport, and none anywhere else in the harness. The probe asserts its own instrument before its result: unless the scale really splits the two heights and the page really receives the resize event, it fails rather than passes. It also fails if no viewport in the set caps more tightly than the layout height would have, since then the visual-viewport budget is untested, and if a viewport's panel already fitted before the shrink, since then it cannot detect a stale cap. The re-placed cap lands on the same value as placing fresh at that viewport (210.4, 85.6, 65.0px), and the panel's width is unchanged by the re-placement. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Stop the sticky bars' backdrop depending on background-attachment: fixed Reported from an iPhone: on the diff page the sticky bar sweeps cyan to purple as you scroll while the page behind it holds still, and snaps when you scroll back up a little. The gutters beside it are correct throughout, so the page's own backdrop - body::after, a real position: fixed layer - is fine; it is the bars' copy of it that moves. page-surface-gradient asked for that copy with background-attachment: fixed, which makes the viewport the positioning area. iOS Safari does not honour it and anchors the layer to the document instead, so each bar shows whatever slice of the ramp its document position lands on. Chromium does honour the property, which is why every emulated check passed and only a device caught it. Derive the alignment from geometry instead, and ask for no engine feature at all. body::after is a viewport-wide, 100vh-tall box pinned to the viewport's top-left corner. Each bar keeps a constant viewport offset - .header at the top, #diff-header at --app-header-height, which the ResizeObserver already keeps current - and both are flush with the viewport's right edge at every width. So size the copy to the viewport, anchor it to that right edge, and push it up by the bar's own offset: the bar then paints body::after's own slice by construction, on any engine. Three details that are load-bearing: * 100vw, not 100%. The two bars are as wide as the viewport only on a narrow screen; at 1440 the action rail insets them by 81.6px (and widens further on hover), so a percentage would compress the whole ramp into what the rail leaves. Nothing moves their right edge, which is why that is the anchor. * background-origin: border-box. #diff-header carries 1rem of side padding and 0.25rem on top, and the default padding-box origin would offset the copy by exactly that. * longhands, with the solid-colour fallback shorthand first and no shorthand after them - a later `background:` line silently resets size, position and origin to their initial values. The Filters popover has no constant viewport offset of its own, so diff-overview.js hands it the one it just computed, in the same place it sets top and left. Measured in the harness against the page itself: shot B hides .app with visibility: hidden, which leaves body::before/body::after painting alone with layout and scroll untouched, so "seamless" is a pixel comparison rather than a judgement. Worst per-channel distance from that backdrop, over 40 samples at 1440x900, 768x1024, 390x844, 844x390 and 320x600, each at three scroll offsets and with the popover placed twice: bars popover before, attachment honoured 3 6 before, attachment denied 24 112 after, attachment honoured 3 6 after, attachment denied 3 6 "Denied" injects background-attachment: scroll !important - it takes away the one property iOS ignores. The before/after rows differ only in whether the engine grants it, and the after rows are identical to each other: the code path no longer has an opinion about it. That denial is not a full iOS emulation - WebKit anchors the layer to the document, which also produces the sweep, and nothing in Chromium reproduces that - so the device pass is still the gate on the sweep itself. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Give the diff page's screenshot tab the short label the extract page uses On a phone the three tabs render as a full-width vertical stack. base.html's wrap detector flips the page into body.wrapped-tabs when the tab row does not fit its container, and iOS font metrics put this row over the line at 390px - an iPhone's width. 01420ac7 stopped the verdict flip-flopping between tabs; it did not give the row room to fit. The extract page's tab for the same pane is called "Screenshot". It renders on one line on the same phone at the same width, which is the evidence this change rests on: the two pages then carry the identical three labels, the diff page's row asks for no more width than the extract page's, and #diff-header offers it slightly more room. Measured at 390px: diff page, this change row 275.4px offered 374.0px extract page row 280.8px offered 372.4px diff page, before row 330.6px offered 374.0px So the phone's own extract-page row settles the diff page, with no iOS font model in the argument. In Chromium the narrowest viewport that still fits on one line moves from 347px to 292px. "Screenshot" is also what translations/README.md asks for - the shortest suitable wording - and it makes the two sibling pages agree, which they should have done anyway. The catalogs are regenerated for the moved msgid reference; the empty msgstrs this exposes are filled in the commit that follows. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Keep the tab row inside the sticky bar when the bar runs out of room Reported from an iPhone held sideways: the diff page's tabs are not there. The bar's height budget is max-height: calc(50svh - the top menu) with overflow: auto, and in landscape its contents do not fit inside that - iOS gives the two datetime <select>s enough intrinsic width that From and To each take their own row, so the bar wants title + From + To + Filters + tabs where it has room for about three of them. What scrolls out of the visible box is the last child: the tab row, which is the page's navigation, with no scrollbar on a touch screen to say it is there. This one does not need the device. Chromium overflows the same bar on its own at 736x414, 667x375, 780x360, 844x330 and 844x300, and at 844x390 with From/To forced onto their own rows - measured as the tab row's rect falling outside So spend the budget on the row that can afford to give. #settings is the only child allowed to shrink and the only one that scrolls; #diff-watch-title and .tabs are flex: none and are therefore inside the bar at any viewport height. The shrink needs min-height: 0 on #settings - a flex item's automatic minimum size is its content, which is exactly what was pushing the tabs out. diff-overview.js follows the Filters button while the bar scrolls under the fixed panel, and the element that scrolls is now a descendant rather than the bar itself. Scroll events do not bubble, so that listener moves to capture; it then catches #settings and #diff-header's own last-resort scroll alike, and the close-when-the-button-leaves-the-bar test is unchanged because it already compares against #diff-header's rect. Verified: the tab row is inside the bar at all nine viewports above, and with the pre-change diff.css and diff-overview.js served back over the wire in the same run, six of the nine fail - the check is not watching an empty room. The popover drift probe in the harness was retargeted at whatever inside the bar actually scrolls, and proved red first: with the capture flag removed the panel drifts 3.0px off its button at 390x390 and 12.0px at 280x300. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Close the seam between the active tab and the diff card Reported from the device: a gap between the tabs and the content on Text and Screenshot, and none on Extract Data. It is #diff-header's own 0.25rem of bottom padding. The tab row is the bar's last child, and the active tab and #diff-ui below it are both var(--color-background) - so those 4px paint the page gradient across what is otherwise one continuous white surface, and the tab stops reading as the front edge of the card it belongs to. Extract Data looks right because its tabs are not in a bar at all: the same measurement on that page is 0. Measured at 1440x900, 768x1024, 390x844, 844x390 and 320x600, on both the Text and Screenshot tabs: 3.2-4.0px before, and -0.8 to 0.0 after, with the tab and the card confirmed to be the same computed colour in every frame. The check fails only on a positive gap: a sub-pixel overlap of white on white has nothing to show. The top padding stays - nothing sits above the title. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Land at the top of a tab's pane when the switch resizes the page under it Reported from the device: the Screenshot tab starts scrolled down a bit. The recording shows it plainly - for about a second after the tap the tip's first line sits behind the sticky bar, then Safari settles it. Order of events. The browser performs the fragment jump to #screenshot first. Only then does this file's own hashchange handler hide #settings, and the outgoing #text pane stop being :target - which takes the document from ~8000px to a fraction of a screen. The engine is left holding a scroll offset for a page that no longer exists, and clamps it to the new maximum rather than re-running the jump it has already done. On iOS that maximum is never zero: styles.scss floors the shell at min-height: 100vh, and 100vh there is the toolbar-collapsed viewport, so with the toolbar showing the document outruns the visual viewport by the toolbar's own height and the stale offset has somewhere to survive. So re-run the jump once the layout has stopped moving. scrollIntoView rather than scrollTo(0, 0): .tab-pane-inner already declares the offset the sticky stack needs as scroll-margin-top, and this asks for the same alignment the browser was asked for, against the layout that actually resulted. setTimeout(0) rather than requestAnimationFrame, because that scroll-margin-top is written in terms of --diff-header-height and diff-render.js's ResizeObserver updates it during the rendering update, after animation frame callbacks have run - and hiding #settings is precisely what changes it. Honest limit: Chromium is not the engine that gets this wrong. With the document floored above the viewport it lands 1px off before this change and 0 after, at 390x844 and 390x500 - the right direction, but a pixel is not the line the phone showed. The mechanism above is read off the recording and the code, not reproduced here, and the device is what settles it. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Drop the sticky design on a short viewport rather than compensate for it A phone held sideways is the one place the sticky bar cannot pay for itself. 50svh of a 390px-tall viewport is 195px, the top menu takes 55 of that, and what is left has to hold From, To, Filters and the tab row - three of those on their own lines, because iOS gives the two datetime selects enough width that From and To cannot share one. The previous two commits kept the tab row inside the bar by shrinking #settings into a scroller, then took the title and some padding back to widen the margin. Both were right about the budget and neither made the bar usable: the reporter's answer was still "it's only showing the From line and I have to scroll", and that is reproducible here - at 780x360 and 844x330 the To selector, the Filters button and both version arrows are clipped out of So below 500px of viewport height, give up the budget instead of spending it. Both bars become ordinary static blocks: no height cap, no inner scroller, title and every control present at once, and the page scrolls as one document. Neither bar keeps its pre-composited copy of the page gradient, which is not optional - that copy is aligned by pushing it up by the bar's own *constant* viewport offset, and a bar that scrolls has no constant offset, so it would slide out of phase with body::after's fixed layer. With nothing scrolling underneath, the real fixed backdrop shows through and is aligned by construction. The trade, plainly: in landscape and in a short desktop window the controls and the tab row scroll away with the diff, so switching tabs means scrolling back up. That is the right side to be wrong on when the alternative is controls you cannot see. Portrait and the desktop are untouched and still sticky - asserted, not assumed. Three things this turned up that the plan did not anticipate, each of which the change itself breaks and so belongs here: * The minimap is sticky inside the diff card at an offset equal to the sticky stack's height. With that stack gone it would pin itself 150-227px down the viewport under nothing - past halfway on a 390px-tall one. Pinned to the viewport instead, which is now free. * The Filters panel is position: fixed and followed its button only through a capture-phase listener on the bar, because the bar held still and #settings moved inside it. Static mode is the mirror image - nothing moves inside the bar and the bar itself moves with the document - so the panel drifted by the full scroll distance: 200px at 844x390, 926x428 and 1440x480. The same follow/close rule now runs from the window's scroll as well, and carries both closing tests; each is inert in the mode it was not written for. * place() had no flip-above branch, and compact_verify.py carried the reason as an assertion: the bar's own 50svh cap kept its bottom edge inside the top half of the viewport, so below the button was the roomier side by construction. Removing the cap removes the premise, and that assertion is what caught it - at 280x300, 147.6px above the button against 104.2px below. place() picks the roomier side again. The load-path scroll fix is the fourth, and it is the one worth reading twice. upstream's tabs.js rewrites an empty hash to the first tab's, and the browser performs that jump the moment the diff is built - before diff-render.js has measured the bar, so scroll-margin-top is still resolving against its --diff-header-height: 0 fallback. Traced at 844x390: the jump lands at scrollY 231 against a 16px margin at t=132ms, and the margin becomes its real 202px at t=167ms with the offset left where it was. Under a sticky bar that is invisible, because the chrome is pinned whatever the offset is; with the bar in flow the same 231px put every control above the top of the page and a landscape phone opened on a bare diff. So the alignment re-runs when the quantity it was wrong about settles - the bar's own size - guarded on the reader still being where the last alignment left them, so that rotating the phone mid-read cannot throw someone back to the top of the pane. Portrait benefits too: the page used to open 224px into the text diff and now opens at 1. .tab-pane-inner's scroll-margin-top is deliberately NOT dropped to 1rem here. It reads as an allowance for pinned chrome and therefore like dead weight once nothing is pinned, but the same number is right in both modes for two different reasons: sticky, it clears chrome painted over the pane; static, it keeps chrome that sits above the pane on screen. The comment says so, next to the measurement. 20d75707, which hid the watch title in landscape to buy budget room, is rebased out rather than left in and undone - there is no budget to buy any more, and the question of whether hiding it was acceptable goes away with it. Verified in four browser runs that vary the two changed files independently, because the popover check needs the new stylesheet with the old script: serving both old puts the page back in sticky mode, where it passes for the wrong reason. Every run reads the served diff.css and diff-overview.js back over the wire, hashes them, and fails if they are not the pair it meant to serve. On the branch: PASS, no failures. Reachability went red at 10 of 14 short viewports on the old pair, the gradient copy at 7, the minimap at 7, the load landing at 5, and the popover drift at 3 on the new-CSS/old-JS pair. The full branch harness and the previous round's probe both re-run green. One instrument repair, and it had been green while measuring nothing. The popover drift check scrolls whatever inside the bar can move and re-reads the gap - but page.click() scrolls the toggle into view to click it, which leaves that scroller already parked at its maximum, so "scroll to the maximum" was a no-op and the check compared two identical readings. It now drives the scroller to whichever end it is not at, and proves the point by comparing the two readings rather than by testing for non-zero. Re-proved red on the real code: with the capture flag removed the panel drifts 45px off its button. Nothing inside the bar overflows in this engine at any viewport any more - that was always iOS's wider metrics - so the condition is forced with an explicit max-height rather than hunted for. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Give the tab-to-card seam a whole pixel of overlap instead of none The thin blue line under the active tab is the page gradient, not a border - nothing at that boundary has one, the active tab and #diff-ui are both var(--color-background). It can only be a device-pixel row where neither white paints, and whether such a row exists comes down to a single number: the bar's bottom edge minus the card's top edge. Measured across 14 viewports at device pixel ratios 1, 2 and 3, on both the Text and Screenshot tabs, that number is 0.813px, 0.203px or exactly 0.000 - 84 readings, not one of them a whole pixel, and 0.000 at 390x844 on the Screenshot tab, which is the exact case reported. At zero the outcome is not a decision the stylesheet makes; it belongs to whatever the engine does with two edges on the same coordinate, which is why a DPR 3 phone can show a line a DPR 1 desktop does not. Chromium rounds the same way on both sides and paints no sliver at DPR 3 either - walking the bar's height in thirds of a pixel did not produce one. So this is the margin being fixed, not a pixel being chased: after the change the worst of the same 84 readings is exactly 1.000px. The alternative - a white skirt under the active tab, box-shadow: 0 1px 0 - would have been more precisely scoped and does not work. The shadow hangs below the bar's last child and #diff-header's overflow: auto, which is what enforces its height cap, clips it. Measured with the card held 3px away so the skirt and the card are separable: 0 device rows of the 3 it needs at 844x390 and 926x428, and 0 to 2 in the sticky modes. The negative margin is what survives the clip, because it moves the card rather than painting past an edge. Scoped to body.difference-page, which matters more than it looks. diff.css is not diff-page-only: preview.py and processors/extract.py serve it too, both pages have a #diff-ui of their own, and neither has a bar above it to absorb the pixel. Asserted rather than reasoned - the preview page's computed margin-top is 0px with this stylesheet and with the previous one, while the diff page's moves to -1px. The comment on the activity-strip rule above claimed diff.css was loaded only by the diff page; that was wrong and is corrected, since the scope it describes is what actually does the work. What the overlap costs, stated rather than left to be found: where the bar is sticky, nothing. A device-pixel diff of the seam band at 390x844 and DPR 3 comes back with zero differing pixels, because the bar is opaque and paints over the card. Below the short-viewport breakpoint the bar has no background of its own, so the card's white also reaches the bottom pixel of the inactive tabs and the 5px gaps between them - 4374 device pixels at 844x390, gradient turning white, and side-by-side crops of the two are indistinguishable. The tab row ends up sitting flush on the card instead of floating a pixel above it, which is how tabs normally meet their panel and is the same continuity the active tab was asked for. Full branch harness, the previous round's probe and this round's all re-run green against the change. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Reuse upstream's Previous and Next for the version arrows' labels 297bd365 gave the version arrows `title` and `aria-label` of their own, "Previous version" and "Next version". Those are two msgids this branch introduced and no catalog has ever carried, so on a German or Japanese page the arrows' accessible name is English. The upstream msgids `Previous` and `Next` say the same thing in this context - the arrows live inside the version bar, between the From and To selectors - and are non-empty in 15 of 17 catalogs; the two empty ones are en_GB and en_US, where the English msgid is the correct rendering anyway. The visible span on each arrow already uses these msgids, so the accessible name and the (header-hidden) label now agree instead of differing by a word. The cost is that the accessible name drops "version": a screen reader says "Previous" rather than "Previous version". On a control that sits between two datetime selectors that is enough, and it is the price of not asking fifteen translators for a string they can already spell. The catalogs are regenerated: both msgids disappear from the .pot and from all 17 .po files. No .mo file changes, because neither msgid had a msgstr anywhere to compile. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Correct the French translation of "Previous" fr renders the msgid `Previous` as "Aperçu", which is French for *preview*. That is a wrong word, not a style choice: the same catalog gives "Aperçu" as the msgstr for `Preview` and for `Activate preview`, and it evidently leaked from there. The msgid renders in exactly two places, both of them prev/next pairs - this page's version arrows and preview.html's own pager - and fr already translates `Next` as "Suivant", so "Précédent"/"Suivant" is the pair the catalog is otherwise using. On the preview page the current string produces an "Aperçu" button on the page called Aperçu. Left alone deliberately: de renders `Previous` as "Zurück" (*back*). That is defensible on a back arrow and is a translator's call, not an error. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Put the watch's name on the top line and drop the diff page's second title row The diff page printed the same thing twice. The menu's top line has always shown the watch URL, and 297bd365 added a sticky <h1> under it showing watch.label - which for any watch without a custom title *is* that URL. Two rows of chrome, 30px of the sticky budget, and on the common case the identical string on both. dgtlmoon's read (#4412): put it all on the top line, use the title if it is set, and take back the left inset while we are there. That is what this does, and it removes code rather than adding any. The top line's <a> is unchanged as a link - same href, same fade-out mask, same flex-shrink that keeps a long value off the menu icons. Only its text changes: current_diff_label (watch.label - title, then the fetched page_title, then the URL) with the URL as the fallback, so an untitled watch reads exactly as it does today. The five views that already pass current_diff_url pass the label beside it. The hover text now carries both, label then URL on its own line, and only when they differ. A title covers the URL that used to be printed here, and the URL is also where the link goes - so it has to stay readable without following it. That also keeps the long-title-on-long-press property the <h1> had. The heart hides under $desktop-wide-breakpoint on this page only. That is the width where the hamburger appears and the row is at its tightest, so it is where the name needs the space; on a desktop diff page and on every other page the heart is untouched. It is a media query rather than a {% if %} because the heart's reason to exist is that it is always there - folding it away with the rest of the narrow row is a different thing from taking it off the page. Left-justification: the line's text started $common-gap twice in - the menu row's own padding plus the link's own left margin. On the row's first item the second one only repeats the first, so it goes and the row's padding stays. 8.8px back, and the line still shares a left edge with the diff below it rather than sitting hard against the viewport. The right side of both is untouched. No new or removed translatable strings, so no catalog churn. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Fill the empty msgstr cells this page's own strings left behind 30 cells across 15 catalogs. No new msgids: extract_messages produces a byte-identical .pot (bar POT-Creation-Date) and update_catalog leaves every .po unchanged, which is the control that nothing here is a source change wearing a translation's clothes. Filters - empty in all 15 non-English catalogs since the diff page's options popover was added. Each value is the plural form that catalog already uses for "Global Filters" and "Filters & Triggers", not a fresh translation: Filtry, Filter, Filtros, Filtres, Filter, Filtri, フィルタ, 필터, Filtry, Filtros, Фильтры, Filtreler, Фільтри, 过滤器, 過濾器. Screenshot - this is a regression the diff-header work introduced and owes. The tab used to read "Current screenshot", which every catalog translates; shortening it to "Screenshot" moved the tab onto a msgid that nine of them (cs, de, es, fr, it, pt_BR, tr, uk, zh) had never filled, so those users got an English tab where they had a translated one. Each value is that catalog's own "Current screenshot" with the qualifier dropped - Snímek obrazovky, Captura de pantalla, Capture d'écran, Ekran görüntüsü, Скріншот, 截图 - and for de, it and pt_BR the word those catalogs already use is literally "Screenshot" (Aktueller Screenshot / Screenshot corrente / Screenshot atual), which is what dennis W302 flags and CI excludes. Italian's five pre-existing gaps on this page, deliberately in scope: the guide's multi-language rule is to fix every affected catalog in one session rather than fix one language and move on, and an Italian diff page was showing untranslated From, To, Ignore Whitespace, Keyboard:, Extract Data and Jump to next difference next to strings that were translated. Ignora spazi and Estrai dati come from the same catalog's "Ignore whitespace" and "Extract text" / "Extract as CSV"; successiva from its "Next" -> "Successivo" and differenza from its "Pixel Difference Sensitivity". Compiled with setup.py compile_catalog. en_GB and en_US recompile byte-identical, so the 15 changed .mo files are exactly the 15 changed .po files. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Let the watch title be dragged when it is too long to fit Asked for on the PR: "does the title at the top accept overflow scroll hidden so it can be scrolled with your finger/touch/drag?" It could not - .current-diff-url was overflow: hidden with a fixed right-edge fade, so the end of a long title was reachable only by hovering for the title attribute, which a phone does not have. It is now a horizontal scroll container with its scrollbar hidden in both engines. A tap still follows the link; browsers distinguish a pan from a click, so the anchor keeps its navigation. overflow-y stays hidden because the content is one non-wrapping line and an axis that can move by a rounding error lets a diagonal swipe jog the text. The fade has to follow the scroll, or the ending someone just dragged over to read is the part that is dimmed. static/js/scrollable-title.js maintains fade-left / fade-right against scrollLeft and adds js-fades to say it is doing so; under that class the CSS drops the static mask and paints only the edge that is actually cut off, or a two-ended gradient when both are, or nothing at all when the title fits. The static right-hand fade stays as the no-JS default, which is correct there because without a handler the line can only ever be cut on the right. In base.html rather than diff-overview.js: restock_diff/difference.py and image_ssim_diff/preview.py both render this line and neither loads that file. The script guards on the element existing, so it is inert on every other page. Accepted tradeoff: with the anchor focused, ArrowLeft/Right still runs version navigation on diff pages via the existing window keydown handler rather than scrolling the line. Keyboard users keep the hover title attribute, as today. styles.css is spliced, not rebuilt. parts/_top_menu.scss compiles into styles.css, whose committed bytes do not match a fresh compile of its own scss - an older sass folded rgba(255,255,255,X) to hsla(0,0%,100%,X) and hoisted nested declarations in a different order (#heartpath, .button-tag, .tab-pane-inner, .watch-table img.favicon, #browser_steps li and more). That drift is upstream's and predates this branch. So rather than commit a full rebuild and carry all of it, the scss was compiled twice - at the parent commit and with this change - and only the span that differs between those two compiles was transplanted into the committed file. The drift is identical in both compiles and cancels. The result differs from the parent by 795 bytes, all inside the .current-diff-url rules, and is byte-identical everywhere else; diff.css is untouched. A reviewer who runs npm run build will see a dirty styles.css - that mismatch is the pre-existing drift, not this commit. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Cut diff-overview.js back from a slab to a file Raised on the PR as "a slab of javascript... why cant this be a few lines of jquery". Measured before arguing: upstream's file is 190 lines (144 code, 20 comment) and this branch had grown it to 476 (284 code, 142 comment). Half the growth was comment, written at review density rather than this project's, and that is what makes it read huge. 476 -> 370 (256 code, 72 comment). Every long forensic comment is cut to the constraint it was protecting - the measurement that motivated it survives as a clause, the war story does not. Nothing load-bearing was deleted: the capture-phase scroll listener still says why capture, the visual-viewport listener still says why it exists separately from resize, and place() still says why offsetHeight is read after the cap. Converted to jQuery idiom where that is genuinely shorter, so the file reads in one voice: element lookup, class toggling, open/close, and the event bindings that need no listener options. The geometry stays native, because there is no jQuery shorthand for it - getBoundingClientRect, getComputedStyle, offsetHeight, visualViewport and ResizeObserver are the added code, and popover placement is what they are for. Two registrations deliberately stay on addEventListener and say so in place: the bar's scroll listener needs capture (scroll does not bubble and the element that scrolls is a descendant) and both scroll listeners need passive, neither of which jQuery's .on() can pass. The one structural change is that the four copies of "step both selects one position and rebuild the query string" collapse into diffStepHref(direction), called by the buttons at setup and by the arrow keys at press time. The guards are unchanged: the buttons are only touched when both selects have a selection, the arrow handler still requires both a live step and a button href, and it still returns early while an input, textarea or select has focus. Not done here, and offered instead as a follow-up: moving the popover to the native Popover API. It would delete the dismissal wiring, but placement stays manual without CSS anchor positioning, which is not yet safe cross-browser - and it would churn code that is tested and green. No new translatable strings, so the catalogs are untouched. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Point the top-menu comment at the rule it describes The previous commit inserted the fade variable and mixin between this block and .current-diff-url, which left the comment describing the declaration above it rather than the rule it was written for. It also still said the line "trims" its text, which is now only half true - it scrolls, and the trim is what the fade stands in for. Comment only: sass emits byte-identical css, so styles.css is unchanged. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> --------- Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> Co-authored-by: Surveyor <surveyor@agents.matrixsi.com> Co-authored-by: Engineer <engineer@agents.matrixsi.com> Co-authored-by: Architect <architect@agents.matrixsi.com> Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>	2026-09-16 14:19:00 +02:00
dgtlmoonandClaude Opus 5	7102544310	xPath filters - Stop LC_COLLATE deciding what contains() means (#4437) (#4438) ... Every watch whose xPath filter used contains() started reporting "Warning, no filters were found" on pages whose HTML plainly contained the target. 18 unrelated watches broke in the same hour, browser and plain-requests fetchers alike, and the saved snapshot was perfect every time. elementpath implements the XPath string functions on top of locale.strxfrm: def contains(self, a, b): return self.strxfrm(b) in self.strxfrm(a) Under LC_COLLATE=C, strxfrm() is the identity function and that is an ordinary substring test. Under a real locale it returns a binary collation key, and a substring of a collation key is not the collation key of the substring - so contains(), starts-with(), ends-with() and substring-before/after() return false for EVERY input. Reduced to one line, no document needed: LC_COLLATE=C contains("xx month xx", "month") -> True LC_COLLATE=en_US.UTF-8 contains("xx month xx", "month") -> False Name tests, axes and '=' are untouched, which is exactly why it read as "did the page change layout?" - //div kept working while //div[contains(.,"month")] returned nothing. No code change caused this. Generating the image's locales (#4429) made ENV LC_ALL=en_US.UTF-8 satisfiable for the first time; flask_app's setlocale(LC_ALL, ...) had been raising locale.Error and leaving us in C, and once it succeeded it took LC_COLLATE with it. That is why the bug survived a bisect to 0.60.2 and why reinstalling the exact pip set from a working install did not shift it - it could only be found by diffing the two containers. Same image base, same Python 3.11.16, lxml 6.1.3, libxml2 2.14.6, elementpath 5.1.1, same HTML: good-old 0.60.4 setlocale FAILED: unsupported locale setting 67 matches bad-new 0.60.5 setlocale en_US.UTF-8 0 matches Fixed in both places, because either alone leaves a hole: - flask_app sets LC_CTYPE/LC_NUMERIC/LC_MONETARY/LC_TIME individually instead of LC_ALL. This block exists to make prices render correctly and it still does - 1234567 is still "1,234,567" - it just no longer touches collation. - html_tools.xpath_filter() pins the Unicode codepoint collation per evaluation, so a filter means the same thing whatever an operator puts in LANG/LC_ALL, and does not depend on a distant module's locale bookkeeping. forms.py's XPath validation pins it too, so validation cannot accept an expression that then behaves differently at check time. Per XPath 3.1 the default collation is codepoint and must not consult LC_COLLATE, so the underlying behaviour is arguably an elementpath bug; the pin above holds regardless. Verified end to end inside the failing container: LC_COLLATE=C, LC_NUMERIC=en_US.UTF-8, thousands separators intact, and the reported filter back from 0 to 1190797 chars of output. Tested: new unit test covers contains/starts-with/ends-with under a UTF-8 collation and was checked to fail without the fix; 325 unit tests pass. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-14 20:33:35 +02:00
dgtlmoonandClaude Opus 5	718295f30b	UI - Search modal - keep the search scoped to the tag you were viewing (#4425) ... The modal submitted the active tag as `tags=<uuid>`, but the watchlist filters on `tag` - nothing reads `tags`. So searching from inside a tag view silently searched every watch, despite the label promising "URL or Title in '<tag>'". The test pulls the hidden field straight out of the rendered modal and feeds it back to the watchlist, so the field name can't drift from the arg again. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-12 17:48:48 +02:00
3 people	cd66f9ed54	Search modal native submit (#4427) ... * UI - Search - Fix search modal navigating to the host root on sub-path deployments base_path was referenced by search-modal.js but never defined, so searching always jumped to the host root instead of the X-Forwarded-Prefix sub-path. * UI - Search modal - submit natively instead of rebuilding the URL in JS Alternative to defining a `base_path` JS global: give the search form a server-rendered `action`, so url_for() supplies the reverse-proxy sub-path the same way every other link on the page already does, and let the browser submit. Drops the submit handler and the Enter handler from search-modal.js - Enter in the input reaches the footer's submit button via implicit submission, which also runs the `required` validation the synthetic `new Event('submit')` skipped. The hidden tag field is only rendered when a tag is active, so a plain search no longer carries an empty value. Also drops the nginx-job grep for the rendered markup - test_search.py already covers the sub-path case, and asserting on an exact HTML attribute string from a shell grep breaks on any unrelated edit to that tag. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: ponstream24 <87808547+ponstream24@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-12 17:09:10 +02:00
dgtlmoonandClaude Opus 5	8d938b5966	Browser fetchers - Follow re-navigation, and one content-ready deadline (re-targets #4421 + #4422 at master) (#4426) ... * Browser fetchers - Judge a fetch on the document we end up extracting, not the first navigation The goal is to compare the text of the page the browser lands on, even when the site navigates again after the first response. Both fetchers were bound to the first navigation, which shows up as two different bugs: 1. pyppeteer hangs until the hard processing timeout. Its navigation watcher is bound to the loaderId of the navigation it started, so when the site replaces that document the 'load' it waits for never arrives for that loaderId. With timeout=0 and setDefaultNavigationTimeout(0) there is nothing to break the wait, so goto() blocks until PUPPETEER_MAX_PROCESSING_TIMEOUT_SECONDS (180s) kills the fetch and the watch records an empty xpath_data - while the browser is sitting on a fully loaded page. Traced on slated.com: 0.24s goto start 0.74s main frame networkIdle loaderId=A0E0E0B2 <- never gets 'load' 2.72s main frame init loaderId=56D2B5EF <- re-navigated to get.slated.com 3.49s main frame load loaderId=56D2B5EF <- fires for the new document 25.2s goto still hanging, frame._loaderId is now 56D2B5EF Now the navigation races goto() against the main frame firing 'load', bounded by BROWSER_NAVIGATION_TIMEOUT_SECONDS (default 30), and falls back to the document we can see. slated.com / getastra.com / addupsolutions.com went from a 180s timeout with no content to 200 with full content in 5-35s. 2. Both fetchers reported the status of the interstitial. A site that gates unseen visitors with an error status plus a client-side redirect (reported against fotokoch.de: 503 + meta refresh, then a 200 with the real page) failed the watch even though the content was present, and the only workaround was ignore_status_codes, which also hides genuine 404s and 500s forever. The fetchers now keep the latest main-frame document response and judge on that - the refresh lands during the existing extra_wait, so the 200 wins. Playwright also waits for a settled load state before extracting, which is what produced "Execution context was destroyed, most likely because of a navigation" when the refresh collided with extraction. The navigation-response tracker is installed once per page and shared between the fetcher and action_goto_url() rather than each navigation adding its own listener - 'response' fires once per HTTP response, hundreds of times on a heavy page, so the callbacks are worth not duplicating. Verified one listener remains after an install plus four navigations. Selenium is unaffected either way - it hardcodes status_code = 200 because WebDriver cannot see the HTTP status. Tested: new test_renavigation.py covers the interstitial case end to end and was checked to fail without the fix and pass with it, on both fetchers. The test endpoint gates on last-seen time rather than a hit count, because a counter lets the second check see a clean 200 and the test then passes without the fix. Full browser suite 11 passed on playwright and on pyppeteer, 494 unit tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Puppeteer fetcher - One content-ready deadline instead of a stopLoading watchdog per frame event Page.stopLoading is what stops a page that would otherwise load forever waiting on a subresource that never answers, so that we can still screenshot and scrape what rendered. That intent was right, but it was implemented as a fire-and-forget task armed by every frame event, which measured on a single fetch of an iframe-heavy page came to: 14 watchdog tasks spawned 11 page-wide Page.stopLoading calls 3 tasks outliving the fetch and firing against a closed page Page.stopLoading takes no frame or loader argument - it is the Stop button, and it stops the whole page. Verified directly: one call stopped a pending main frame and a pending iframe in the same instant. So the other 10 calls were redundant, and because they landed at arbitrary later times they could stop a *subsequent* navigation we actually wanted - which is the likeliest reason the same URL fetched in 8s on one run and 35s on the next. Replaced with a single deadline, awaited inline so nothing can outlive the fetch (there is no create_task left in this file at all): navigate (bounded) -> wait the configured delay -> Page.stopLoading -> extract The delay is measured from when navigation finished, not from when it started. Anchoring it to the start would quietly rob a slow-loading page of its settle time, and letting JS-rendered content appear after load is the whole point of the setting. Verified with a server that takes 5s to answer and renders via JS 2s after load: total 9.6s for a 4s delay, and the late content is captured. Because a page is never reliably "finished" - many sites navigate as part of their normal design - the delay restarts when the MAIN frame replaces its document, so a redirect or interstitial gets the same settle time the first document got. Iframes do not restart it, and it is capped by BROWSER_CONTENT_READY_MAX_RESETS (default 2). Only the existing "wait n seconds before extracting text" stays user-facing; BROWSER_NAVIGATION_TIMEOUT_SECONDS is a safety net with a sane default rather than a second knob for users to reason about. This matches what other scrapers do: bound the navigation, do not fail when it times out, settle, then extract. Timings are also more predictable now - the four reported URLs went from 5-35s of variance to 4.6-7.2s at a 3s delay, all with full content and a 200. Tested: 11 passed pyppeteer browser suite, 7 passed playwright, 494 unit + llm. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-12 17:08:28 +02:00
Jeff Hedlund	2f60135cd0	feature: Implemented optional override link (#4290) ... * Implemented optional link to override feature * Added translation strings (untranslated)	2026-09-09 16:06:24 +02:00
dgtlmoon	7f3645f4a8	API - DELETE for Watch history #4397 (#4403)	2026-09-09 15:46:00 +02:00
3 people	455e0228ca	Tests - Fix TestHistoryPathTraversal on macOS, and cover the containment check (#4390) ... * Compare against a resolved data_dir in the history path-traversal test Watch.history resolves entries with os.path.realpath, so test_normal_snapshot_entry_is_accepted compared a resolved path against an unresolved data_dir. On macOS the datastore lives under /tmp, which is a symlink to /private/tmp, so the assertion fails for a path that is in fact inside the directory. The guard is correct; the test was not. Resolve both sides, matching what the production code does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Add a test that actually exercises the history containment check Disabling the containment check in Watch.history left every test in TestHistoryPathTraversal passing. os.path.basename() reduces both traversal fixtures ('/etc/passwd', '../../etc/passwd') to 'passwd', so neither reaches the check — they stop at the os.path.exists() test below it. A bare '..' survives basename() and resolves to the parent of data_dir, which exists, so the containment check is what rejects it. With that check disabled this new test is the only one in the class that fails. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: GG5533 <285285461+GG5533@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-09-09 15:43:54 +02:00
dgtlmoon	3a71777499	UI - Fixing lots of actions that should be a POST style action which led to a 404 ... Also solves some of GHSA-56fq-63vj-9992 Add-watch-UI should be POST/CSRF protected	2026-09-04 12:38:20 +02:00
dgtlmoon	e0fb224d41	UI - Fixing lots of actions that should be a POST style action which led to a 404 (#4370)	2026-09-04 10:56:23 +02:00
dgtlmoon	050172129e	Translations - support for translation overlays/local translations of existing strings (#4365) ... * Translations - support for translation overlays/local translations of existing strings * Dont forget translations_overlay for pip	2026-09-03 11:15:46 +02:00
Andrew PeabodyandAndrew Peabody	3d6de42924	fix(conditions): support zero values in condition filtering and json logic conversion (#4362) ... * fix(conditions): support zero values in condition filtering and json logic conversion - Fix filter_complete_rules dropping rules where value is 0/0.0 due to 0 == False in Python - Fix convert_to_jsonlogic raising EmptyConditionRuleRowNotUsable on value=0 due to truthiness check - Fix str != 'None' type comparison typo in convert_to_jsonlogic - Add comprehensive unit tests covering zero value condition filtering, conversion, and execution * chore: re-trigger CI checks --------- Co-authored-by: Andrew Peabody <apeabody@users.noreply.github.com>	2026-09-03 08:57:34 +02:00
dgtlmoonandClaude Opus 5	fbd9472218	fix(api): accept an existing tag UUID in the watch tag field, and deprecate it (#4361) ... `tag` on POST /watch was documented as taking a tag UUID, but the value went to add_tag(title): a UUID silently created a junk tag *titled* with that UUID and never applied the tag the caller asked for. `tags` (UUIDs) was the only thing that worked. - `tag=` now resolves an existing tag UUID to that tag, still falling back to title matching/creation for names. A UUID-shaped value matching nothing is skipped with a warning rather than becoming a group named after a UUID. - Blank tokens ("One,,Two,") no longer store False in watch['tags'] - add_tag() returns False for an empty title and it was appended unguarded. Consumers tolerate it (get_all_tags_for_watch() dictfilt()s over known tags) but it is not valid data. - add_tag()'s title search is extracted to tag_uuid_for_title(), so existence can be tested without creating as a side effect. add_tag()'s contract is unchanged. - api-spec: `tag` is marked `deprecated: true` (so Redoc renders the badge) and states plainly that it takes names, not UUIDs. `tags` now says what it really does - applied verbatim, never creates, unknown UUIDs stored as dangling refs. Rendered docs rebuilt. Every claim in the new field docs is asserted in test_api_tags.py against the real API. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-02 16:46:27 +02:00
dgtlmoonandClaude Opus 5	2669a5cee6	Feature/Watch limit - Adding test (#4360) ... * Env var - PAGE_WATCH_LIMIT enhancements * Rebuild API docs * Bump APi doc version * test: cover PAGE_WATCH_LIMIT across every add path - API create returns 429; API import returns 429 and refuses the batch whole - quick-add and the UI importer flash the limit (importer once per file, not per row) and hand unimported URLs back - clone at the limit no longer KeyErrors - an instance already over the limit still loads from disk and stays editable, only new watches are refused - the Info tab shows the limit only when one is set - add_watch() with no request context returns None instead of raising from flash() - an absent, empty or unparseable env var all mean unlimited Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-02 15:43:59 +02:00