507 Commits

Include renames

Author	SHA1	Message	Date
dgtlmoon	d474d0d88d	test tweaks	2026-09-17 10:33:54 +02:00
4 people	77fb923de6	fix(http): send a single Date header on werkzeug built-in server (#4372) ... * fix(http): send a single Date header on werkzeug built-in server Static resources served via werkzeug send_from_directory/send_file get a Date header injected into the WSGI response by make_conditional() (werkzeug/wrappers/response.py:752-757). When the app runs on Werkzeug's built-in server -- the default path started through socketio.run(..., allow_unsafe_werkzeug=True) in changedetectionio/__init__.py:694 and used by the docker entrypoint -- BaseHTTPRequestHandler.send_response() (werkzeug/serving.py:271) emits its own Date header line as well, so the wire response carries two Date headers. RFC 9110 forbids this and nginx rejects the response with "upstream sent duplicate header line" (issue #4299, see also #4101). Fix: a global after_request hook pops the application-side Date copy so only the server's single header reaches the wire. Verified safe on gunicorn too, which also emits its own Date header. Test: new tests/test_duplicate_date_header.py hits the live_server over real HTTP with http.client (the Flask test client talks to the WSGI app directly and never sees the server-added header) and asserts the Date header appears exactly once, on the exact static resources named in the issue. Fails on unfixed code with two identical Date lines; passes with the fix. Fixes #4299 * Apply suggestion from @dgtlmoon * Tidy the #4299 Date header fix and its test flask_app.py: the applied suggestion landed with a 3-space indent and trailing whitespace - the latter was the only W291 in the file, which .ruff.toml selects. test_duplicate_date_header.py: - drop the 10s socket wait loop, pytest-flask's live_server already blocks until the port accepts connections - drop the unused `app` fixture argument (live_server depends on it) - stop hardcoding jquery-3.6.0.min.js: asserting 200 on a vendored filename turns a jQuery bump into a failure in a file about HTTP headers. Any send_from_directory() response exercises the same path, so styles.css alone is enough. Still red before the fix (two identical Date lines) and green after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: dgtlmoon <leigh@morresi.net> Co-authored-by: dgtlmoon <dgtlmoon@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 07:16:00 +02:00
5 people	a5f85da4af	UI - Keep history diff title and controls visible while scrolling (#4412) ... * Keep history diff title and controls visible while scrolling The watch label, version selectors, diff options and tabs move into a sticky #diff-header, and the top menu above it (watch URL, EDIT, theme and GitHub links) becomes sticky too, so the whole header stays on screen while the diff scrolls. The minimap and anchor jumps offset by the measured height of both bars instead of a fixed value. Heights are measured on load and kept current with a ResizeObserver where it is available, falling back to the window resize and hashchange events as the restock graph and queue sparkline already do, plus one deferred measure at load - the tab switch hides #settings and so resizes the header without a window resize, and opening a link already at #screenshot does the same with no hashchange to follow. The app header is stored before the diff header is measured, since the latter's cap is derived from the former. A sticky bar must be opaque to the content scrolling beneath it, but the page's own backdrop is a fixed full-viewport gradient layer over the page colour, so a flat --color-background panel reads as a white band cutting across it. Instead each bar repaints that same backdrop via a shared page-surface-gradient mixin: the three gradient stops pre-composited against --color-background-page with color-mix() at the layer's 0.91 opacity, with background-attachment: fixed so they stay registered with body::after. Children of #diff-header are centred with a flex column, mirroring the align-items: center that section.content applies to the non-sticky siblings, so the controls panel and tabs keep their intrinsic width and standard colours. The top menu is made sticky only under body.difference-page. Upstream already wants this globally (see the @todo in parts/_top_menu.scss) but held off because the bar has no background of its own; scoping it here keeps that decision separate. Sticky makes .header a stacking context, so its mobile drawer is capped at the bar's z-index of 30 - high enough to cover #diff-header, low enough that the action rail's hover flyout and toast notifications still paint over the bar as before. The activity strip is fixed on body in the root stacking context, so it would have painted through the capped drawer; on this page it drops just below the bar rather than raising the bar past the rail. Dropping the 40px section.content padding and tightening the title's own padding closes the gap between the two bars, and #diff-header's half-viewport cap now excludes the top menu so the sticky stack stays within the same budget on short viewports. .app-main's 0.55rem gap goes too on this page: it sits between the two sticky bars, so the diff header would otherwise start 8.8px below the top menu and slide up to meet it over the first 8.8px of scroll. That gap cannot survive sticking - it is outside both bars, so the diff would scroll through it - and the bars are flush at every scroll position instead. Co-authored-by: Engineer <engineer@agents.matrixsi.com> Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Condense the history diff header into a compact toolbar The sticky bar the previous commit introduced takes 297px of a 1000px desktop viewport and 422px of a 390x844 phone - at that width its own 50dvh cap is already clipping it, so the shipped page scrolls a header inside itself before the diff has moved at all. Compact it to three rows and that becomes 154px and 228px, without hiding any control. The watch title takes the full width of the bar on its own row above the controls, centred, 0.9rem, one line with an ellipsis. It truncates only when it is wider than the whole viewport - a 184-character title shows in full at 1440px and ellipsizes at 768 and 390 - and the bar keeps exactly the same height either way, which a two-line clamp would not. The full text goes in the title attribute so hover reveals what the ellipsis hides, the same bargain the watch URL above it already makes. The From/To labels shrink to chips rather than hiding. display: none takes a <label> out of the accessibility tree: verified through Chromium's accessibility tree, the selects report "From" and "To" with the chips and "" without them. Sighted users need them too, since two identically formatted datetime selects side by side have only their order to tell them apart, and on a phone they stack so even that stops helping. The chips cost horizontal space and no height at all. Dropping the shipped width: 4rem sizes them to their text, and "From" is wider than "To", so a min-width floor keeps the stacked selects flush. The seven diff options move behind a Filters button. The fieldset stays inline, inside the form, until diff-overview.js swaps it for the popover, so with scripting off the options are exactly as reachable as they are today. The panel is position: fixed, not absolute: #diff-header carries overflow: auto to enforce its 50dvh cap, which clips an absolutely positioned descendant at the bar's bottom edge. The bar is sticky with no transform, filter or contain, so it is not a containing block for fixed and the panel escapes the clip; the script parks it under the button and keeps it inside the viewport, with aria-expanded, Escape, outside-click and a reposition on resize. The keyboard-nav prompt and the two button words are hidden, leaving the arrows; both links carry the full wording as aria-label and title so the accessible name survives the CSS. The tabs' vertical padding halves on this page, 11px off every scroll position, leaving their horizontal padding and so their widths alone. Narrow viewports need one more thing, and font-size alone cannot give it: once the chip takes inline width the select clips its own displayed value, and two snapshots from the same day differ only by the meridiem, so they read identically while closed. #settings, #diff-form, .diff-fieldset and the span are all sized to max-content, which is the select's text, so the container shrinks by exactly as much as the text does - headroom measured 0.0px at every size from 0.9rem to 0.7rem. The chain is pinned to the row first and only then does 0.75rem buy room: +19.3px on the demo pair and +6.3px against the longest realistic English value, which still clips at 0.8rem. All of it hangs off the one 700px breakpoint, and every selector in it carries an ID because the rules it overrides are written with one and a class-only override would lose silently. Co-authored-by: Architect <architect@agents.matrixsi.com> Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Rebuild translation catalog for the new diff header strings The compact header adds three msgids - Filters, Previous version and Next version - and the lint-translations job re-runs extract_messages, update_catalog and compile_catalog and fails on any resulting diff, so the catalog has to travel with the strings that created it. It failed on 297bd365 and took the four test-application jobs down with it, since they depend on it. Purely mechanical: the extract adds exactly those three entries to messages.pot and to each of the seventeen catalogs, all with an empty msgstr, and changes nothing else. The .mo files are unchanged, because an empty msgstr is not written to the compiled catalog. POT-Creation-Date is deliberately left at its committed value rather than restamped. The job filters that line out of the diff it checks, so restamping buys nothing, and 0.60.4 rewrites the same header lines - a restamp here would conflict with the release commit over a timestamp. Kept out of the header commit so the change being reviewed stays the template, the stylesheet and the script. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Make the sticky diff header full-bleed and pin its cap to svh Two iOS-only faults in the sticky bar, both reported from a real iPhone and neither reproducible in Chromium, emulated or not. Both are properties of 43b07d5b rather than of the compact toolbar on top of it. section.content insets its children by $common-gap so they clear the left rail, while the top menu above stays flush against it, so the bar left an 8.8px strip of real page background down each side. That strip is invisible only while the bar's gradient stays aligned with the page's, which is what background-attachment: fixed buys - and iOS Safari does not honour it. There each box samples its own origin, so the bar paints the start of the ramp (#5ad8f7) while the strips paint whatever the document shows at that scroll offset (#9150bf at depth), and they read as mismatched borders that hold still while the diff scrolls past. Cancelling the inset removes the thing that has to stay matched, on every engine, instead of blending across it. The width has to come from align-self: stretch, not width: 100%. .content-main centres its children, so an item is fit-content unless it stretches, and a percentage resolves against the parent without the negative margins - it would leave the bar 17.6px short. Measured flush against the top menu at 1440, 768 and 390: 1358.4/1358.4, 768/768, 390/390. Note the desktop figure is not the 1440px viewport, because the rail takes 81.6px; the invariant is the top menu's box, not the window's. The height cap moves from dvh to svh. dvh tracks the dynamic viewport, which on iOS grows and shrinks as Safari's toolbar collapses and expands during a scroll: the cap breathed mid-scroll, changing the header's height and shifting the content below it, and each transition also fired resize and re-ran the sticky measurement behind it. svh is defined as the toolbar-expanded minimum and holds still for the life of the page. It is the smaller of the two, which is the safe side to be wrong on for a cap, and the plain vh line stays as the fallback for engines without the new units. This one has no Chromium instrument - there is no dynamic toolbar to simulate - so it is verified on the device, not in the harness. No control moves and no height changes: the bar grows outwards by exactly the margin it takes back, and the sticky stack still measures 153.9px at 1440, 230.7px at 768 and 227.5px at 390, the same as before this commit. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Stop the tab wrap verdict depending on which tab is selected base.html decides body.wrapped-tabs - the vertical tab stack - by flipping the tab list to flex-wrap and checking whether any tab lands on a second row. It re-runs on load, on resize and on every tab switch. Both sides of that comparison move, so in a band of widths the answer changes per tab: a phone showed the three tabs in a row in one screenshot and stacked in a video of the same page. The container moves because _tabs.scss says, in upstream's own comment, that .tabs must take the full width of the centred column "so the wrap-detector JS has the real available space to measure against" - and the two properties that would do it sit commented out directly beneath it. Inside the compact header, which centres its children, .tabs is therefore sized to its own content and the detector measures against a container that follows the thing it is judging. align-self: stretch here fixes that, scoped to this page; un-commenting the global lines would change every page in the app and this branch has no business doing that. The tab row is re-centred on the ul, which is a grid, so the visual is unchanged. The content moves because the active tab's label is bold. At 355px the Text and Extract Data states measure 332.9 and 330.5 against a 339.0 container and stay on one row, while Current screenshot - the widest label - goes over and wraps to three. Removing only the bold rule collapses all three states to 330.5 and the flipping stops, which is what identifies the weight rather than the container as the second variable. Dropping the bold on this page is the fix: the active tab already carries its own background-colour and its own text colour from _tabs.scss, so it loses the third of three signals and nothing else. Reserving room for the bold instead - a hidden bold copy behind every label - stabilises it too, but widens the row ~14px in every state, which pushes 360px from one row into the stack. Measured stable across 320, 344, 350, 355, 358, 360, 375, 390 and 412px, with the one-row layout now holding down to 350px where before it was unstable from 350 to 358. iOS put the same band at the phone's own width because its fonts are wider. Nothing here is engine-specific - the instability is arithmetic, and Chromium reproduces it once you look at the widths where the numbers cross. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Keep the Filters popover reachable and anchored on a small screen Two defects in the same placement code, both outside the viewport range the desktop frames covered. The panel is position: fixed, because #diff-header carries overflow: auto for its 50svh cap and would clip an absolutely positioned descendant. Fixed also means nothing the document does can scroll the panel: whatever hangs off the bottom edge at placement time stays unreachable. It was parked below the button at full height regardless, so in a landscape phone viewport - 844x390 - it spanned y=150.4 to 431.6 and the last three filters sat past the 390px edge. Replaced, at y=396.6..423.6, could not be clicked at all. place() now measures the room below the button and caps max-height to it, and the panel scrolls inside that cap. Room below is always the roomier side, so there is no flip to decide: the bar's own cap keeps its bottom edge inside 50svh, so below the button there is always at least half the viewport. The harness asserts that premise rather than trusting it. The cap is handed to max-height less the panel's own padding and borders, since the panel is content-box and switching it to border-box would fold the padding into min-width and narrow it by 24px. The bar is also its own scroll container, which the code did not account for - its comment claimed only a resize could move the button out from under the panel. On a short viewport the 50svh cap engages and #diff-header scrolls independently: 16px of scroll at 390x390 moved the button 16px while the fixed panel stayed at y=171, and 30px at 320x480. The popover visibly comes away from its trigger and can end up over the wrong control. It now re-places on the bar's scroll, and closes instead once the button has scrolled out of the bar altogether - reachable below 280x300, where the bar has 120px of travel. Verified in Chromium at 844x390, 740x360, 390x390, 320x480, 280x300, 390x844 and 1440x1000: the panel stays inside the viewport at every one, the last filter is clicked for real rather than asserted on by rect, and the gap between button and panel holds at 4px across the bar's whole scroll range. All seven fail on the tree without this change - twelve failures, the click timing out on the three short viewports. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Keep the Filters popover's height budget current with the visual viewport place() already budgets against the smaller of documentElement.clientHeight and visualViewport.height, because a fixed element is positioned against the layout viewport while iOS shrinks the visual one behind its chrome. But the open panel was re-placed only on window resize and on #diff-header scroll, and neither of those fires when the visual viewport alone changes - an expanding toolbar or the on-screen keyboard does exactly that. The cap then stays at the height it was computed with and the bottom filters go back behind the chrome, which is the failure the cap was added to prevent. That half of place() was recorded as unverifiable, on the grounds that Chromium has no dynamic toolbar and the two viewports are always equal here. They are equal only at page scale 1. CDP Emulation.setPageScaleFactor is pinch-zoom: it shrinks the visual viewport against a layout viewport that does not move, and fires VisualViewport resize - the same split iOS produces. So both halves are measurable, and the harness now measures them at 390x844@2, 844x390@1.5 and 390x390@1.5: * placing while the visual viewport is already the smaller one caps the panel to it - 85.6px against a layout-budget 215.6px at 844x390 - which is what shows the Math.min binds rather than sitting inert; * on the tree without this commit, shrinking the viewport under an open panel left the cap untouched and the panel overhung the visible area by 122px at both short viewports and 46.8px at 390x844. Three failures, one per viewport, and none anywhere else in the harness. The probe asserts its own instrument before its result: unless the scale really splits the two heights and the page really receives the resize event, it fails rather than passes. It also fails if no viewport in the set caps more tightly than the layout height would have, since then the visual-viewport budget is untested, and if a viewport's panel already fitted before the shrink, since then it cannot detect a stale cap. The re-placed cap lands on the same value as placing fresh at that viewport (210.4, 85.6, 65.0px), and the panel's width is unchanged by the re-placement. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Stop the sticky bars' backdrop depending on background-attachment: fixed Reported from an iPhone: on the diff page the sticky bar sweeps cyan to purple as you scroll while the page behind it holds still, and snaps when you scroll back up a little. The gutters beside it are correct throughout, so the page's own backdrop - body::after, a real position: fixed layer - is fine; it is the bars' copy of it that moves. page-surface-gradient asked for that copy with background-attachment: fixed, which makes the viewport the positioning area. iOS Safari does not honour it and anchors the layer to the document instead, so each bar shows whatever slice of the ramp its document position lands on. Chromium does honour the property, which is why every emulated check passed and only a device caught it. Derive the alignment from geometry instead, and ask for no engine feature at all. body::after is a viewport-wide, 100vh-tall box pinned to the viewport's top-left corner. Each bar keeps a constant viewport offset - .header at the top, #diff-header at --app-header-height, which the ResizeObserver already keeps current - and both are flush with the viewport's right edge at every width. So size the copy to the viewport, anchor it to that right edge, and push it up by the bar's own offset: the bar then paints body::after's own slice by construction, on any engine. Three details that are load-bearing: * 100vw, not 100%. The two bars are as wide as the viewport only on a narrow screen; at 1440 the action rail insets them by 81.6px (and widens further on hover), so a percentage would compress the whole ramp into what the rail leaves. Nothing moves their right edge, which is why that is the anchor. * background-origin: border-box. #diff-header carries 1rem of side padding and 0.25rem on top, and the default padding-box origin would offset the copy by exactly that. * longhands, with the solid-colour fallback shorthand first and no shorthand after them - a later `background:` line silently resets size, position and origin to their initial values. The Filters popover has no constant viewport offset of its own, so diff-overview.js hands it the one it just computed, in the same place it sets top and left. Measured in the harness against the page itself: shot B hides .app with visibility: hidden, which leaves body::before/body::after painting alone with layout and scroll untouched, so "seamless" is a pixel comparison rather than a judgement. Worst per-channel distance from that backdrop, over 40 samples at 1440x900, 768x1024, 390x844, 844x390 and 320x600, each at three scroll offsets and with the popover placed twice: bars popover before, attachment honoured 3 6 before, attachment denied 24 112 after, attachment honoured 3 6 after, attachment denied 3 6 "Denied" injects background-attachment: scroll !important - it takes away the one property iOS ignores. The before/after rows differ only in whether the engine grants it, and the after rows are identical to each other: the code path no longer has an opinion about it. That denial is not a full iOS emulation - WebKit anchors the layer to the document, which also produces the sweep, and nothing in Chromium reproduces that - so the device pass is still the gate on the sweep itself. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Give the diff page's screenshot tab the short label the extract page uses On a phone the three tabs render as a full-width vertical stack. base.html's wrap detector flips the page into body.wrapped-tabs when the tab row does not fit its container, and iOS font metrics put this row over the line at 390px - an iPhone's width. 01420ac7 stopped the verdict flip-flopping between tabs; it did not give the row room to fit. The extract page's tab for the same pane is called "Screenshot". It renders on one line on the same phone at the same width, which is the evidence this change rests on: the two pages then carry the identical three labels, the diff page's row asks for no more width than the extract page's, and #diff-header offers it slightly more room. Measured at 390px: diff page, this change row 275.4px offered 374.0px extract page row 280.8px offered 372.4px diff page, before row 330.6px offered 374.0px So the phone's own extract-page row settles the diff page, with no iOS font model in the argument. In Chromium the narrowest viewport that still fits on one line moves from 347px to 292px. "Screenshot" is also what translations/README.md asks for - the shortest suitable wording - and it makes the two sibling pages agree, which they should have done anyway. The catalogs are regenerated for the moved msgid reference; the empty msgstrs this exposes are filled in the commit that follows. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Keep the tab row inside the sticky bar when the bar runs out of room Reported from an iPhone held sideways: the diff page's tabs are not there. The bar's height budget is max-height: calc(50svh - the top menu) with overflow: auto, and in landscape its contents do not fit inside that - iOS gives the two datetime <select>s enough intrinsic width that From and To each take their own row, so the bar wants title + From + To + Filters + tabs where it has room for about three of them. What scrolls out of the visible box is the last child: the tab row, which is the page's navigation, with no scrollbar on a touch screen to say it is there. This one does not need the device. Chromium overflows the same bar on its own at 736x414, 667x375, 780x360, 844x330 and 844x300, and at 844x390 with From/To forced onto their own rows - measured as the tab row's rect falling outside So spend the budget on the row that can afford to give. #settings is the only child allowed to shrink and the only one that scrolls; #diff-watch-title and .tabs are flex: none and are therefore inside the bar at any viewport height. The shrink needs min-height: 0 on #settings - a flex item's automatic minimum size is its content, which is exactly what was pushing the tabs out. diff-overview.js follows the Filters button while the bar scrolls under the fixed panel, and the element that scrolls is now a descendant rather than the bar itself. Scroll events do not bubble, so that listener moves to capture; it then catches #settings and #diff-header's own last-resort scroll alike, and the close-when-the-button-leaves-the-bar test is unchanged because it already compares against #diff-header's rect. Verified: the tab row is inside the bar at all nine viewports above, and with the pre-change diff.css and diff-overview.js served back over the wire in the same run, six of the nine fail - the check is not watching an empty room. The popover drift probe in the harness was retargeted at whatever inside the bar actually scrolls, and proved red first: with the capture flag removed the panel drifts 3.0px off its button at 390x390 and 12.0px at 280x300. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Close the seam between the active tab and the diff card Reported from the device: a gap between the tabs and the content on Text and Screenshot, and none on Extract Data. It is #diff-header's own 0.25rem of bottom padding. The tab row is the bar's last child, and the active tab and #diff-ui below it are both var(--color-background) - so those 4px paint the page gradient across what is otherwise one continuous white surface, and the tab stops reading as the front edge of the card it belongs to. Extract Data looks right because its tabs are not in a bar at all: the same measurement on that page is 0. Measured at 1440x900, 768x1024, 390x844, 844x390 and 320x600, on both the Text and Screenshot tabs: 3.2-4.0px before, and -0.8 to 0.0 after, with the tab and the card confirmed to be the same computed colour in every frame. The check fails only on a positive gap: a sub-pixel overlap of white on white has nothing to show. The top padding stays - nothing sits above the title. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Land at the top of a tab's pane when the switch resizes the page under it Reported from the device: the Screenshot tab starts scrolled down a bit. The recording shows it plainly - for about a second after the tap the tip's first line sits behind the sticky bar, then Safari settles it. Order of events. The browser performs the fragment jump to #screenshot first. Only then does this file's own hashchange handler hide #settings, and the outgoing #text pane stop being :target - which takes the document from ~8000px to a fraction of a screen. The engine is left holding a scroll offset for a page that no longer exists, and clamps it to the new maximum rather than re-running the jump it has already done. On iOS that maximum is never zero: styles.scss floors the shell at min-height: 100vh, and 100vh there is the toolbar-collapsed viewport, so with the toolbar showing the document outruns the visual viewport by the toolbar's own height and the stale offset has somewhere to survive. So re-run the jump once the layout has stopped moving. scrollIntoView rather than scrollTo(0, 0): .tab-pane-inner already declares the offset the sticky stack needs as scroll-margin-top, and this asks for the same alignment the browser was asked for, against the layout that actually resulted. setTimeout(0) rather than requestAnimationFrame, because that scroll-margin-top is written in terms of --diff-header-height and diff-render.js's ResizeObserver updates it during the rendering update, after animation frame callbacks have run - and hiding #settings is precisely what changes it. Honest limit: Chromium is not the engine that gets this wrong. With the document floored above the viewport it lands 1px off before this change and 0 after, at 390x844 and 390x500 - the right direction, but a pixel is not the line the phone showed. The mechanism above is read off the recording and the code, not reproduced here, and the device is what settles it. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Drop the sticky design on a short viewport rather than compensate for it A phone held sideways is the one place the sticky bar cannot pay for itself. 50svh of a 390px-tall viewport is 195px, the top menu takes 55 of that, and what is left has to hold From, To, Filters and the tab row - three of those on their own lines, because iOS gives the two datetime selects enough width that From and To cannot share one. The previous two commits kept the tab row inside the bar by shrinking #settings into a scroller, then took the title and some padding back to widen the margin. Both were right about the budget and neither made the bar usable: the reporter's answer was still "it's only showing the From line and I have to scroll", and that is reproducible here - at 780x360 and 844x330 the To selector, the Filters button and both version arrows are clipped out of So below 500px of viewport height, give up the budget instead of spending it. Both bars become ordinary static blocks: no height cap, no inner scroller, title and every control present at once, and the page scrolls as one document. Neither bar keeps its pre-composited copy of the page gradient, which is not optional - that copy is aligned by pushing it up by the bar's own *constant* viewport offset, and a bar that scrolls has no constant offset, so it would slide out of phase with body::after's fixed layer. With nothing scrolling underneath, the real fixed backdrop shows through and is aligned by construction. The trade, plainly: in landscape and in a short desktop window the controls and the tab row scroll away with the diff, so switching tabs means scrolling back up. That is the right side to be wrong on when the alternative is controls you cannot see. Portrait and the desktop are untouched and still sticky - asserted, not assumed. Three things this turned up that the plan did not anticipate, each of which the change itself breaks and so belongs here: * The minimap is sticky inside the diff card at an offset equal to the sticky stack's height. With that stack gone it would pin itself 150-227px down the viewport under nothing - past halfway on a 390px-tall one. Pinned to the viewport instead, which is now free. * The Filters panel is position: fixed and followed its button only through a capture-phase listener on the bar, because the bar held still and #settings moved inside it. Static mode is the mirror image - nothing moves inside the bar and the bar itself moves with the document - so the panel drifted by the full scroll distance: 200px at 844x390, 926x428 and 1440x480. The same follow/close rule now runs from the window's scroll as well, and carries both closing tests; each is inert in the mode it was not written for. * place() had no flip-above branch, and compact_verify.py carried the reason as an assertion: the bar's own 50svh cap kept its bottom edge inside the top half of the viewport, so below the button was the roomier side by construction. Removing the cap removes the premise, and that assertion is what caught it - at 280x300, 147.6px above the button against 104.2px below. place() picks the roomier side again. The load-path scroll fix is the fourth, and it is the one worth reading twice. upstream's tabs.js rewrites an empty hash to the first tab's, and the browser performs that jump the moment the diff is built - before diff-render.js has measured the bar, so scroll-margin-top is still resolving against its --diff-header-height: 0 fallback. Traced at 844x390: the jump lands at scrollY 231 against a 16px margin at t=132ms, and the margin becomes its real 202px at t=167ms with the offset left where it was. Under a sticky bar that is invisible, because the chrome is pinned whatever the offset is; with the bar in flow the same 231px put every control above the top of the page and a landscape phone opened on a bare diff. So the alignment re-runs when the quantity it was wrong about settles - the bar's own size - guarded on the reader still being where the last alignment left them, so that rotating the phone mid-read cannot throw someone back to the top of the pane. Portrait benefits too: the page used to open 224px into the text diff and now opens at 1. .tab-pane-inner's scroll-margin-top is deliberately NOT dropped to 1rem here. It reads as an allowance for pinned chrome and therefore like dead weight once nothing is pinned, but the same number is right in both modes for two different reasons: sticky, it clears chrome painted over the pane; static, it keeps chrome that sits above the pane on screen. The comment says so, next to the measurement. 20d75707, which hid the watch title in landscape to buy budget room, is rebased out rather than left in and undone - there is no budget to buy any more, and the question of whether hiding it was acceptable goes away with it. Verified in four browser runs that vary the two changed files independently, because the popover check needs the new stylesheet with the old script: serving both old puts the page back in sticky mode, where it passes for the wrong reason. Every run reads the served diff.css and diff-overview.js back over the wire, hashes them, and fails if they are not the pair it meant to serve. On the branch: PASS, no failures. Reachability went red at 10 of 14 short viewports on the old pair, the gradient copy at 7, the minimap at 7, the load landing at 5, and the popover drift at 3 on the new-CSS/old-JS pair. The full branch harness and the previous round's probe both re-run green. One instrument repair, and it had been green while measuring nothing. The popover drift check scrolls whatever inside the bar can move and re-reads the gap - but page.click() scrolls the toggle into view to click it, which leaves that scroller already parked at its maximum, so "scroll to the maximum" was a no-op and the check compared two identical readings. It now drives the scroller to whichever end it is not at, and proves the point by comparing the two readings rather than by testing for non-zero. Re-proved red on the real code: with the capture flag removed the panel drifts 45px off its button. Nothing inside the bar overflows in this engine at any viewport any more - that was always iOS's wider metrics - so the condition is forced with an explicit max-height rather than hunted for. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Give the tab-to-card seam a whole pixel of overlap instead of none The thin blue line under the active tab is the page gradient, not a border - nothing at that boundary has one, the active tab and #diff-ui are both var(--color-background). It can only be a device-pixel row where neither white paints, and whether such a row exists comes down to a single number: the bar's bottom edge minus the card's top edge. Measured across 14 viewports at device pixel ratios 1, 2 and 3, on both the Text and Screenshot tabs, that number is 0.813px, 0.203px or exactly 0.000 - 84 readings, not one of them a whole pixel, and 0.000 at 390x844 on the Screenshot tab, which is the exact case reported. At zero the outcome is not a decision the stylesheet makes; it belongs to whatever the engine does with two edges on the same coordinate, which is why a DPR 3 phone can show a line a DPR 1 desktop does not. Chromium rounds the same way on both sides and paints no sliver at DPR 3 either - walking the bar's height in thirds of a pixel did not produce one. So this is the margin being fixed, not a pixel being chased: after the change the worst of the same 84 readings is exactly 1.000px. The alternative - a white skirt under the active tab, box-shadow: 0 1px 0 - would have been more precisely scoped and does not work. The shadow hangs below the bar's last child and #diff-header's overflow: auto, which is what enforces its height cap, clips it. Measured with the card held 3px away so the skirt and the card are separable: 0 device rows of the 3 it needs at 844x390 and 926x428, and 0 to 2 in the sticky modes. The negative margin is what survives the clip, because it moves the card rather than painting past an edge. Scoped to body.difference-page, which matters more than it looks. diff.css is not diff-page-only: preview.py and processors/extract.py serve it too, both pages have a #diff-ui of their own, and neither has a bar above it to absorb the pixel. Asserted rather than reasoned - the preview page's computed margin-top is 0px with this stylesheet and with the previous one, while the diff page's moves to -1px. The comment on the activity-strip rule above claimed diff.css was loaded only by the diff page; that was wrong and is corrected, since the scope it describes is what actually does the work. What the overlap costs, stated rather than left to be found: where the bar is sticky, nothing. A device-pixel diff of the seam band at 390x844 and DPR 3 comes back with zero differing pixels, because the bar is opaque and paints over the card. Below the short-viewport breakpoint the bar has no background of its own, so the card's white also reaches the bottom pixel of the inactive tabs and the 5px gaps between them - 4374 device pixels at 844x390, gradient turning white, and side-by-side crops of the two are indistinguishable. The tab row ends up sitting flush on the card instead of floating a pixel above it, which is how tabs normally meet their panel and is the same continuity the active tab was asked for. Full branch harness, the previous round's probe and this round's all re-run green against the change. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Reuse upstream's Previous and Next for the version arrows' labels 297bd365 gave the version arrows `title` and `aria-label` of their own, "Previous version" and "Next version". Those are two msgids this branch introduced and no catalog has ever carried, so on a German or Japanese page the arrows' accessible name is English. The upstream msgids `Previous` and `Next` say the same thing in this context - the arrows live inside the version bar, between the From and To selectors - and are non-empty in 15 of 17 catalogs; the two empty ones are en_GB and en_US, where the English msgid is the correct rendering anyway. The visible span on each arrow already uses these msgids, so the accessible name and the (header-hidden) label now agree instead of differing by a word. The cost is that the accessible name drops "version": a screen reader says "Previous" rather than "Previous version". On a control that sits between two datetime selectors that is enough, and it is the price of not asking fifteen translators for a string they can already spell. The catalogs are regenerated: both msgids disappear from the .pot and from all 17 .po files. No .mo file changes, because neither msgid had a msgstr anywhere to compile. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Correct the French translation of "Previous" fr renders the msgid `Previous` as "Aperçu", which is French for *preview*. That is a wrong word, not a style choice: the same catalog gives "Aperçu" as the msgstr for `Preview` and for `Activate preview`, and it evidently leaked from there. The msgid renders in exactly two places, both of them prev/next pairs - this page's version arrows and preview.html's own pager - and fr already translates `Next` as "Suivant", so "Précédent"/"Suivant" is the pair the catalog is otherwise using. On the preview page the current string produces an "Aperçu" button on the page called Aperçu. Left alone deliberately: de renders `Previous` as "Zurück" (*back*). That is defensible on a back arrow and is a translator's call, not an error. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Put the watch's name on the top line and drop the diff page's second title row The diff page printed the same thing twice. The menu's top line has always shown the watch URL, and 297bd365 added a sticky <h1> under it showing watch.label - which for any watch without a custom title *is* that URL. Two rows of chrome, 30px of the sticky budget, and on the common case the identical string on both. dgtlmoon's read (#4412): put it all on the top line, use the title if it is set, and take back the left inset while we are there. That is what this does, and it removes code rather than adding any. The top line's <a> is unchanged as a link - same href, same fade-out mask, same flex-shrink that keeps a long value off the menu icons. Only its text changes: current_diff_label (watch.label - title, then the fetched page_title, then the URL) with the URL as the fallback, so an untitled watch reads exactly as it does today. The five views that already pass current_diff_url pass the label beside it. The hover text now carries both, label then URL on its own line, and only when they differ. A title covers the URL that used to be printed here, and the URL is also where the link goes - so it has to stay readable without following it. That also keeps the long-title-on-long-press property the <h1> had. The heart hides under $desktop-wide-breakpoint on this page only. That is the width where the hamburger appears and the row is at its tightest, so it is where the name needs the space; on a desktop diff page and on every other page the heart is untouched. It is a media query rather than a {% if %} because the heart's reason to exist is that it is always there - folding it away with the rest of the narrow row is a different thing from taking it off the page. Left-justification: the line's text started $common-gap twice in - the menu row's own padding plus the link's own left margin. On the row's first item the second one only repeats the first, so it goes and the row's padding stays. 8.8px back, and the line still shares a left edge with the diff below it rather than sitting hard against the viewport. The right side of both is untouched. No new or removed translatable strings, so no catalog churn. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Fill the empty msgstr cells this page's own strings left behind 30 cells across 15 catalogs. No new msgids: extract_messages produces a byte-identical .pot (bar POT-Creation-Date) and update_catalog leaves every .po unchanged, which is the control that nothing here is a source change wearing a translation's clothes. Filters - empty in all 15 non-English catalogs since the diff page's options popover was added. Each value is the plural form that catalog already uses for "Global Filters" and "Filters & Triggers", not a fresh translation: Filtry, Filter, Filtros, Filtres, Filter, Filtri, フィルタ, 필터, Filtry, Filtros, Фильтры, Filtreler, Фільтри, 过滤器, 過濾器. Screenshot - this is a regression the diff-header work introduced and owes. The tab used to read "Current screenshot", which every catalog translates; shortening it to "Screenshot" moved the tab onto a msgid that nine of them (cs, de, es, fr, it, pt_BR, tr, uk, zh) had never filled, so those users got an English tab where they had a translated one. Each value is that catalog's own "Current screenshot" with the qualifier dropped - Snímek obrazovky, Captura de pantalla, Capture d'écran, Ekran görüntüsü, Скріншот, 截图 - and for de, it and pt_BR the word those catalogs already use is literally "Screenshot" (Aktueller Screenshot / Screenshot corrente / Screenshot atual), which is what dennis W302 flags and CI excludes. Italian's five pre-existing gaps on this page, deliberately in scope: the guide's multi-language rule is to fix every affected catalog in one session rather than fix one language and move on, and an Italian diff page was showing untranslated From, To, Ignore Whitespace, Keyboard:, Extract Data and Jump to next difference next to strings that were translated. Ignora spazi and Estrai dati come from the same catalog's "Ignore whitespace" and "Extract text" / "Extract as CSV"; successiva from its "Next" -> "Successivo" and differenza from its "Pixel Difference Sensitivity". Compiled with setup.py compile_catalog. en_GB and en_US recompile byte-identical, so the 15 changed .mo files are exactly the 15 changed .po files. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Let the watch title be dragged when it is too long to fit Asked for on the PR: "does the title at the top accept overflow scroll hidden so it can be scrolled with your finger/touch/drag?" It could not - .current-diff-url was overflow: hidden with a fixed right-edge fade, so the end of a long title was reachable only by hovering for the title attribute, which a phone does not have. It is now a horizontal scroll container with its scrollbar hidden in both engines. A tap still follows the link; browsers distinguish a pan from a click, so the anchor keeps its navigation. overflow-y stays hidden because the content is one non-wrapping line and an axis that can move by a rounding error lets a diagonal swipe jog the text. The fade has to follow the scroll, or the ending someone just dragged over to read is the part that is dimmed. static/js/scrollable-title.js maintains fade-left / fade-right against scrollLeft and adds js-fades to say it is doing so; under that class the CSS drops the static mask and paints only the edge that is actually cut off, or a two-ended gradient when both are, or nothing at all when the title fits. The static right-hand fade stays as the no-JS default, which is correct there because without a handler the line can only ever be cut on the right. In base.html rather than diff-overview.js: restock_diff/difference.py and image_ssim_diff/preview.py both render this line and neither loads that file. The script guards on the element existing, so it is inert on every other page. Accepted tradeoff: with the anchor focused, ArrowLeft/Right still runs version navigation on diff pages via the existing window keydown handler rather than scrolling the line. Keyboard users keep the hover title attribute, as today. styles.css is spliced, not rebuilt. parts/_top_menu.scss compiles into styles.css, whose committed bytes do not match a fresh compile of its own scss - an older sass folded rgba(255,255,255,X) to hsla(0,0%,100%,X) and hoisted nested declarations in a different order (#heartpath, .button-tag, .tab-pane-inner, .watch-table img.favicon, #browser_steps li and more). That drift is upstream's and predates this branch. So rather than commit a full rebuild and carry all of it, the scss was compiled twice - at the parent commit and with this change - and only the span that differs between those two compiles was transplanted into the committed file. The drift is identical in both compiles and cancels. The result differs from the parent by 795 bytes, all inside the .current-diff-url rules, and is byte-identical everywhere else; diff.css is untouched. A reviewer who runs npm run build will see a dirty styles.css - that mismatch is the pre-existing drift, not this commit. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Cut diff-overview.js back from a slab to a file Raised on the PR as "a slab of javascript... why cant this be a few lines of jquery". Measured before arguing: upstream's file is 190 lines (144 code, 20 comment) and this branch had grown it to 476 (284 code, 142 comment). Half the growth was comment, written at review density rather than this project's, and that is what makes it read huge. 476 -> 370 (256 code, 72 comment). Every long forensic comment is cut to the constraint it was protecting - the measurement that motivated it survives as a clause, the war story does not. Nothing load-bearing was deleted: the capture-phase scroll listener still says why capture, the visual-viewport listener still says why it exists separately from resize, and place() still says why offsetHeight is read after the cap. Converted to jQuery idiom where that is genuinely shorter, so the file reads in one voice: element lookup, class toggling, open/close, and the event bindings that need no listener options. The geometry stays native, because there is no jQuery shorthand for it - getBoundingClientRect, getComputedStyle, offsetHeight, visualViewport and ResizeObserver are the added code, and popover placement is what they are for. Two registrations deliberately stay on addEventListener and say so in place: the bar's scroll listener needs capture (scroll does not bubble and the element that scrolls is a descendant) and both scroll listeners need passive, neither of which jQuery's .on() can pass. The one structural change is that the four copies of "step both selects one position and rebuild the query string" collapse into diffStepHref(direction), called by the buttons at setup and by the arrow keys at press time. The guards are unchanged: the buttons are only touched when both selects have a selection, the arrow handler still requires both a live step and a button href, and it still returns early while an input, textarea or select has focus. Not done here, and offered instead as a follow-up: moving the popover to the native Popover API. It would delete the dismissal wiring, but placement stays manual without CSS anchor positioning, which is not yet safe cross-browser - and it would churn code that is tested and green. No new translatable strings, so the catalogs are untouched. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> * Point the top-menu comment at the rule it describes The previous commit inserted the fade variable and mixin between this block and .current-diff-url, which left the comment describing the declaration above it rather than the rule it was written for. It also still said the line "trims" its text, which is now only half true - it scrolls, and the trim is what the fade stands in for. Comment only: sass emits byte-identical css, so styles.css is unchanged. Co-authored-by: Jeff Hedlund <jhedlund@gmail.com> Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> --------- Signed-off-by: Jeff Hedlund <jhedlund@gmail.com> Co-authored-by: Surveyor <surveyor@agents.matrixsi.com> Co-authored-by: Engineer <engineer@agents.matrixsi.com> Co-authored-by: Architect <architect@agents.matrixsi.com> Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>	2026-09-16 14:19:00 +02:00
dgtlmoonandClaude Opus 5	7102544310	xPath filters - Stop LC_COLLATE deciding what contains() means (#4437) (#4438) ... Every watch whose xPath filter used contains() started reporting "Warning, no filters were found" on pages whose HTML plainly contained the target. 18 unrelated watches broke in the same hour, browser and plain-requests fetchers alike, and the saved snapshot was perfect every time. elementpath implements the XPath string functions on top of locale.strxfrm: def contains(self, a, b): return self.strxfrm(b) in self.strxfrm(a) Under LC_COLLATE=C, strxfrm() is the identity function and that is an ordinary substring test. Under a real locale it returns a binary collation key, and a substring of a collation key is not the collation key of the substring - so contains(), starts-with(), ends-with() and substring-before/after() return false for EVERY input. Reduced to one line, no document needed: LC_COLLATE=C contains("xx month xx", "month") -> True LC_COLLATE=en_US.UTF-8 contains("xx month xx", "month") -> False Name tests, axes and '=' are untouched, which is exactly why it read as "did the page change layout?" - //div kept working while //div[contains(.,"month")] returned nothing. No code change caused this. Generating the image's locales (#4429) made ENV LC_ALL=en_US.UTF-8 satisfiable for the first time; flask_app's setlocale(LC_ALL, ...) had been raising locale.Error and leaving us in C, and once it succeeded it took LC_COLLATE with it. That is why the bug survived a bisect to 0.60.2 and why reinstalling the exact pip set from a working install did not shift it - it could only be found by diffing the two containers. Same image base, same Python 3.11.16, lxml 6.1.3, libxml2 2.14.6, elementpath 5.1.1, same HTML: good-old 0.60.4 setlocale FAILED: unsupported locale setting 67 matches bad-new 0.60.5 setlocale en_US.UTF-8 0 matches Fixed in both places, because either alone leaves a hole: - flask_app sets LC_CTYPE/LC_NUMERIC/LC_MONETARY/LC_TIME individually instead of LC_ALL. This block exists to make prices render correctly and it still does - 1234567 is still "1,234,567" - it just no longer touches collation. - html_tools.xpath_filter() pins the Unicode codepoint collation per evaluation, so a filter means the same thing whatever an operator puts in LANG/LC_ALL, and does not depend on a distant module's locale bookkeeping. forms.py's XPath validation pins it too, so validation cannot accept an expression that then behaves differently at check time. Per XPath 3.1 the default collation is codepoint and must not consult LC_COLLATE, so the underlying behaviour is arguably an elementpath bug; the pin above holds regardless. Verified end to end inside the failing container: LC_COLLATE=C, LC_NUMERIC=en_US.UTF-8, thousands separators intact, and the reported filter back from 0 to 1190797 chars of output. Tested: new unit test covers contains/starts-with/ends-with under a UTF-8 collation and was checked to fail without the fix; 325 unit tests pass. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-14 20:33:35 +02:00
dgtlmoonandClaude Opus 5	718295f30b	UI - Search modal - keep the search scoped to the tag you were viewing (#4425) ... The modal submitted the active tag as `tags=<uuid>`, but the watchlist filters on `tag` - nothing reads `tags`. So searching from inside a tag view silently searched every watch, despite the label promising "URL or Title in '<tag>'". The test pulls the hidden field straight out of the rendered modal and feeds it back to the watchlist, so the field name can't drift from the arg again. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-12 17:48:48 +02:00
3 people	cd66f9ed54	Search modal native submit (#4427) ... * UI - Search - Fix search modal navigating to the host root on sub-path deployments base_path was referenced by search-modal.js but never defined, so searching always jumped to the host root instead of the X-Forwarded-Prefix sub-path. * UI - Search modal - submit natively instead of rebuilding the URL in JS Alternative to defining a `base_path` JS global: give the search form a server-rendered `action`, so url_for() supplies the reverse-proxy sub-path the same way every other link on the page already does, and let the browser submit. Drops the submit handler and the Enter handler from search-modal.js - Enter in the input reaches the footer's submit button via implicit submission, which also runs the `required` validation the synthetic `new Event('submit')` skipped. The hidden tag field is only rendered when a tag is active, so a plain search no longer carries an empty value. Also drops the nginx-job grep for the rendered markup - test_search.py already covers the sub-path case, and asserting on an exact HTML attribute string from a shell grep breaks on any unrelated edit to that tag. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: ponstream24 <87808547+ponstream24@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-12 17:09:10 +02:00
dgtlmoonandClaude Opus 5	8d938b5966	Browser fetchers - Follow re-navigation, and one content-ready deadline (re-targets #4421 + #4422 at master) (#4426) ... * Browser fetchers - Judge a fetch on the document we end up extracting, not the first navigation The goal is to compare the text of the page the browser lands on, even when the site navigates again after the first response. Both fetchers were bound to the first navigation, which shows up as two different bugs: 1. pyppeteer hangs until the hard processing timeout. Its navigation watcher is bound to the loaderId of the navigation it started, so when the site replaces that document the 'load' it waits for never arrives for that loaderId. With timeout=0 and setDefaultNavigationTimeout(0) there is nothing to break the wait, so goto() blocks until PUPPETEER_MAX_PROCESSING_TIMEOUT_SECONDS (180s) kills the fetch and the watch records an empty xpath_data - while the browser is sitting on a fully loaded page. Traced on slated.com: 0.24s goto start 0.74s main frame networkIdle loaderId=A0E0E0B2 <- never gets 'load' 2.72s main frame init loaderId=56D2B5EF <- re-navigated to get.slated.com 3.49s main frame load loaderId=56D2B5EF <- fires for the new document 25.2s goto still hanging, frame._loaderId is now 56D2B5EF Now the navigation races goto() against the main frame firing 'load', bounded by BROWSER_NAVIGATION_TIMEOUT_SECONDS (default 30), and falls back to the document we can see. slated.com / getastra.com / addupsolutions.com went from a 180s timeout with no content to 200 with full content in 5-35s. 2. Both fetchers reported the status of the interstitial. A site that gates unseen visitors with an error status plus a client-side redirect (reported against fotokoch.de: 503 + meta refresh, then a 200 with the real page) failed the watch even though the content was present, and the only workaround was ignore_status_codes, which also hides genuine 404s and 500s forever. The fetchers now keep the latest main-frame document response and judge on that - the refresh lands during the existing extra_wait, so the 200 wins. Playwright also waits for a settled load state before extracting, which is what produced "Execution context was destroyed, most likely because of a navigation" when the refresh collided with extraction. The navigation-response tracker is installed once per page and shared between the fetcher and action_goto_url() rather than each navigation adding its own listener - 'response' fires once per HTTP response, hundreds of times on a heavy page, so the callbacks are worth not duplicating. Verified one listener remains after an install plus four navigations. Selenium is unaffected either way - it hardcodes status_code = 200 because WebDriver cannot see the HTTP status. Tested: new test_renavigation.py covers the interstitial case end to end and was checked to fail without the fix and pass with it, on both fetchers. The test endpoint gates on last-seen time rather than a hit count, because a counter lets the second check see a clean 200 and the test then passes without the fix. Full browser suite 11 passed on playwright and on pyppeteer, 494 unit tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Puppeteer fetcher - One content-ready deadline instead of a stopLoading watchdog per frame event Page.stopLoading is what stops a page that would otherwise load forever waiting on a subresource that never answers, so that we can still screenshot and scrape what rendered. That intent was right, but it was implemented as a fire-and-forget task armed by every frame event, which measured on a single fetch of an iframe-heavy page came to: 14 watchdog tasks spawned 11 page-wide Page.stopLoading calls 3 tasks outliving the fetch and firing against a closed page Page.stopLoading takes no frame or loader argument - it is the Stop button, and it stops the whole page. Verified directly: one call stopped a pending main frame and a pending iframe in the same instant. So the other 10 calls were redundant, and because they landed at arbitrary later times they could stop a *subsequent* navigation we actually wanted - which is the likeliest reason the same URL fetched in 8s on one run and 35s on the next. Replaced with a single deadline, awaited inline so nothing can outlive the fetch (there is no create_task left in this file at all): navigate (bounded) -> wait the configured delay -> Page.stopLoading -> extract The delay is measured from when navigation finished, not from when it started. Anchoring it to the start would quietly rob a slow-loading page of its settle time, and letting JS-rendered content appear after load is the whole point of the setting. Verified with a server that takes 5s to answer and renders via JS 2s after load: total 9.6s for a 4s delay, and the late content is captured. Because a page is never reliably "finished" - many sites navigate as part of their normal design - the delay restarts when the MAIN frame replaces its document, so a redirect or interstitial gets the same settle time the first document got. Iframes do not restart it, and it is capped by BROWSER_CONTENT_READY_MAX_RESETS (default 2). Only the existing "wait n seconds before extracting text" stays user-facing; BROWSER_NAVIGATION_TIMEOUT_SECONDS is a safety net with a sane default rather than a second knob for users to reason about. This matches what other scrapers do: bound the navigation, do not fail when it times out, settle, then extract. Timings are also more predictable now - the four reported URLs went from 5-35s of variance to 4.6-7.2s at a 3s delay, all with full content and a 200. Tested: 11 passed pyppeteer browser suite, 7 passed playwright, 494 unit + llm. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-12 17:08:28 +02:00
Jeff Hedlund	2f60135cd0	feature: Implemented optional override link (#4290) ... * Implemented optional link to override feature * Added translation strings (untranslated)	2026-09-09 16:06:24 +02:00
dgtlmoon	7f3645f4a8	API - DELETE for Watch history #4397 (#4403)	2026-09-09 15:46:00 +02:00
3 people	455e0228ca	Tests - Fix TestHistoryPathTraversal on macOS, and cover the containment check (#4390) ... * Compare against a resolved data_dir in the history path-traversal test Watch.history resolves entries with os.path.realpath, so test_normal_snapshot_entry_is_accepted compared a resolved path against an unresolved data_dir. On macOS the datastore lives under /tmp, which is a symlink to /private/tmp, so the assertion fails for a path that is in fact inside the directory. The guard is correct; the test was not. Resolve both sides, matching what the production code does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Add a test that actually exercises the history containment check Disabling the containment check in Watch.history left every test in TestHistoryPathTraversal passing. os.path.basename() reduces both traversal fixtures ('/etc/passwd', '../../etc/passwd') to 'passwd', so neither reaches the check — they stop at the os.path.exists() test below it. A bare '..' survives basename() and resolves to the parent of data_dir, which exists, so the containment check is what rejects it. With that check disabled this new test is the only one in the class that fails. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: GG5533 <285285461+GG5533@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-09-09 15:43:54 +02:00
dgtlmoon	3a71777499	UI - Fixing lots of actions that should be a POST style action which led to a 404 ... Also solves some of GHSA-56fq-63vj-9992 Add-watch-UI should be POST/CSRF protected	2026-09-04 12:38:20 +02:00
dgtlmoon	e0fb224d41	UI - Fixing lots of actions that should be a POST style action which led to a 404 (#4370)	2026-09-04 10:56:23 +02:00
dgtlmoon	050172129e	Translations - support for translation overlays/local translations of existing strings (#4365) ... * Translations - support for translation overlays/local translations of existing strings * Dont forget translations_overlay for pip	2026-09-03 11:15:46 +02:00
Andrew PeabodyandAndrew Peabody	3d6de42924	fix(conditions): support zero values in condition filtering and json logic conversion (#4362) ... * fix(conditions): support zero values in condition filtering and json logic conversion - Fix filter_complete_rules dropping rules where value is 0/0.0 due to 0 == False in Python - Fix convert_to_jsonlogic raising EmptyConditionRuleRowNotUsable on value=0 due to truthiness check - Fix str != 'None' type comparison typo in convert_to_jsonlogic - Add comprehensive unit tests covering zero value condition filtering, conversion, and execution * chore: re-trigger CI checks --------- Co-authored-by: Andrew Peabody <apeabody@users.noreply.github.com>	2026-09-03 08:57:34 +02:00
dgtlmoonandClaude Opus 5	fbd9472218	fix(api): accept an existing tag UUID in the watch tag field, and deprecate it (#4361) ... `tag` on POST /watch was documented as taking a tag UUID, but the value went to add_tag(title): a UUID silently created a junk tag *titled* with that UUID and never applied the tag the caller asked for. `tags` (UUIDs) was the only thing that worked. - `tag=` now resolves an existing tag UUID to that tag, still falling back to title matching/creation for names. A UUID-shaped value matching nothing is skipped with a warning rather than becoming a group named after a UUID. - Blank tokens ("One,,Two,") no longer store False in watch['tags'] - add_tag() returns False for an empty title and it was appended unguarded. Consumers tolerate it (get_all_tags_for_watch() dictfilt()s over known tags) but it is not valid data. - add_tag()'s title search is extracted to tag_uuid_for_title(), so existence can be tested without creating as a side effect. add_tag()'s contract is unchanged. - api-spec: `tag` is marked `deprecated: true` (so Redoc renders the badge) and states plainly that it takes names, not UUIDs. `tags` now says what it really does - applied verbatim, never creates, unknown UUIDs stored as dangling refs. Rendered docs rebuilt. Every claim in the new field docs is asserted in test_api_tags.py against the real API. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-02 16:46:27 +02:00
dgtlmoonandClaude Opus 5	2669a5cee6	Feature/Watch limit - Adding test (#4360) ... * Env var - PAGE_WATCH_LIMIT enhancements * Rebuild API docs * Bump APi doc version * test: cover PAGE_WATCH_LIMIT across every add path - API create returns 429; API import returns 429 and refuses the batch whole - quick-add and the UI importer flash the limit (importer once per file, not per row) and hand unimported URLs back - clone at the limit no longer KeyErrors - an instance already over the limit still loads from disk and stays editable, only new watches are refused - the Info tab shows the limit only when one is set - add_watch() with no request context returns None instead of raising from flash() - an absent, empty or unparseable env var all mean unlimited Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-02 15:43:59 +02:00
Andrew Peabody	7708ffa986	fix(llm): support Gemini flash-lite & reasoning models, strip extra_body on 400 retry (#4342) ... - Define _NO_TEMPERATURE_MODEL_KEYWORDS constant and omit temperature=0 on initial request - Omit thinkingConfig in _thinking_extra_body for flash-lite models - Unconditionally strip rejected sampling params and extra_body thinkingConfig on BadRequestError (handling Gemini 400 INVALID_ARGUMENT without named details) - Add comprehensive unit tests in test_llm_client_gemini.py	2026-09-02 10:45:39 +02:00
dgtlmoon	97317398ac	LLM - Per watch/group settings and UI tidyup (#4354) ... * UI - LLM section tidyup * Rebuild translatiosn * UI - Fixing language for prompt adjustement to be more clear * UI - clarify field action * UI - clarify layout for sections * WIP * test tweaks	2026-09-02 10:07:34 +02:00
dgtlmoon	10521dd5e1	UTF-8/Lone Surrogate encoding test improvements related to #4273 (#4353)	2026-09-01 17:55:13 +02:00
dgtlmoon	227b011d73	Memory usage - small improvement by tidying up module imports (#4352)	2026-09-01 17:50:50 +02:00
dgtlmoon	dce6f23cf9	Scheduler+API Bug - if an invalid timezone was set (through edit of watch or API) it could have crashed the scheduler, Added timezone to the official API docs (#4350) ... * Scheduler+API Bug - if an invalid timezone was set (through edit of watch or API) it could have crashed the scheduler, Added `timezone` to the official API docs * adding missing files	2026-09-01 17:00:03 +02:00
dgtlmoonandClaude Opus 5	143a0f116f	fix(llm): handle reasoning blocks that aren't a well-formed <think>...</think> pair (#4349) ... Follow-up to #4340. The <think> stripper added there only matches a closed pair, but a reasoning scratchpad routinely contains JSON of its own ("initially I thought {"important": false}, but..."), so any leftover scratchpad lets _extract_json lock onto a discarded intermediate answer. Three shapes slipped through, all of which inverted the verdict to important=false and therefore silently suppressed the notification: - opener stripped by the provider/chat template, only </think> comes back over the wire - <thinking> spelled out in full - unterminated block, i.e. the response was cut off mid-thought by max_tokens The first two are now stripped. The third raises ValueError, because a truncated response holds no answer at all - only the abandoned guess. Raising routes it to the existing handler in evaluator.py, which passes the change through as important rather than dropping it; parse_eval_response deliberately does not catch ValueError, since its own fallback (important=False) would suppress the notification instead. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-01 12:42:16 +02:00
Andrew Peabody	9e54ed3adb	fix(llm): string boolean parsing, reasoning model <think> stripping, and timestamp diff annotation (#4340) ... - Simplify _to_bool using changedetectionio.strtobool.strtobool - Strip <think>...</think> reasoning blocks in _extract_json for reasoning models - Fix _annotate_moved_lines short-circuit so standalone relative timestamps are always annotated - Add comprehensive unit tests in test_response_parser and test_prompt_builder	2026-09-01 11:52:40 +02:00
dgtlmoon	7b61c20e68	Python 3.10 EOL October/2026 - Dropping support - conflicts with jsomschema and litellm (#4347)	2026-08-31 21:19:12 +02:00
dgtlmoon	5e7c9f7aa6	Puppeteer/Playwright CSP bypass - test refactor (#4346)	2026-08-31 16:44:00 +02:00
Andrew Peabody	be75168570	fix(backup): add utf-8 encoding when writing url list files (#4345)	2026-08-31 13:41:46 +02:00
dgtlmoon	c24f338f95	Add watch UI - Tidying up logic around browser selection/visibility	2026-08-26 12:29:55 +02:00
TowyTowyandClaude	5e9a8583e2	Restock detection - fix inverted condition that skipped the OpenGraph availability fallback (#4262) ... * Restock detection - fix inverted condition that skipped the OpenGraph availability fallback get_itemprop_availability() only dug through OpenGraph properties when price was missing OR when availability was ALREADY found - so on pages that expose price via JSON-LD but stock state only via the OpenGraph commerce tags (<meta property="product:availability">), the availability (and currency) was silently dropped. The inner loop already guards each field with 'if not value.get(...)', so the outer gate now matches that intent: dig OpenGraph when any of price/availability/currency is still missing. Co-Authored-By: Claude <noreply@anthropic.com> * Normalise availability after the OpenGraph fallback, not before An availability found via OpenGraph was left raw, so the Facebook commerce spelling 'in stock' never matched the 'instock' test and the product read as out of stock. --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-26 10:59:29 +02:00
Karl Q.	465ff40bde	feat: make Playwright CSP bypass configurable (#4298)	2026-08-25 22:24:10 +02:00
dgtlmoon	bb2e6adece	Add watch UI should show which browser(s) are available and set default browser (#4334)	2026-08-25 20:15:59 +02:00
dgtlmoon	d11b584fe4	LIBXML Data integrity fix, libxml must be locked across threads for any XML parsing (#4325)	2026-08-24 19:21:07 +02:00
dgtlmoon	f42da28dab	GHSA-23mp-8222-96fr - reflected XSS in /diff/<uuid>/download-patch via from_version - Using built-in plain text response instead. Reported-by: Mayssare Amakhtari (@cy3erm) ... Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> (#4324)	2026-08-24 16:24:22 +02:00
dgtlmoon	b1bb5371d5	GHSA-q85w-c766-h5g8 - Advisory for four unprotected routes in price_data_follower, check_proxy and language handling that missing. Reported-by: Mayssare Amakhtari (@cy3erm) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-24 13:47:31 +02:00
dgtlmoon	288926b44e	GHSA-4j9m-cxq4-9c36 - Improved validation Stored CSS Injection via tag_colour Field (#4317)	2026-08-20 17:59:02 +02:00
dgtlmoon	0956318000	RSS - RSS pubDate is wrong on any non-UTC container: local time is relabelled as UTC - #4309 (#4314)	2026-08-20 15:49:29 +02:00
3 people	01cf56c8fb	feature: allow a watch or group to append to the inherited AI Change Summary prompt (#4294) ... Co-authored-by: snowyukitty <270071858+snowyukitty@users.noreply.github.com> Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>	2026-08-20 15:14:59 +02:00
dgtlmoon	6b9040256a	CSRF - Test fix (#4316)	2026-08-20 15:13:16 +02:00
Northern Monkey	4bfce3c4b0	Security: hide the running version from anonymous visitors when a password is set (#2190) (#4306)	2026-08-20 11:09:13 +02:00
dgtlmoon	89a41c0dda	GHSA-3v57-xv5m-v7m7 - Multiple front-end endpoints lacking CSRF protection	2026-08-20 10:59:38 +02:00
dgtlmoon	acc4533f51	Security - GHSA-mh42-m7cg-49fr - Authenticated path traversal to arbitrary file write via the watch processor field (enum bypassed through /imports/import)	2026-08-20 10:56:13 +02:00
dgtlmoon	6f2e4aa9af	UI - Move rendering of each watchlist row out of the watchlist main template so it can be re-used by the realtime update (#4305)	2026-08-16 18:07:34 +02:00
snowyukittyandsnowyukitty	21f19b239a	fix: avoid division by zero when logging watch load rate (#4289) ... Co-authored-by: snowyukitty <270071858+snowyukitty@users.noreply.github.com>	2026-08-16 14:26:39 +02:00
snowyukittyandsnowyukitty	3e3300896f	fix: invalidate favicon cache when clearing watch history (#4286) ... Co-authored-by: snowyukitty <270071858+snowyukitty@users.noreply.github.com>	2026-08-16 14:24:45 +02:00
dgtlmoon	c3361943aa	Improved content type detection - adding regression test	2026-08-15 18:59:59 +02:00
snowyukittyandsnowyukitty	5d47a0764d	fix: sanitize lone unicode surrogates before filtering JSON (#4293) ... Co-authored-by: snowyukitty <270071858+snowyukitty@users.noreply.github.com>	2026-08-15 17:24:47 +02:00
Jan Kahmen	aac6fcfa59	fix: Visual Selector renders scraped selectors as text, not markup (#4282)	2026-08-04 09:11:33 +02:00
dgtlmoon	fdfa3859ee	Watch history - fix for restock.previous_price returns the first-ever price once a watch has 3+ snapshots (#4268)	2026-07-22 11:23:53 +02:00
Willy Farrell	966f89736b	fix: watches auto-tagged by group url_match_pattern regex missing from group tab filter (#4267)	2026-07-22 08:44:22 +02:00
dgtlmoon	71d332d5a0	UI full refactor / upgrade (#4183)	2026-07-16 15:08:29 +02:00
Ehsan	5eb2638ffc	fix: report the actual 1-based failed browser-step number (#4200) (#4258)	2026-07-14 14:48:36 +02:00