Compare commits

...
105 Commits
Author SHA1 Message Date
skkzsh d789fe3ea5 i18n: Fix untranslated labels (#4488)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
* gettext "No proxy"

* Fix untranslated Settings label

* Unify `.po` header fields across all languages
2026-09-25 18:31:40 +02:00
skkzsh 61f03cdc69 Use lazy_pgettext for condition Match label (#4480)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-25 11:31:00 +02:00
dgtlmoon 6704fdb29e Title extract - Limit search window and improve regex - reduce memory usage (#4487) 2026-09-25 11:26:21 +02:00
dgtlmoon 15615ccf6f LLM - Disable by default the fetching of price mappings from GitHub on startup 2026-09-25 09:32:02 +02:00
dgtlmoon 1df1131095 UI - CSS cleanups in LLM settings area
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-25 08:22:33 +02:00
dependabot[bot] 7370395fd3 Update python-socketio requirement from <6,>=5.11.0 to >=5.17.0,<6 (#4486)
Updates the requirements on [python-socketio](https://github.com/sponsors/miguelgrinberg) to permit the latest version.
- [Commits](https://github.com/sponsors/miguelgrinberg/commits)

---
updated-dependencies:
- dependency-name: python-socketio
  dependency-version: 5.17.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-25 08:05:18 +02:00
dependabot[bot] 3328d8ad73 Update flask-wtf requirement from ~=1.2 to ~=1.3 (#4485)
Updates the requirements on [flask-wtf](https://github.com/pallets-eco/flask-wtf) to permit the latest version.
- [Release notes](https://github.com/pallets-eco/flask-wtf/releases)
- [Changelog](https://github.com/pallets-eco/flask-wtf/blob/main/docs/changes.rst)
- [Commits](https://github.com/pallets-eco/flask-wtf/compare/v1.2.0...v1.3.0)

---
updated-dependencies:
- dependency-name: flask-wtf
  dependency-version: 1.3.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-25 08:05:02 +02:00
dgtlmoon fd7c9db23e UI - CSS - Mobile - Just remove disabled items, cleaner output.
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-21 18:06:05 +02:00
dgtlmoon 474c70f37c UI - CSS - Mobile - Fixing padding on edit pages 2026-09-21 18:00:42 +02:00
dgtlmoon 8689ad9dfa UI - CSS - Mobile - Add little home button at top 2026-09-21 17:57:32 +02:00
dgtlmoon 59827a98f8 UI - CSS - Mobile improvements 2026-09-21 17:52:36 +02:00
dgtlmoon 63b0ad0ccd Merge branch 'master' of github.com:dgtlmoon/changedetection.io
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-20 20:29:58 +02:00
dgtlmoon 167597628d LLM - History/difference page - CSRF fix for LLM summary fetching between versions 2026-09-20 20:29:36 +02:00
dependabot[bot] 95f06c5004 Update selenium requirement from ~=4.31.0 to ~=4.49.0 (#4456)
Updates the requirements on [selenium](https://github.com/SeleniumHQ/Selenium) to permit the latest version.
- [Release notes](https://github.com/SeleniumHQ/Selenium/releases)
- [Commits](https://github.com/SeleniumHQ/Selenium/compare/selenium-4.31.0...selenium-4.49.0)

---
updated-dependencies:
- dependency-name: selenium
  dependency-version: 4.49.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-20 20:28:19 +02:00
dgtlmoon ed217644fa Our first basic PWA mobile integration! Look for the QR-Code in Settings (#4472) 2026-09-20 19:45:11 +02:00
dgtlmoon 239a69dcc4 UI - Prevent accidental change of LLM Key (#4473) 2026-09-20 19:44:06 +02:00
dgtlmoon de4637c0bd LLM - Fixes for temperature/reasoning GPT-5 etc 2026-09-20 18:09:55 +02:00
dgtlmoon 11916a89e6 LLM - UI nonblocking (#4469)
* Make LLM summary from UI non blocking

* Increase logging
2026-09-20 16:00:12 +02:00
dgtlmoon 4da220be62 LLM timeout improvements (frontend) (#4470) 2026-09-20 15:59:58 +02:00
dgtlmoon a165df2da9 LLM UI - When LLM is disabled, hide it from the groups settings also (#4471) 2026-09-20 15:54:41 +02:00
dgtlmoon dd588bf3e3 Make LLM summary from UI non blocking (#4467)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-20 13:01:14 +02:00
dgtlmoon 1b7219755f Groups - UI fix for #4451 and tweaks (#4466) 2026-09-20 09:11:16 +02:00
dgtlmoon 7086a1769b UI - LLM - Tidy up styling 2026-09-20 08:10:30 +02:00
Corey Quinnanddgtlmoon 8776c205dc UI: make editing a configured LLM model discoverable (#4400)
Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>
2026-09-20 08:06:04 +02:00
skkzsh 9c5600087a i18n: Add ja translations for new UI (#4464)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-19 10:46:15 +02:00
skkzsh 0426ce0424 i18n: Use msgctxt and correct ja translations for -ed forms (#4460)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
* Annotate fragmented gettext locations with marker comments (#4074)

* update_catalog

* Re-add translator marker comments after UI refactor

* Use `msgctxt` and correct ja translations for -ed forms

# Conflicts:
#	changedetectionio/blueprint/ui/templates/edit.html
#	changedetectionio/translations/messages.pot
2026-09-18 15:22:24 +02:00
dgtlmoon 438419b3ab Merge branch 'master' of github.com:dgtlmoon/changedetection.io 2026-09-18 15:02:10 +02:00
dgtlmoon d436405da7 UI - sidebar - background fixes 2026-09-18 14:59:59 +02:00
dependabot[bot] 21a1d18f3e Update chardet requirement from >2.3.0 to >=7.6.0 (#4455)
Updates the requirements on [chardet](https://github.com/chardet/chardet) to permit the latest version.
- [Release notes](https://github.com/chardet/chardet/releases)
- [Changelog](https://github.com/chardet/chardet/blob/main/docs/changelog.rst)
- [Commits](https://github.com/chardet/chardet/compare/3.0.0...7.6.0)

---
updated-dependencies:
- dependency-name: chardet
  dependency-version: 7.6.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-18 13:56:25 +02:00
dgtlmoon bb37ea23d2 System - Ensure import+API update values are verified types (#4458) 2026-09-18 13:55:50 +02:00
dgtlmoon a2bc2bf9ae UI - Horizontal lines and styling fixes for #4424
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-18 12:39:57 +02:00
dgtlmoon 3f864cad5b UI - Sidebar spacing/colour fixes 2026-09-18 12:00:16 +02:00
dgtlmoon d3e1c17092 UI - Styling - Remove background on sidebar, more contrast is easier on the eyes 2026-09-18 11:48:41 +02:00
dgtlmoon 77e80d39f1 API - Rebuild docs 2026-09-18 11:42:51 +02:00
dgtlmoon 86d5800a00 UI - Set language on reverse proxy/subdirectory hosting was acting wrong, just redirect it to watchlist so theres no failures. (#4453)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-17 22:35:29 +02:00
dgtlmoon 72d3c171d4 UI - CSS - Mobile misc fixes 2026-09-17 22:24:25 +02:00
dgtlmoon e38e02aa6b Update stock-not-in-stock.js - spanish
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-17 20:15:37 +02:00
dgtlmoon 077ae823e6 Docs - Adding price graph/restock info 2026-09-17 17:48:09 +02:00
dgtlmoon 5e5e0d8f36 Adding HTTP browser caching of UI elements (#4450) 2026-09-17 16:16:42 +02:00
dgtlmoon 593e9cc48c 0.60.7
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-17 11:22:29 +02:00
Dennis Gaidaanddgtlmoon b91082b38a Set Cache-Control: no-store on dynamic responses by default (#4319)
* Set Cache-Control: no-store on dynamic responses by default

Pages with per-session content (settings, CSRF-token-bearing forms,
watch data) had no Cache-Control header, so a misconfigured CDN or
reverse proxy sitting in front of the app could cache and replay them
across requests/sessions — most commonly surfacing as "CSRF tokens do
not match" after the edge served a stale cached page. Routes that
already set their own Cache-Control (static assets, screenshots,
favicons) are left untouched.

* Adding tests and moving function

---------

Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>
2026-09-17 11:20:31 +02:00
dgtlmoon 36ca14df82 UI - Make the 'share watch' optional, reduces visual noise, not really used by non-power users (#4448) 2026-09-17 10:54:43 +02:00
dgtlmoon 12b86f19b5 UI - CSS - Header menu item color fix 2026-09-17 10:39:26 +02:00
dgtlmoon 9c4c95dd44 UI - Watch list - Move processor/mode to its own column 2026-09-17 09:18:39 +02:00
dgtlmoon d3e33f14eb UI - Watchlist - Better alignment and icons 2026-09-17 09:10:30 +02:00
dgtlmoon 706dcb646c UI - Diff page - tabs should be consistent size with rest of app 2026-09-17 08:34:53 +02:00
dgtlmoon f2e7b88cd3 UI - Watch list - Tidy up padding between cells 2026-09-17 08:31:32 +02:00
77fb923de6 fix(http): send a single Date header on werkzeug built-in server (#4372)
* fix(http): send a single Date header on werkzeug built-in server

Static resources served via werkzeug send_from_directory/send_file get a
Date header injected into the WSGI response by make_conditional()
(werkzeug/wrappers/response.py:752-757). When the app runs on Werkzeug's
built-in server -- the default path started through
socketio.run(..., allow_unsafe_werkzeug=True) in changedetectionio/__init__.py:694
and used by the docker entrypoint -- BaseHTTPRequestHandler.send_response()
(werkzeug/serving.py:271) emits its own Date header line as well, so the
wire response carries two Date headers. RFC 9110 forbids this and nginx
rejects the response with "upstream sent duplicate header line" (issue
#4299, see also #4101).

Fix: a global after_request hook pops the application-side Date copy so
only the server's single header reaches the wire. Verified safe on
gunicorn too, which also emits its own Date header.

Test: new tests/test_duplicate_date_header.py hits the live_server over
real HTTP with http.client (the Flask test client talks to the WSGI app
directly and never sees the server-added header) and asserts the Date
header appears exactly once, on the exact static resources named in the
issue. Fails on unfixed code with two identical Date lines; passes with
the fix.

Fixes #4299

* Apply suggestion from @dgtlmoon

* Tidy the #4299 Date header fix and its test

flask_app.py: the applied suggestion landed with a 3-space indent and
trailing whitespace - the latter was the only W291 in the file, which
.ruff.toml selects.

test_duplicate_date_header.py:
- drop the 10s socket wait loop, pytest-flask's live_server already
  blocks until the port accepts connections
- drop the unused `app` fixture argument (live_server depends on it)
- stop hardcoding jquery-3.6.0.min.js: asserting 200 on a vendored
  filename turns a jQuery bump into a failure in a file about HTTP
  headers. Any send_from_directory() response exercises the same path,
  so styles.css alone is enough.

Still red before the fix (two identical Date lines) and green after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: dgtlmoon <leigh@morresi.net>
Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 07:16:00 +02:00
dgtlmoon 3d51a04a19 "Add watch" UI tweaks (#4446) 2026-09-17 06:37:51 +02:00
dgtlmoon 5af6bba8c5 UI - Giving checkbox operations way more contrast
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-16 16:26:20 +02:00
dgtlmoon 3c37de6bb7 General licence cleanups (#4445) 2026-09-16 16:20:46 +02:00
a5f85da4af UI - Keep history diff title and controls visible while scrolling (#4412)
* Keep history diff title and controls visible while scrolling

The watch label, version selectors, diff options and tabs move into a
sticky #diff-header, and the top menu above it (watch URL, EDIT, theme
and GitHub links) becomes sticky too, so the whole header stays on screen
while the diff scrolls. The minimap and anchor jumps offset by the
measured height of both bars instead of a fixed value. Heights are
measured on load and kept current with a ResizeObserver where it is
available, falling back to the window resize and hashchange events as the
restock graph and queue sparkline already do, plus one deferred measure at
load - the tab switch hides #settings and so resizes the header without a
window resize, and opening a link already at #screenshot does the same
with no hashchange to follow. The app header is stored before the diff
header is measured, since the latter's cap is derived from the former.

A sticky bar must be opaque to the content scrolling beneath it, but the
page's own backdrop is a fixed full-viewport gradient layer over the page
colour, so a flat --color-background panel reads as a white band cutting
across it. Instead each bar repaints that same backdrop via a shared
page-surface-gradient mixin: the three gradient stops pre-composited
against --color-background-page with color-mix() at the layer's 0.91
opacity, with background-attachment: fixed so they stay registered with
body::after. Children of #diff-header are centred with a flex column,
mirroring the align-items: center that section.content applies to the
non-sticky siblings, so the controls panel and tabs keep their intrinsic
width and standard colours.

The top menu is made sticky only under body.difference-page. Upstream
already wants this globally (see the @todo in parts/_top_menu.scss) but
held off because the bar has no background of its own; scoping it here
keeps that decision separate. Sticky makes .header a stacking context, so
its mobile drawer is capped at the bar's z-index of 30 - high enough to
cover #diff-header, low enough that the action rail's hover flyout and
toast notifications still paint over the bar as before. The activity
strip is fixed on body in the root stacking context, so it would have
painted through the capped drawer; on this page it drops just below the
bar rather than raising the bar past the rail.

Dropping the 40px section.content padding and tightening the title's own
padding closes the gap between the two bars, and #diff-header's
half-viewport cap now excludes the top menu so the sticky stack stays
within the same budget on short viewports. .app-main's 0.55rem gap goes
too on this page: it sits between the two sticky bars, so the diff header
would otherwise start 8.8px below the top menu and slide up to meet it
over the first 8.8px of scroll. That gap cannot survive sticking - it is
outside both bars, so the diff would scroll through it - and the bars are
flush at every scroll position instead.

Co-authored-by: Engineer <engineer@agents.matrixsi.com>
Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Condense the history diff header into a compact toolbar

The sticky bar the previous commit introduced takes 297px of a 1000px
desktop viewport and 422px of a 390x844 phone - at that width its own
50dvh cap is already clipping it, so the shipped page scrolls a header
inside itself before the diff has moved at all. Compact it to three
rows and that becomes 154px and 228px, without hiding any control.

The watch title takes the full width of the bar on its own row above
the controls, centred, 0.9rem, one line with an ellipsis. It truncates
only when it is wider than the whole viewport - a 184-character title
shows in full at 1440px and ellipsizes at 768 and 390 - and the bar
keeps exactly the same height either way, which a two-line clamp would
not. The full text goes in the title attribute so hover reveals what
the ellipsis hides, the same bargain the watch URL above it already
makes.

The From/To labels shrink to chips rather than hiding. display: none
takes a <label> out of the accessibility tree: verified through
Chromium's accessibility tree, the selects report "From" and "To" with
the chips and "" without them. Sighted users need them too, since two
identically formatted datetime selects side by side have only their
order to tell them apart, and on a phone they stack so even that stops
helping. The chips cost horizontal space and no height at all. Dropping
the shipped width: 4rem sizes them to their text, and "From" is wider
than "To", so a min-width floor keeps the stacked selects flush.

The seven diff options move behind a Filters button. The fieldset stays
inline, inside the form, until diff-overview.js swaps it for the
popover, so with scripting off the options are exactly as reachable as
they are today. The panel is position: fixed, not absolute: #diff-header
carries overflow: auto to enforce its 50dvh cap, which clips an
absolutely positioned descendant at the bar's bottom edge. The bar is
sticky with no transform, filter or contain, so it is not a containing
block for fixed and the panel escapes the clip; the script parks it
under the button and keeps it inside the viewport, with aria-expanded,
Escape, outside-click and a reposition on resize.

The keyboard-nav prompt and the two button words are hidden, leaving
the arrows; both links carry the full wording as aria-label and title
so the accessible name survives the CSS. The tabs' vertical padding
halves on this page, 11px off every scroll position, leaving their
horizontal padding and so their widths alone.

Narrow viewports need one more thing, and font-size alone cannot give
it: once the chip takes inline width the select clips its own displayed
value, and two snapshots from the same day differ only by the meridiem,
so they read identically while closed. #settings, #diff-form,
.diff-fieldset and the span are all sized to max-content, which is the
select's text, so the container shrinks by exactly as much as the text
does - headroom measured 0.0px at every size from 0.9rem to 0.7rem. The
chain is pinned to the row first and only then does 0.75rem buy room:
+19.3px on the demo pair and +6.3px against the longest realistic
English value, which still clips at 0.8rem. All of it hangs off the one
700px breakpoint, and every selector in it carries an ID because the
rules it overrides are written with one and a class-only override would
lose silently.

Co-authored-by: Architect <architect@agents.matrixsi.com>
Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Rebuild translation catalog for the new diff header strings

The compact header adds three msgids - Filters, Previous version and
Next version - and the lint-translations job re-runs extract_messages,
update_catalog and compile_catalog and fails on any resulting diff, so
the catalog has to travel with the strings that created it. It failed
on 297bd365 and took the four test-application jobs down with it, since
they depend on it.

Purely mechanical: the extract adds exactly those three entries to
messages.pot and to each of the seventeen catalogs, all with an empty
msgstr, and changes nothing else. The .mo files are unchanged, because
an empty msgstr is not written to the compiled catalog.

POT-Creation-Date is deliberately left at its committed value rather
than restamped. The job filters that line out of the diff it checks, so
restamping buys nothing, and 0.60.4 rewrites the same header lines - a
restamp here would conflict with the release commit over a timestamp.

Kept out of the header commit so the change being reviewed stays the
template, the stylesheet and the script.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Make the sticky diff header full-bleed and pin its cap to svh

Two iOS-only faults in the sticky bar, both reported from a real iPhone and
neither reproducible in Chromium, emulated or not. Both are properties of
43b07d5b rather than of the compact toolbar on top of it.

section.content insets its children by $common-gap so they clear the left
rail, while the top menu above stays flush against it, so the bar left an
8.8px strip of real page background down each side. That strip is invisible
only while the bar's gradient stays aligned with the page's, which is what
background-attachment: fixed buys - and iOS Safari does not honour it. There
each box samples its own origin, so the bar paints the start of the ramp
(#5ad8f7) while the strips paint whatever the document shows at that scroll
offset (#9150bf at depth), and they read as mismatched borders that hold
still while the diff scrolls past. Cancelling the inset removes the thing
that has to stay matched, on every engine, instead of blending across it.

The width has to come from align-self: stretch, not width: 100%.
.content-main centres its children, so an item is fit-content unless it
stretches, and a percentage resolves against the parent without the negative
margins - it would leave the bar 17.6px short. Measured flush against the top
menu at 1440, 768 and 390: 1358.4/1358.4, 768/768, 390/390. Note the desktop
figure is not the 1440px viewport, because the rail takes 81.6px; the
invariant is the top menu's box, not the window's.

The height cap moves from dvh to svh. dvh tracks the dynamic viewport, which
on iOS grows and shrinks as Safari's toolbar collapses and expands during a
scroll: the cap breathed mid-scroll, changing the header's height and shifting
the content below it, and each transition also fired resize and re-ran the
sticky measurement behind it. svh is defined as the toolbar-expanded minimum
and holds still for the life of the page. It is the smaller of the two, which
is the safe side to be wrong on for a cap, and the plain vh line stays as the
fallback for engines without the new units. This one has no Chromium
instrument - there is no dynamic toolbar to simulate - so it is verified on
the device, not in the harness.

No control moves and no height changes: the bar grows outwards by exactly the
margin it takes back, and the sticky stack still measures 153.9px at 1440,
230.7px at 768 and 227.5px at 390, the same as before this commit.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Stop the tab wrap verdict depending on which tab is selected

base.html decides body.wrapped-tabs - the vertical tab stack - by flipping the
tab list to flex-wrap and checking whether any tab lands on a second row. It
re-runs on load, on resize and on every tab switch. Both sides of that
comparison move, so in a band of widths the answer changes per tab: a phone
showed the three tabs in a row in one screenshot and stacked in a video of the
same page.

The container moves because _tabs.scss says, in upstream's own comment, that
.tabs must take the full width of the centred column "so the wrap-detector JS
has the real available space to measure against" - and the two properties that
would do it sit commented out directly beneath it. Inside the compact header,
which centres its children, .tabs is therefore sized to its own content and the
detector measures against a container that follows the thing it is judging.
align-self: stretch here fixes that, scoped to this page; un-commenting the
global lines would change every page in the app and this branch has no business
doing that. The tab row is re-centred on the ul, which is a grid, so the visual
is unchanged.

The content moves because the active tab's label is bold. At 355px the Text and
Extract Data states measure 332.9 and 330.5 against a 339.0 container and stay
on one row, while Current screenshot - the widest label - goes over and wraps to
three. Removing only the bold rule collapses all three states to 330.5 and the
flipping stops, which is what identifies the weight rather than the container as
the second variable. Dropping the bold on this page is the fix: the active tab
already carries its own background-colour and its own text colour from
_tabs.scss, so it loses the third of three signals and nothing else. Reserving
room for the bold instead - a hidden bold copy behind every label - stabilises
it too, but widens the row ~14px in every state, which pushes 360px from one row
into the stack. Measured stable across 320, 344, 350, 355, 358, 360, 375, 390
and 412px, with the one-row layout now holding down to 350px where before it
was unstable from 350 to 358.

iOS put the same band at the phone's own width because its fonts are wider.
Nothing here is engine-specific - the instability is arithmetic, and Chromium
reproduces it once you look at the widths where the numbers cross.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Keep the Filters popover reachable and anchored on a small screen

Two defects in the same placement code, both outside the viewport range the
desktop frames covered.

The panel is position: fixed, because #diff-header carries overflow: auto for
its 50svh cap and would clip an absolutely positioned descendant. Fixed also
means nothing the document does can scroll the panel: whatever hangs off the
bottom edge at placement time stays unreachable. It was parked below the button
at full height regardless, so in a landscape phone viewport - 844x390 - it
spanned y=150.4 to 431.6 and the last three filters sat past the 390px edge.
Replaced, at y=396.6..423.6, could not be clicked at all. place() now measures
the room below the button and caps max-height to it, and the panel scrolls
inside that cap. Room below is always the roomier side, so there is no flip to
decide: the bar's own cap keeps its bottom edge inside 50svh, so below the
button there is always at least half the viewport. The harness asserts that
premise rather than trusting it. The cap is handed to max-height less the
panel's own padding and borders, since the panel is content-box and switching it
to border-box would fold the padding into min-width and narrow it by 24px.

The bar is also its own scroll container, which the code did not account for -
its comment claimed only a resize could move the button out from under the
panel. On a short viewport the 50svh cap engages and #diff-header scrolls
independently: 16px of scroll at 390x390 moved the button 16px while the fixed
panel stayed at y=171, and 30px at 320x480. The popover visibly comes away from
its trigger and can end up over the wrong control. It now re-places on the bar's
scroll, and closes instead once the button has scrolled out of the bar
altogether - reachable below 280x300, where the bar has 120px of travel.

Verified in Chromium at 844x390, 740x360, 390x390, 320x480, 280x300, 390x844
and 1440x1000: the panel stays inside the viewport at every one, the last filter
is clicked for real rather than asserted on by rect, and the gap between button
and panel holds at 4px across the bar's whole scroll range. All seven fail on
the tree without this change - twelve failures, the click timing out on the
three short viewports.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Keep the Filters popover's height budget current with the visual viewport

place() already budgets against the smaller of documentElement.clientHeight and
visualViewport.height, because a fixed element is positioned against the layout
viewport while iOS shrinks the visual one behind its chrome. But the open panel
was re-placed only on window resize and on #diff-header scroll, and neither of
those fires when the visual viewport alone changes - an expanding toolbar or the
on-screen keyboard does exactly that. The cap then stays at the height it was
computed with and the bottom filters go back behind the chrome, which is the
failure the cap was added to prevent.

That half of place() was recorded as unverifiable, on the grounds that Chromium
has no dynamic toolbar and the two viewports are always equal here. They are
equal only at page scale 1. CDP Emulation.setPageScaleFactor is pinch-zoom: it
shrinks the visual viewport against a layout viewport that does not move, and
fires VisualViewport resize - the same split iOS produces. So both halves are
measurable, and the harness now measures them at 390x844@2, 844x390@1.5 and
390x390@1.5:

  * placing while the visual viewport is already the smaller one caps the panel
    to it - 85.6px against a layout-budget 215.6px at 844x390 - which is what
    shows the Math.min binds rather than sitting inert;
  * on the tree without this commit, shrinking the viewport under an open panel
    left the cap untouched and the panel overhung the visible area by 122px at
    both short viewports and 46.8px at 390x844. Three failures, one per
    viewport, and none anywhere else in the harness.

The probe asserts its own instrument before its result: unless the scale really
splits the two heights and the page really receives the resize event, it fails
rather than passes. It also fails if no viewport in the set caps more tightly
than the layout height would have, since then the visual-viewport budget is
untested, and if a viewport's panel already fitted before the shrink, since then
it cannot detect a stale cap.

The re-placed cap lands on the same value as placing fresh at that viewport
(210.4, 85.6, 65.0px), and the panel's width is unchanged by the re-placement.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Stop the sticky bars' backdrop depending on background-attachment: fixed

Reported from an iPhone: on the diff page the sticky bar sweeps cyan to purple
as you scroll while the page behind it holds still, and snaps when you scroll
back up a little. The gutters beside it are correct throughout, so the page's
own backdrop - body::after, a real position: fixed layer - is fine; it is the
bars' copy of it that moves.

page-surface-gradient asked for that copy with background-attachment: fixed,
which makes the viewport the positioning area. iOS Safari does not honour it and
anchors the layer to the document instead, so each bar shows whatever slice of
the ramp its document position lands on. Chromium does honour the property,
which is why every emulated check passed and only a device caught it.

Derive the alignment from geometry instead, and ask for no engine feature at
all. body::after is a viewport-wide, 100vh-tall box pinned to the viewport's
top-left corner. Each bar keeps a constant viewport offset - .header at the top,
#diff-header at --app-header-height, which the ResizeObserver already keeps
current - and both are flush with the viewport's right edge at every width. So
size the copy to the viewport, anchor it to that right edge, and push it up by
the bar's own offset: the bar then paints body::after's own slice by
construction, on any engine.

Three details that are load-bearing:

  * 100vw, not 100%. The two bars are as wide as the viewport only on a narrow
    screen; at 1440 the action rail insets them by 81.6px (and widens further on
    hover), so a percentage would compress the whole ramp into what the rail
    leaves. Nothing moves their right edge, which is why that is the anchor.
  * background-origin: border-box. #diff-header carries 1rem of side padding and
    0.25rem on top, and the default padding-box origin would offset the copy by
    exactly that.
  * longhands, with the solid-colour fallback shorthand first and no shorthand
    after them - a later `background:` line silently resets size, position and
    origin to their initial values.

The Filters popover has no constant viewport offset of its own, so
diff-overview.js hands it the one it just computed, in the same place it sets
top and left.

Measured in the harness against the page itself: shot B hides .app with
visibility: hidden, which leaves body::before/body::after painting alone with
layout and scroll untouched, so "seamless" is a pixel comparison rather than a
judgement. Worst per-channel distance from that backdrop, over 40 samples at
1440x900, 768x1024, 390x844, 844x390 and 320x600, each at three scroll offsets
and with the popover placed twice:

                                     bars   popover
    before, attachment honoured         3         6
    before, attachment denied          24       112
    after,  attachment honoured         3         6
    after,  attachment denied           3         6

"Denied" injects background-attachment: scroll !important - it takes away the
one property iOS ignores. The before/after rows differ only in whether the
engine grants it, and the after rows are identical to each other: the code path
no longer has an opinion about it. That denial is not a full iOS emulation -
WebKit anchors the layer to the document, which also produces the sweep, and
nothing in Chromium reproduces that - so the device pass is still the gate on
the sweep itself.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Give the diff page's screenshot tab the short label the extract page uses

On a phone the three tabs render as a full-width vertical stack. base.html's
wrap detector flips the page into body.wrapped-tabs when the tab row does not
fit its container, and iOS font metrics put this row over the line at 390px -
an iPhone's width. 01420ac7 stopped the verdict flip-flopping between tabs; it
did not give the row room to fit.

The extract page's tab for the same pane is called "Screenshot". It renders on
one line on the same phone at the same width, which is the evidence this change
rests on: the two pages then carry the identical three labels, the diff page's
row asks for no more width than the extract page's, and #diff-header offers it
slightly more room. Measured at 390px:

  diff page, this change     row 275.4px   offered 374.0px
  extract page               row 280.8px   offered 372.4px
  diff page, before          row 330.6px   offered 374.0px

So the phone's own extract-page row settles the diff page, with no iOS font
model in the argument. In Chromium the narrowest viewport that still fits on
one line moves from 347px to 292px.

"Screenshot" is also what translations/README.md asks for - the shortest
suitable wording - and it makes the two sibling pages agree, which they should
have done anyway. The catalogs are regenerated for the moved msgid reference;
the empty msgstrs this exposes are filled in the commit that follows.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Keep the tab row inside the sticky bar when the bar runs out of room

Reported from an iPhone held sideways: the diff page's tabs are not there. The
bar's height budget is max-height: calc(50svh - the top menu) with overflow:
auto, and in landscape its contents do not fit inside that - iOS gives the two
datetime <select>s enough intrinsic width that From and To each take their own
row, so the bar wants title + From + To + Filters + tabs where it has room for
about three of them. What scrolls out of the visible box is the last child: the
tab row, which is the page's navigation, with no scrollbar on a touch screen to
say it is there.

This one does not need the device. Chromium overflows the same bar on its own at
736x414, 667x375, 780x360, 844x330 and 844x300, and at 844x390 with From/To
forced onto their own rows - measured as the tab row's rect falling outside

So spend the budget on the row that can afford to give. #settings is the only
child allowed to shrink and the only one that scrolls; #diff-watch-title and
.tabs are flex: none and are therefore inside the bar at any viewport height.
The shrink needs min-height: 0 on #settings - a flex item's automatic minimum
size is its content, which is exactly what was pushing the tabs out.

diff-overview.js follows the Filters button while the bar scrolls under the
fixed panel, and the element that scrolls is now a descendant rather than the
bar itself. Scroll events do not bubble, so that listener moves to capture; it
then catches #settings and #diff-header's own last-resort scroll alike, and the
close-when-the-button-leaves-the-bar test is unchanged because it already
compares against #diff-header's rect.

Verified: the tab row is inside the bar at all nine viewports above, and with
the pre-change diff.css and diff-overview.js served back over the wire in the
same run, six of the nine fail - the check is not watching an empty room. The
popover drift probe in the harness was retargeted at whatever inside the bar
actually scrolls, and proved red first: with the capture flag removed the panel
drifts 3.0px off its button at 390x390 and 12.0px at 280x300.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Close the seam between the active tab and the diff card

Reported from the device: a gap between the tabs and the content on Text and
Screenshot, and none on Extract Data.

It is #diff-header's own 0.25rem of bottom padding. The tab row is the bar's
last child, and the active tab and #diff-ui below it are both
var(--color-background) - so those 4px paint the page gradient across what is
otherwise one continuous white surface, and the tab stops reading as the front
edge of the card it belongs to. Extract Data looks right because its tabs are
not in a bar at all: the same measurement on that page is 0.

Measured at 1440x900, 768x1024, 390x844, 844x390 and 320x600, on both the Text
and Screenshot tabs: 3.2-4.0px before, and -0.8 to 0.0 after, with the tab and
the card confirmed to be the same computed colour in every frame. The check
fails only on a positive gap: a sub-pixel overlap of white on white has nothing
to show. The top padding stays - nothing sits above the title.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Land at the top of a tab's pane when the switch resizes the page under it

Reported from the device: the Screenshot tab starts scrolled down a bit. The
recording shows it plainly - for about a second after the tap the tip's first
line sits behind the sticky bar, then Safari settles it.

Order of events. The browser performs the fragment jump to #screenshot first.
Only then does this file's own hashchange handler hide #settings, and the
outgoing #text pane stop being :target - which takes the document from ~8000px
to a fraction of a screen. The engine is left holding a scroll offset for a page
that no longer exists, and clamps it to the new maximum rather than re-running
the jump it has already done. On iOS that maximum is never zero: styles.scss
floors the shell at min-height: 100vh, and 100vh there is the toolbar-collapsed
viewport, so with the toolbar showing the document outruns the visual viewport
by the toolbar's own height and the stale offset has somewhere to survive.

So re-run the jump once the layout has stopped moving. scrollIntoView rather
than scrollTo(0, 0): .tab-pane-inner already declares the offset the sticky
stack needs as scroll-margin-top, and this asks for the same alignment the
browser was asked for, against the layout that actually resulted.
setTimeout(0) rather than requestAnimationFrame, because that scroll-margin-top
is written in terms of --diff-header-height and diff-render.js's ResizeObserver
updates it during the rendering update, after animation frame callbacks have
run - and hiding #settings is precisely what changes it.

Honest limit: Chromium is not the engine that gets this wrong. With the
document floored above the viewport it lands 1px off before this change and 0
after, at 390x844 and 390x500 - the right direction, but a pixel is not the line
the phone showed. The mechanism above is read off the recording and the code,
not reproduced here, and the device is what settles it.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Drop the sticky design on a short viewport rather than compensate for it

A phone held sideways is the one place the sticky bar cannot pay for itself.
50svh of a 390px-tall viewport is 195px, the top menu takes 55 of that, and what
is left has to hold From, To, Filters and the tab row - three of those on their
own lines, because iOS gives the two datetime selects enough width that From and
To cannot share one. The previous two commits kept the tab row inside the bar by
shrinking #settings into a scroller, then took the title and some padding back
to widen the margin. Both were right about the budget and neither made the bar
usable: the reporter's answer was still "it's only showing the From line and I
have to scroll", and that is reproducible here - at 780x360 and 844x330 the To
selector, the Filters button and both version arrows are clipped out of

So below 500px of viewport height, give up the budget instead of spending it.
Both bars become ordinary static blocks: no height cap, no inner scroller, title
and every control present at once, and the page scrolls as one document. Neither
bar keeps its pre-composited copy of the page gradient, which is not optional -
that copy is aligned by pushing it up by the bar's own *constant* viewport
offset, and a bar that scrolls has no constant offset, so it would slide out of
phase with body::after's fixed layer. With nothing scrolling underneath, the
real fixed backdrop shows through and is aligned by construction.

The trade, plainly: in landscape and in a short desktop window the controls and
the tab row scroll away with the diff, so switching tabs means scrolling back
up. That is the right side to be wrong on when the alternative is controls you
cannot see. Portrait and the desktop are untouched and still sticky - asserted,
not assumed.

Three things this turned up that the plan did not anticipate, each of which the
change itself breaks and so belongs here:

  * The minimap is sticky inside the diff card at an offset equal to the sticky
    stack's height. With that stack gone it would pin itself 150-227px down the
    viewport under nothing - past halfway on a 390px-tall one. Pinned to the
    viewport instead, which is now free.

  * The Filters panel is position: fixed and followed its button only through a
    capture-phase listener on the bar, because the bar held still and #settings
    moved inside it. Static mode is the mirror image - nothing moves inside the
    bar and the bar itself moves with the document - so the panel drifted by the
    full scroll distance: 200px at 844x390, 926x428 and 1440x480. The same
    follow/close rule now runs from the window's scroll as well, and carries
    both closing tests; each is inert in the mode it was not written for.

  * place() had no flip-above branch, and compact_verify.py carried the reason
    as an assertion: the bar's own 50svh cap kept its bottom edge inside the top
    half of the viewport, so below the button was the roomier side by
    construction. Removing the cap removes the premise, and that assertion is
    what caught it - at 280x300, 147.6px above the button against 104.2px below.
    place() picks the roomier side again.

The load-path scroll fix is the fourth, and it is the one worth reading twice.
upstream's tabs.js rewrites an empty hash to the first tab's, and the browser
performs that jump the moment the diff is built - before diff-render.js has
measured the bar, so scroll-margin-top is still resolving against its
--diff-header-height: 0 fallback. Traced at 844x390: the jump lands at scrollY
231 against a 16px margin at t=132ms, and the margin becomes its real 202px at
t=167ms with the offset left where it was. Under a sticky bar that is invisible,
because the chrome is pinned whatever the offset is; with the bar in flow the
same 231px put every control above the top of the page and a landscape phone
opened on a bare diff. So the alignment re-runs when the quantity it was wrong
about settles - the bar's own size - guarded on the reader still being where the
last alignment left them, so that rotating the phone mid-read cannot throw
someone back to the top of the pane. Portrait benefits too: the page used to
open 224px into the text diff and now opens at 1.

.tab-pane-inner's scroll-margin-top is deliberately NOT dropped to 1rem here.
It reads as an allowance for pinned chrome and therefore like dead weight once
nothing is pinned, but the same number is right in both modes for two different
reasons: sticky, it clears chrome painted over the pane; static, it keeps chrome
that sits above the pane on screen. The comment says so, next to the measurement.

20d75707, which hid the watch title in landscape to buy budget room, is rebased
out rather than left in and undone - there is no budget to buy any more, and the
question of whether hiding it was acceptable goes away with it.

Verified in four browser runs that vary the two changed files independently,
because the popover check needs the new stylesheet with the old script: serving
both old puts the page back in sticky mode, where it passes for the wrong
reason. Every run reads the served diff.css and diff-overview.js back over the
wire, hashes them, and fails if they are not the pair it meant to serve. On the
branch: PASS, no failures. Reachability went red at 10 of 14 short viewports on
the old pair, the gradient copy at 7, the minimap at 7, the load landing at 5,
and the popover drift at 3 on the new-CSS/old-JS pair. The full branch harness
and the previous round's probe both re-run green.

One instrument repair, and it had been green while measuring nothing. The
popover drift check scrolls whatever inside the bar can move and re-reads the
gap - but page.click() scrolls the toggle into view to click it, which leaves
that scroller already parked at its maximum, so "scroll to the maximum" was a
no-op and the check compared two identical readings. It now drives the scroller
to whichever end it is not at, and proves the point by comparing the two
readings rather than by testing for non-zero. Re-proved red on the real code:
with the capture flag removed the panel drifts 45px off its button. Nothing
inside the bar overflows in this engine at any viewport any more - that was
always iOS's wider metrics - so the condition is forced with an explicit
max-height rather than hunted for.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Give the tab-to-card seam a whole pixel of overlap instead of none

The thin blue line under the active tab is the page gradient, not a border -
nothing at that boundary has one, the active tab and #diff-ui are both
var(--color-background). It can only be a device-pixel row where neither white
paints, and whether such a row exists comes down to a single number: the bar's
bottom edge minus the card's top edge.

Measured across 14 viewports at device pixel ratios 1, 2 and 3, on both the Text
and Screenshot tabs, that number is 0.813px, 0.203px or exactly 0.000 - 84
readings, not one of them a whole pixel, and 0.000 at 390x844 on the Screenshot
tab, which is the exact case reported. At zero the outcome is not a decision the
stylesheet makes; it belongs to whatever the engine does with two edges on the
same coordinate, which is why a DPR 3 phone can show a line a DPR 1 desktop does
not. Chromium rounds the same way on both sides and paints no sliver at DPR 3
either - walking the bar's height in thirds of a pixel did not produce one. So
this is the margin being fixed, not a pixel being chased: after the change the
worst of the same 84 readings is exactly 1.000px.

The alternative - a white skirt under the active tab, box-shadow: 0 1px 0 - would
have been more precisely scoped and does not work. The shadow hangs below the
bar's last child and #diff-header's overflow: auto, which is what enforces its
height cap, clips it. Measured with the card held 3px away so the skirt and the
card are separable: 0 device rows of the 3 it needs at 844x390 and 926x428, and
0 to 2 in the sticky modes. The negative margin is what survives the clip,
because it moves the card rather than painting past an edge.

Scoped to body.difference-page, which matters more than it looks. diff.css is
not diff-page-only: preview.py and processors/extract.py serve it too, both
pages have a #diff-ui of their own, and neither has a bar above it to absorb the
pixel. Asserted rather than reasoned - the preview page's computed margin-top is
0px with this stylesheet and with the previous one, while the diff page's moves
to -1px. The comment on the activity-strip rule above claimed diff.css was
loaded only by the diff page; that was wrong and is corrected, since the scope
it describes is what actually does the work.

What the overlap costs, stated rather than left to be found: where the bar is
sticky, nothing. A device-pixel diff of the seam band at 390x844 and DPR 3 comes
back with zero differing pixels, because the bar is opaque and paints over the
card. Below the short-viewport breakpoint the bar has no background of its own,
so the card's white also reaches the bottom pixel of the inactive tabs and the
5px gaps between them - 4374 device pixels at 844x390, gradient turning white,
and side-by-side crops of the two are indistinguishable. The tab row ends up
sitting flush on the card instead of floating a pixel above it, which is how
tabs normally meet their panel and is the same continuity the active tab was
asked for.

Full branch harness, the previous round's probe and this round's all re-run
green against the change.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Reuse upstream's Previous and Next for the version arrows' labels

297bd365 gave the version arrows `title` and `aria-label` of their own,
"Previous version" and "Next version". Those are two msgids this branch
introduced and no catalog has ever carried, so on a German or Japanese page
the arrows' accessible name is English. The upstream msgids `Previous` and
`Next` say the same thing in this context - the arrows live inside the
version bar, between the From and To selectors - and are non-empty in 15 of
17 catalogs; the two empty ones are en_GB and en_US, where the English msgid
is the correct rendering anyway.

The visible span on each arrow already uses these msgids, so the accessible
name and the (header-hidden) label now agree instead of differing by a word.
The cost is that the accessible name drops "version": a screen reader says
"Previous" rather than "Previous version". On a control that sits between two
datetime selectors that is enough, and it is the price of not asking fifteen
translators for a string they can already spell.

The catalogs are regenerated: both msgids disappear from the .pot and from
all 17 .po files. No .mo file changes, because neither msgid had a msgstr
anywhere to compile.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Correct the French translation of "Previous"

fr renders the msgid `Previous` as "Aperçu", which is French for *preview*.
That is a wrong word, not a style choice: the same catalog gives "Aperçu" as
the msgstr for `Preview` and for `Activate preview`, and it evidently leaked
from there.

The msgid renders in exactly two places, both of them prev/next pairs - this
page's version arrows and preview.html's own pager - and fr already translates
`Next` as "Suivant", so "Précédent"/"Suivant" is the pair the catalog is
otherwise using. On the preview page the current string produces an "Aperçu"
button on the page called Aperçu.

Left alone deliberately: de renders `Previous` as "Zurück" (*back*). That is
defensible on a back arrow and is a translator's call, not an error.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Put the watch's name on the top line and drop the diff page's second title row

The diff page printed the same thing twice. The menu's top line has always
shown the watch URL, and 297bd365 added a sticky <h1> under it showing
watch.label - which for any watch without a custom title *is* that URL. Two
rows of chrome, 30px of the sticky budget, and on the common case the identical
string on both.

dgtlmoon's read (#4412): put it all on the top line, use the title if it is
set, and take back the left inset while we are there. That is what this does,
and it removes code rather than adding any.

The top line's <a> is unchanged as a link - same href, same fade-out mask, same
flex-shrink that keeps a long value off the menu icons. Only its text changes:
current_diff_label (watch.label - title, then the fetched page_title, then the
URL) with the URL as the fallback, so an untitled watch reads exactly as it
does today. The five views that already pass current_diff_url pass the label
beside it.

The hover text now carries both, label then URL on its own line, and only when
they differ. A title covers the URL that used to be printed here, and the URL
is also where the link goes - so it has to stay readable without following it.
That also keeps the long-title-on-long-press property the <h1> had.

The heart hides under $desktop-wide-breakpoint on this page only. That is the
width where the hamburger appears and the row is at its tightest, so it is
where the name needs the space; on a desktop diff page and on every other page
the heart is untouched. It is a media query rather than a {% if %} because the
heart's reason to exist is that it is always there - folding it away with the
rest of the narrow row is a different thing from taking it off the page.

Left-justification: the line's text started $common-gap twice in - the menu
row's own padding plus the link's own left margin. On the row's first item the
second one only repeats the first, so it goes and the row's padding stays.
8.8px back, and the line still shares a left edge with the diff below it rather
than sitting hard against the viewport. The right side of both is untouched.

No new or removed translatable strings, so no catalog churn.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Fill the empty msgstr cells this page's own strings left behind

30 cells across 15 catalogs. No new msgids: extract_messages produces a
byte-identical .pot (bar POT-Creation-Date) and update_catalog leaves every .po
unchanged, which is the control that nothing here is a source change wearing a
translation's clothes.

Filters - empty in all 15 non-English catalogs since the diff page's options
popover was added. Each value is the plural form that catalog already uses for
"Global Filters" and "Filters & Triggers", not a fresh translation: Filtry,
Filter, Filtros, Filtres, Filter, Filtri, フィルタ, 필터, Filtry, Filtros,
Фильтры, Filtreler, Фільтри, 过滤器, 過濾器.

Screenshot - this is a regression the diff-header work introduced and owes.
The tab used to read "Current screenshot", which every catalog translates;
shortening it to "Screenshot" moved the tab onto a msgid that nine of them
(cs, de, es, fr, it, pt_BR, tr, uk, zh) had never filled, so those users got an
English tab where they had a translated one. Each value is that catalog's own
"Current screenshot" with the qualifier dropped - Snímek obrazovky, Captura de
pantalla, Capture d'écran, Ekran görüntüsü, Скріншот, 截图 - and for de, it and
pt_BR the word those catalogs already use is literally "Screenshot"
(Aktueller Screenshot / Screenshot corrente / Screenshot atual), which is what
dennis W302 flags and CI excludes.

Italian's five pre-existing gaps on this page, deliberately in scope: the
guide's multi-language rule is to fix every affected catalog in one session
rather than fix one language and move on, and an Italian diff page was showing
untranslated From, To, Ignore Whitespace, Keyboard:, Extract Data and Jump to
next difference next to strings that were translated. Ignora spazi and Estrai
dati come from the same catalog's "Ignore whitespace" and "Extract text" /
"Extract as CSV"; successiva from its "Next" -> "Successivo" and differenza
from its "Pixel Difference Sensitivity".

Compiled with setup.py compile_catalog. en_GB and en_US recompile byte-identical,
so the 15 changed .mo files are exactly the 15 changed .po files.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Let the watch title be dragged when it is too long to fit

Asked for on the PR: "does the title at the top accept overflow scroll hidden so
it can be scrolled with your finger/touch/drag?" It could not - .current-diff-url
was overflow: hidden with a fixed right-edge fade, so the end of a long title was
reachable only by hovering for the title attribute, which a phone does not have.

It is now a horizontal scroll container with its scrollbar hidden in both engines.
A tap still follows the link; browsers distinguish a pan from a click, so the
anchor keeps its navigation. overflow-y stays hidden because the content is one
non-wrapping line and an axis that can move by a rounding error lets a diagonal
swipe jog the text.

The fade has to follow the scroll, or the ending someone just dragged over to read
is the part that is dimmed. static/js/scrollable-title.js maintains fade-left /
fade-right against scrollLeft and adds js-fades to say it is doing so; under that
class the CSS drops the static mask and paints only the edge that is actually cut
off, or a two-ended gradient when both are, or nothing at all when the title fits.
The static right-hand fade stays as the no-JS default, which is correct there
because without a handler the line can only ever be cut on the right.

In base.html rather than diff-overview.js: restock_diff/difference.py and
image_ssim_diff/preview.py both render this line and neither loads that file. The
script guards on the element existing, so it is inert on every other page.

Accepted tradeoff: with the anchor focused, ArrowLeft/Right still runs version
navigation on diff pages via the existing window keydown handler rather than
scrolling the line. Keyboard users keep the hover title attribute, as today.

styles.css is spliced, not rebuilt. parts/_top_menu.scss compiles into styles.css,
whose committed bytes do not match a fresh compile of its own scss - an older sass
folded rgba(255,255,255,X) to hsla(0,0%,100%,X) and hoisted nested declarations in
a different order (#heartpath, .button-tag, .tab-pane-inner, .watch-table
img.favicon, #browser_steps li and more). That drift is upstream's and predates
this branch. So rather than commit a full rebuild and carry all of it, the scss was
compiled twice - at the parent commit and with this change - and only the span that
differs between those two compiles was transplanted into the committed file. The
drift is identical in both compiles and cancels. The result differs from the parent
by 795 bytes, all inside the .current-diff-url rules, and is byte-identical
everywhere else; diff.css is untouched. A reviewer who runs npm run build will see a
dirty styles.css - that mismatch is the pre-existing drift, not this commit.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Cut diff-overview.js back from a slab to a file

Raised on the PR as "a slab of javascript... why cant this be a few lines of
jquery". Measured before arguing: upstream's file is 190 lines (144 code, 20
comment) and this branch had grown it to 476 (284 code, 142 comment). Half the
growth was comment, written at review density rather than this project's, and that
is what makes it read huge. 476 -> 370 (256 code, 72 comment).

Every long forensic comment is cut to the constraint it was protecting - the
measurement that motivated it survives as a clause, the war story does not. Nothing
load-bearing was deleted: the capture-phase scroll listener still says why capture,
the visual-viewport listener still says why it exists separately from resize, and
place() still says why offsetHeight is read after the cap.

Converted to jQuery idiom where that is genuinely shorter, so the file reads in one
voice: element lookup, class toggling, open/close, and the event bindings that need
no listener options. The geometry stays native, because there is no jQuery shorthand
for it - getBoundingClientRect, getComputedStyle, offsetHeight, visualViewport and
ResizeObserver are the added code, and popover placement is what they are for.

Two registrations deliberately stay on addEventListener and say so in place: the
bar's scroll listener needs capture (scroll does not bubble and the element that
scrolls is a descendant) and both scroll listeners need passive, neither of which
jQuery's .on() can pass.

The one structural change is that the four copies of "step both selects one
position and rebuild the query string" collapse into diffStepHref(direction),
called by the buttons at setup and by the arrow keys at press time. The guards are
unchanged: the buttons are only touched when both selects have a selection, the
arrow handler still requires both a live step and a button href, and it still
returns early while an input, textarea or select has focus.

Not done here, and offered instead as a follow-up: moving the popover to the native
Popover API. It would delete the dismissal wiring, but placement stays manual
without CSS anchor positioning, which is not yet safe cross-browser - and it would
churn code that is tested and green.

No new translatable strings, so the catalogs are untouched.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

* Point the top-menu comment at the rule it describes

The previous commit inserted the fade variable and mixin between this block and
.current-diff-url, which left the comment describing the declaration above it
rather than the rule it was written for. It also still said the line "trims" its
text, which is now only half true - it scrolls, and the trim is what the fade
stands in for.

Comment only: sass emits byte-identical css, so styles.css is unchanged.

Co-authored-by: Jeff Hedlund <jhedlund@gmail.com>
Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>

---------

Signed-off-by: Jeff Hedlund <jhedlund@gmail.com>
Co-authored-by: Surveyor <surveyor@agents.matrixsi.com>
Co-authored-by: Engineer <engineer@agents.matrixsi.com>
Co-authored-by: Architect <architect@agents.matrixsi.com>
Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>
2026-09-16 14:19:00 +02:00
dgtlmoonandClaude Opus 5 b61b83c35d Fetch favicon candidates in parallel under one deadline, and stop the two ways it could hang (#4435)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
Each candidate icon got its own fresh 2s AbortController, so the cost was n x 2s: a site
declaring five <link rel="icon"> variants spent 10s+ in here, sequentially, inside the
page - holding a browser, a worker and a proxy connection the whole time - to fetch
decoration.

Capping the total is not enough, and is a trap. Giving up early returns no icon, so
nothing is saved, favicon_is_expired() stays true, and the same cost is paid again on the
very next check - forever. Measured against a page with five hanging icons, one 404 and
one good one:

  sequential, 2s each : 10.0s, found the icon
  total cap only      :  3.0s, found NOTHING (and repeats every check)
  parallel + deadline :  3.0s, found the icon

So: fetch them all concurrently under one shared 3s AbortController and take the first
success in preference order - the array is already sorted largest-first then
apple-touch-icon, so this still returns the preferred icon rather than merely the
quickest to answer.

Two hangs fixed while in here:

- clearTimeout() fired before `await resp.blob()`, leaving the body read unguarded. The
  shared signal now covers it (aborting a signal errors the body stream too), so a slow
  or never-ending body is bounded like the headers.

- the FileReader promise had no reject path and resolved only from onloadend, reading
  reader.result unguarded. A FileReader failure threw inside the callback and left the
  promise permanently pending, with the timer already cleared - the favicon fetch then
  hung forever with nothing to stop it. It now always resolves.

Also skips an oversized icon from Content-Length before pulling its body down the wire,
where the server declares it.

Unchanged: the candidate collection and sort, the data: URI shortcut, the 1MB limit that
matches bump_favicon(). Verified no regression on the ordinary paths - a page with one
working icon still returns it in 0.0s, and a page with no <link> still falls back to
/favicon.ico.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 09:43:43 +02:00
dgtlmoon 04b64fa15b 0.60.6
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-14 20:34:03 +02:00
dgtlmoonandClaude Opus 5 7102544310 xPath filters - Stop LC_COLLATE deciding what contains() means (#4437) (#4438)
Every watch whose xPath filter used contains() started reporting "Warning, no filters were
found" on pages whose HTML plainly contained the target. 18 unrelated watches broke in the same
hour, browser and plain-requests fetchers alike, and the saved snapshot was perfect every time.

elementpath implements the XPath string functions on top of locale.strxfrm:

    def contains(self, a, b):  return self.strxfrm(b) in self.strxfrm(a)

Under LC_COLLATE=C, strxfrm() is the identity function and that is an ordinary substring test.
Under a real locale it returns a binary collation key, and a substring of a collation key is not
the collation key of the substring - so contains(), starts-with(), ends-with() and
substring-before/after() return false for EVERY input. Reduced to one line, no document needed:

    LC_COLLATE=C            contains("xx month xx", "month") -> True
    LC_COLLATE=en_US.UTF-8  contains("xx month xx", "month") -> False

Name tests, axes and '=' are untouched, which is exactly why it read as "did the page change
layout?" - //div kept working while //div[contains(.,"month")] returned nothing.

No code change caused this. Generating the image's locales (#4429) made ENV LC_ALL=en_US.UTF-8
satisfiable for the first time; flask_app's setlocale(LC_ALL, ...) had been raising locale.Error
and leaving us in C, and once it succeeded it took LC_COLLATE with it. That is why the bug
survived a bisect to 0.60.2 and why reinstalling the exact pip set from a working install did not
shift it - it could only be found by diffing the two containers. Same image base, same Python
3.11.16, lxml 6.1.3, libxml2 2.14.6, elementpath 5.1.1, same HTML:

    good-old 0.60.4   setlocale FAILED: unsupported locale setting   67 matches
    bad-new  0.60.5   setlocale en_US.UTF-8                           0 matches

Fixed in both places, because either alone leaves a hole:

 - flask_app sets LC_CTYPE/LC_NUMERIC/LC_MONETARY/LC_TIME individually instead of LC_ALL. This
   block exists to make prices render correctly and it still does - 1234567 is still "1,234,567"
   - it just no longer touches collation.

 - html_tools.xpath_filter() pins the Unicode codepoint collation per evaluation, so a filter
   means the same thing whatever an operator puts in LANG/LC_ALL, and does not depend on a
   distant module's locale bookkeeping. forms.py's XPath validation pins it too, so validation
   cannot accept an expression that then behaves differently at check time.

Per XPath 3.1 the default collation is codepoint and must not consult LC_COLLATE, so the
underlying behaviour is arguably an elementpath bug; the pin above holds regardless.

Verified end to end inside the failing container: LC_COLLATE=C, LC_NUMERIC=en_US.UTF-8,
thousands separators intact, and the reported filter back from 0 to 1190797 chars of output.

Tested: new unit test covers contains/starts-with/ends-with under a UTF-8 collation and was
checked to fail without the fix; 325 unit tests pass.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 20:33:35 +02:00
dgtlmoonandClaude Opus 5 d56ec682fe Puppeteer fetcher - Re-navigation cap must not mean "extract right now" (#4439)
BROWSER_CONTENT_READY_MAX_RESETS exists so a page that re-navigates in a loop cannot extend a
fetch forever. On hitting that cap the content-ready wait broke straight out into
Page.stopLoading and extraction, giving the document we end up on zero settle time - the exact
opposite of what the wait is for.

Measured against a page that hops every 500ms and then renders via JS 2s after the final load:

  before:   2.7s, 130 bytes of an intermediate hop, no final document, no JS-rendered content
  after:   14.1s, final document, JS-rendered content present

It also logged "Content-ready wait of 12s elapsed" immediately before extracting, having waited
0s, which is why this reads as a fetcher that ignores the setting.

Note 0.60.4 could not do this: its wait was an unconditional `await asyncio.sleep(1 + extra_wait)`
after goto(), so every fetch got its settle time no matter how the page behaved.

Now the cap stops the wait from being *restarted*, and the delay is spent one final time before
extracting. Total stays bounded at (max_resets + 2) * extra_wait, and whatever we extract has had
the same settle time every other fetch gets. The stopLoading log line no longer claims a wait
that may not have happened.

Unrelated to #4437 - found while reading #4426 for that investigation, which turned out to be a
locale/collation bug in the filter layer, not a fetcher problem.

Tested: new reset-cap probe checked to fail before and pass after; normal (non-re-navigating)
path unchanged at 12.5s with content intact; 3 passed browser fetcher suite on pyppeteer, 323
unit tests pass.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 20:31:48 +02:00
dgtlmoon b0783bce90 0.60.5
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-14 14:49:55 +02:00
dgtlmoon c9b91e593b Favicon Setting - Adding a note that disabling this feature can speed up your page fetches/use less resources. 2026-09-14 13:12:25 +02:00
dgtlmoonandClaude Opus 5 7a29b5bc73 Page fetching - Terminate runaway page script before extracting, or a spinning renderer eats the whole fetch (#4433)
Page.stopLoading stops the network, not script execution. A page whose JavaScript has
pegged the renderer's main thread keeps that thread indefinitely, and every CDP call that
needs to run script then queues behind it and never returns - page.content, the xPath
scraper, the favicon fetcher. The fetch dies at PUPPETEER_MAX_PROCESSING_TIMEOUT_SECONDS
having extracted nothing, with a core spinning the entire time.

Seen in production on a watch that failed every check for days: the renderer sat at
1.04-1.07 cores for the full 60s budget (sampled every 2s, flat), 33s of which was a
single unanswered Runtime.evaluate, and the watch logged "xpath_data length returned
empty" every time.

Nothing else recovers this state. Runtime.evaluate's own `timeout` parameter bounds an
evaluation once it starts, not time spent queued behind the running task - measured, it
still hung past 15s. Wrapping the call in asyncio.wait_for is worse than useless:
cancelling a pyppeteer request mid-flight leaves the connection unusable, with
"Protocol error: Target closed" on everything after it.

Runtime.terminateExecution is what releases the thread. Against a page that fires load
and then spins forever, through the real fetcher:

  before:  60.6s, BrowserFetchTimedOut, 0 bytes content, no xpath_data, no screenshot
  after:    5.6s, no exception, content + payload, xpath_data present, screenshot 8415b

and the renderer drops from 1.00 to 0.08 cores.

Safe at this point in the fetch: stopLoading has already declared "give me what
rendered", and the content-ready wait above has already had its chance to let late
JS-rendered content appear.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 12:40:53 +02:00
dgtlmoon 943cf9c60a Favicon on/off - Was not actually applying at fetch time (#4432) 2026-09-14 12:33:18 +02:00
滅üanddgtlmoon ba0ef4450e i18n: Update zh_Hant_TW translations (#4430)
* i18n: Update zh_Hant_TW translations

* Rebuild

---------

Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>
2026-09-14 11:32:45 +02:00
dgtlmoonandClaude Opus 5 06446fa26b Debounce the explicit gc.collect() storm on the watch-check path (#4431)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
Several places call gc.collect() after a check to keep C-level memory
(pyppeteer buffers, libxml2 documents, PIL, brotli) from accumulating.
Individually each is reasonable; run concurrently by many fetch workers they
become a storm. Every gc.collect() is a full stop-the-world pass that walks the
whole heap holding the GIL, so at FETCH_WORKERS=50 the process spends most of
its time stopped in the collector - which also starves each worker's asyncio
loop, leaving its CDP websocket unread.

Measured on 153 real puppeteer checks of a live site at FETCH_WORKERS=50, with
an `//div` include filter so the lxml document tree is realistic:

                     collects  objects freed  gc time  checks/sec  CPU/check     RSS
  one per call site       790      2,594,098    91.3s       0.766     1.373s  279.6MB
  debounced to 1s          48      2,219,010     7.3s       1.433     0.731s  275.3MB
  none at all               0              0     0.0s       1.503     0.685s  287.7MB

Debouncing keeps 86% of the reclamation for 6% of the collections: 1.9x the
throughput, half the CPU per check, gc down from 30.8% to 6.4% of wall time,
and a lower resident plateau than collecting every time. It works because the
collector is process-wide - any worker's collection breaks every other worker's
cycles too, so with many workers the calls are overwhelmingly redundant
duplicates rather than independently necessary.

Removing them entirely is slightly faster still, but it was the only
configuration whose RSS had not plateaued by the end of the run, so it is not
the default. EXPLICIT_GC_MIN_INTERVAL=0 restores the previous behaviour.

Collecting a younger generation was measured and rejected: gen 0 freed 1,136
objects against the full pass's 2,594,098, because objects surviving a 10-30s
fetch have already been promoted out of gen 0.

Two call sites are additionally fixed because they could never reclaim anything:

- Watch.py brotli: brotli.Compressor is not gc-tracked, so the collector cannot
  see it - `del` frees it by refcount. Over 60 x 2.2MB compressions, RSS growth
  was +0.9MB with neither mechanism, +0.2MB with gc.collect() alone, and +0.0MB
  with malloc_trim() alone or with both. malloc_trim is the load-bearing line
  and is kept; the collect cost ~31ms of stop-the-world per snapshot save for no
  reclamation.

- puppeteer quit(): runs twice per check (run()'s finally, then the worker's
  safety net) and nulls self.page/self.browser in its own finally blocks, so the
  second call closes nothing and breaks no cycles yet still paid for a full
  collection - 88 calls across 51 checks. Now only collects when it actually
  closed something.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 00:11:47 +02:00
dgtlmoonandClaude Opus 5 16b19a4ab7 Dockerfile - actually generate locales, price formatting was falling back to C (#4429)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
The `locales` package was installed "For presenting price amounts correctly in
the restock/price detection overview", but nothing ever ran locale-gen. The
package only ships /etc/locale.gen; it does not build any locale. So published
images had only C, C.utf8 and POSIX available, which made the existing
`ENV LC_ALL=en_US.UTF-8` unsatisfiable:

    $ docker run --rm --entrypoint bash ghcr.io/dgtlmoon/changedetection.io:latest \
        -c 'locale -a'
    C
    C.utf8
    POSIX

locale.setlocale() in flask_app.py therefore raised, was caught by the existing
`except locale.Error` and logged "Unable to set locale ... is not installed
maybe?", and the process stayed on the C locale. The knock-on effect is that the
format_number_locale / format_int_locale Jinja filters lose their thousands
separators, contradicting format_number_locale's own docstring:

    before:  format_number_locale(1234567.89) -> '1234567.89'
    after:   format_number_locale(1234567.89) -> '1,234,567.89'

Those filters render prices in the watchlist overview
(blueprint/watchlist/templates/watch-overview-single-row.html), so every
published release has been showing unseparated price amounts.

How it got this way: LC_ALL arrived on master in d1b1dd70f (#3340) without a
locale-gen to satisfy it. The matching `RUN locale-gen en_US.UTF-8` was written
back in 3f73695e7 (2024-07-22), but that commit only ever existed on the
unmerged 2486-charset-encoding branch, so master has never had it.

Generate one glibc locale per UI translation in changedetectionio/translations
rather than only en_US, so that operators can override LC_ALL / LANG to any
language the UI actually offers and get correct local formatting (de_DE gives
1.234.567,89, fr_FR gives 1 234 567,89). Territories for bare language codes
come from CLDR likely-subtags, not from uppercasing the code, which would have
been wrong for cs, ja, ko, uk and zh.

Costs ~21MB and ~16s of build time.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 12:12:11 +02:00
dgtlmoonandClaude Opus 5 48723a09dd UI - Search modal - fix Enter dismissing the form instead of searching (#4428)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
Pressing Enter in the search box closed the modal without running the search;
you had to click Search with the mouse.

Implicit form submission fires a click at the submit button, and a
keyboard-synthesised click carries detail 0 and coordinates of 0,0. The
backdrop-click handler only tested the coordinates against the dialog's
bounding box, so 0,0 read as "outside" - it closed the dialog and blanked the
input while the click was still bubbling. By the time the submit ran, `q` was
empty and `required` rejected it, so nothing was searched.

Ignore clicks with detail 0 - only a real pointer can hit the backdrop.

Verified with Chromium against a local instance: Enter now lands on
/?q=<term>, and mouse submit, backdrop click, Escape and Enter-on-empty all
still behave.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 08:25:40 +02:00
dgtlmoonandClaude Opus 5 718295f30b UI - Search modal - keep the search scoped to the tag you were viewing (#4425)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
The modal submitted the active tag as `tags=<uuid>`, but the watchlist filters on
`tag` - nothing reads `tags`. So searching from inside a tag view silently
searched every watch, despite the label promising "URL or Title in '<tag>'".

The test pulls the hidden field straight out of the rendered modal and feeds it
back to the watchlist, so the field name can't drift from the arg again.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 17:48:48 +02:00
dgtlmoon 7cc62f8321 Merge branch 'master' of github.com:dgtlmoon/changedetection.io 2026-09-12 17:09:59 +02:00
cd66f9ed54 Search modal native submit (#4427)
* UI - Search - Fix search modal navigating to the host root on sub-path deployments

base_path was referenced by search-modal.js but never defined, so searching
always jumped to the host root instead of the X-Forwarded-Prefix sub-path.

* UI - Search modal - submit natively instead of rebuilding the URL in JS

Alternative to defining a `base_path` JS global: give the search form a
server-rendered `action`, so url_for() supplies the reverse-proxy sub-path the
same way every other link on the page already does, and let the browser submit.

Drops the submit handler and the Enter handler from search-modal.js - Enter in
the input reaches the footer's submit button via implicit submission, which also
runs the `required` validation the synthetic `new Event('submit')` skipped.

The hidden tag field is only rendered when a tag is active, so a plain search no
longer carries an empty value.

Also drops the nginx-job grep for the rendered markup - test_search.py already
covers the sub-path case, and asserting on an exact HTML attribute string from a
shell grep breaks on any unrelated edit to that tag.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: ponstream24 <87808547+ponstream24@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 17:09:10 +02:00
dgtlmoonandClaude Opus 5 8d938b5966 Browser fetchers - Follow re-navigation, and one content-ready deadline (re-targets #4421 + #4422 at master) (#4426)
* Browser fetchers - Judge a fetch on the document we end up extracting, not the first navigation

The goal is to compare the text of the page the browser lands on, even when the site navigates
again after the first response. Both fetchers were bound to the first navigation, which shows up
as two different bugs:

1. pyppeteer hangs until the hard processing timeout. Its navigation watcher is bound to the
   loaderId of the navigation it started, so when the site replaces that document the 'load' it
   waits for never arrives for that loaderId. With timeout=0 and setDefaultNavigationTimeout(0)
   there is nothing to break the wait, so goto() blocks until
   PUPPETEER_MAX_PROCESSING_TIMEOUT_SECONDS (180s) kills the fetch and the watch records an empty
   xpath_data - while the browser is sitting on a fully loaded page. Traced on slated.com:

     0.24s goto start
     0.74s main frame networkIdle    loaderId=A0E0E0B2   <- never gets 'load'
     2.72s main frame init           loaderId=56D2B5EF   <- re-navigated to get.slated.com
     3.49s main frame load           loaderId=56D2B5EF   <- fires for the new document
    25.2s  goto still hanging, frame._loaderId is now 56D2B5EF

   Now the navigation races goto() against the main frame firing 'load', bounded by
   BROWSER_NAVIGATION_TIMEOUT_SECONDS (default 30), and falls back to the document we can see.
   slated.com / getastra.com / addupsolutions.com went from a 180s timeout with no content to
   200 with full content in 5-35s.

2. Both fetchers reported the status of the interstitial. A site that gates unseen visitors with
   an error status plus a client-side redirect (reported against fotokoch.de: 503 + meta refresh,
   then a 200 with the real page) failed the watch even though the content was present, and the
   only workaround was ignore_status_codes, which also hides genuine 404s and 500s forever.
   The fetchers now keep the latest main-frame document response and judge on that - the refresh
   lands during the existing extra_wait, so the 200 wins.

Playwright also waits for a settled load state before extracting, which is what produced
"Execution context was destroyed, most likely because of a navigation" when the refresh collided
with extraction.

The navigation-response tracker is installed once per page and shared between the fetcher and
action_goto_url() rather than each navigation adding its own listener - 'response' fires once per
HTTP response, hundreds of times on a heavy page, so the callbacks are worth not duplicating.
Verified one listener remains after an install plus four navigations.

Selenium is unaffected either way - it hardcodes status_code = 200 because WebDriver cannot see
the HTTP status.

Tested: new test_renavigation.py covers the interstitial case end to end and was checked to fail
without the fix and pass with it, on both fetchers. The test endpoint gates on last-seen time
rather than a hit count, because a counter lets the second check see a clean 200 and the test
then passes without the fix. Full browser suite 11 passed on playwright and on pyppeteer, 494
unit tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Puppeteer fetcher - One content-ready deadline instead of a stopLoading watchdog per frame event

Page.stopLoading is what stops a page that would otherwise load forever waiting on a subresource
that never answers, so that we can still screenshot and scrape what rendered. That intent was
right, but it was implemented as a fire-and-forget task armed by every frame event, which measured
on a single fetch of an iframe-heavy page came to:

  14 watchdog tasks spawned
  11 page-wide Page.stopLoading calls
  3 tasks outliving the fetch and firing against a closed page

Page.stopLoading takes no frame or loader argument - it is the Stop button, and it stops the whole
page. Verified directly: one call stopped a pending main frame and a pending iframe in the same
instant. So the other 10 calls were redundant, and because they landed at arbitrary later times
they could stop a *subsequent* navigation we actually wanted - which is the likeliest reason the
same URL fetched in 8s on one run and 35s on the next.

Replaced with a single deadline, awaited inline so nothing can outlive the fetch (there is no
create_task left in this file at all):

    navigate (bounded)  ->  wait the configured delay  ->  Page.stopLoading  ->  extract

The delay is measured from when navigation finished, not from when it started. Anchoring it to the
start would quietly rob a slow-loading page of its settle time, and letting JS-rendered content
appear after load is the whole point of the setting. Verified with a server that takes 5s to answer
and renders via JS 2s after load: total 9.6s for a 4s delay, and the late content is captured.

Because a page is never reliably "finished" - many sites navigate as part of their normal design -
the delay restarts when the MAIN frame replaces its document, so a redirect or interstitial gets
the same settle time the first document got. Iframes do not restart it, and it is capped by
BROWSER_CONTENT_READY_MAX_RESETS (default 2).

Only the existing "wait n seconds before extracting text" stays user-facing;
BROWSER_NAVIGATION_TIMEOUT_SECONDS is a safety net with a sane default rather than a second knob
for users to reason about. This matches what other scrapers do: bound the navigation, do not fail
when it times out, settle, then extract.

Timings are also more predictable now - the four reported URLs went from 5-35s of variance to
4.6-7.2s at a 3s delay, all with full content and a 200.

Tested: 11 passed pyppeteer browser suite, 7 passed playwright, 494 unit + llm.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 17:08:28 +02:00
dgtlmoon 0d6b254fdb UI - Really fix #4413 #4419 - Mobile menu fix 2026-09-12 16:24:23 +02:00
dgtlmoon 900a77e828 Worker id/available fetcher (#4415)
* Worker ID # should be available to the content fetchers

* Actually init with the worker ID
2026-09-12 16:04:39 +02:00
dgtlmoon 35f0294274 UI - Really fix #4413 #4419 Counter bubble in minimal sidebar 2026-09-12 16:03:22 +02:00
dgtlmoon de38c6cd4c Rebuild SCSS/CSS 2026-09-12 15:40:49 +02:00
Samar Nathani b5d768c1d6 feat(ui): show count badges in minimal sidebar (#4419)
In minimal sidebar mode, count badges (unread changes, queue size)
were hidden until hover. This makes them always visible so users can
see at a glance how many items need attention without expanding the
rail.

Closes #4413
2026-09-12 15:40:23 +02:00
dgtlmoonandClaude Opus 5 6318fc70f4 Browser fetchers - Report the real status code when Chrome aborts a bodiless error response (#4420)
* Browser fetchers - Report the real status code when Chrome aborts a bodiless error response

Chrome 153+ refuses to commit a navigation when a 4xx/5xx arrives with a zero-length body:
page.goto() raises net::ERR_HTTP_RESPONSE_CODE_FAILURE instead of returning the response. The
response is received fine, we just never get it as a return value, so the raw net:: string
landed in last_error instead of "Error - 404".

Verified against two browser images, same HTTP server:

  Chrome 153    empty-body 404 -> raises ERR_HTTP_RESPONSE_CODE_FAILURE
  Chrome 153    404 with body  -> status=404
  Chromium 119  empty-body 404 -> status=404
  Chromium 119  404 with body  -> status=404

The fix keeps the main-frame response from the 'response' event and hands that back when goto
raises, so .status / .all_headers() and the existing non-200 branch (which also captures the
screenshot) work unchanged. The latest matching response wins, so a redirect chain still
reports its final hop. Any other error re-raises as before, and if no response was captured we
re-raise too - the status is never invented, which keeps older browsers on exactly their old
path.

Two independent navigation sites needed it:

 - browser_steps.py action_goto_url - covers the playwright fetcher, the live Browser Steps UI,
   the Goto URL / Goto site steps, and the CloakBrowser plugin which imports it. This is the one
   that broke CI: content_fetchers/__init__.py forces playwright when a watch has browser steps,
   so test_non_200_errors_report_browsersteps ran the playwright path in the pyppeteer jobs too.

 - puppeteer.py - its own goto retry loop, used when FAST_PUPPETEER_CHROME_FETCHER is set and the
   watch has no browser steps. No test covers that path; verified by driving the fetcher directly.

Note pyppeteer exposes isNavigationRequest / frame / mainFrame as properties where playwright
uses is_navigation_request() as a method. Mixing them up raises 'bool' object is not callable,
which gets swallowed as a renderer page error rather than failing loudly. Checked against the
pinned pyppeteer-ng==2.0.0rc16.

Selenium is unaffected - it hardcodes status_code = 200 because WebDriver cannot see the HTTP
status, so it never reaches the non-200 branch.

Tested with the full CI browser set (test_content, test_errorhandling, test_fetch_data,
test_custom_js_before_content): 10 passed on each of Chrome 153 + playwright, Chrome 153 +
pyppeteer, Chromium 119 + playwright, Chromium 119 + pyppeteer, the last two against a canonical
Dockerfile.chromium119 build so Chromium is the only variable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* CI - Make a fail-fast test abort say what was skipped rather than looking like a total failure

Fail-fast is kept deliberately - the first failure is nearly always the real problem and it keeps
the run short - but nothing said so, which made a single failing assertion read as "every browser
test is broken".

The playwright and pyppeteer jobs each ran four pytest files as four commands in one `run:` block,
which GitHub executes under `bash -e`. tests/visualselector/test_fetch_data.py is the third, so
when one 404 assertion failed there, test_custom_js_before_content.py never ran, and the later
"Headers and requests" and "Restock detection" steps were skipped as a consequence - three test
files silently dropped, reported only as dashes in the job list. run_basic_tests.sh has the same
shape: 8 independent pytest groups under `set -e`, so a failure in the first parallel group hides
the 7 after it.

No behaviour change to when we stop - only to what gets reported:

 - Each browser test file now runs inside its own ::group:: so the log is navigable, and the
   failing file is named in a ::error:: annotation that states plainly that the remaining files
   and steps were SKIPPED, not failed.
 - run_basic_tests.sh gets an ERR trap saying the same thing, with the line number of the group
   that aborted.

Verified the loop stops on the third file, names it and exits 1, that the all-pass path still
exits 0, and that the trap reports the failing line while preserving the exit code. YAML parses
and run_basic_tests.sh passes bash -n.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Fix unit test failure - install the navigation-response tracker only on a page that supports events

action_goto_url() registered its 'response' listener unconditionally, which broke
test_fetch_url_gate.py::TestBrowserStepGotoUrlGate::test_permitted_url_still_navigates:

    self.page.on("response", _keep_navigation_response)
    E  AttributeError: '_RecordingPage' object has no attribute 'on'

The three refusal tests in that class still passed because validate_fetch_url_async() raises before
reaching the listener, so only the permitted-URL case (the one that actually navigates) hit it.

The listener now lives in track_latest_navigation_response(), which returns None for a page that
has no event support instead of raising. That also removes a real inefficiency: registering per
navigation meant a page accumulated a listener per goto(), and 'response' fires for every
subresource - measured 133 events on getastra.com (54 script, 37 image, 19 fetch, 11 xhr, ...) of
which only 2 were navigations. The tracker is installed once per page and shared, verified as one
listener remaining after an install plus four navigations.

Tested: 494 unit + llm tests pass (was 1 failed / 480 passed), and the full browser set still
passes 10/10 on playwright and 10/10 on pyppeteer, so the Chrome 153 "Error - 404" recovery still
works through the shared tracker.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 13:15:44 +02:00
dgtlmoon 5842e7a158 UI - 'Paused' status icon now has red dot instead of yellow #4418
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-11 13:40:09 +02:00
dgtlmoon 07d00d0811 0.60.4
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-10 07:04:49 +02:00
dgtlmoon e990d9909f UI - Watch list - Minor tweak to cell padding
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-09 18:54:57 +02:00
dgtlmoon 1e60bafc52 Rebuild translations 2026-09-09 18:51:00 +02:00
dgtlmoon 8354242121 UI - Watchlist - bring back Restock & Price column to watchlist overview (#4410) 2026-09-09 18:50:46 +02:00
滅üanddgtlmoon f71daee2c4 i18n: Update zh_Hant_TW translations (#4264)
Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>
2026-09-09 18:49:09 +02:00
dgtlmoon d57043be8b Merge branch 'master' of github.com:dgtlmoon/changedetection.io 2026-09-09 18:39:45 +02:00
dgtlmoon 689bbe412c UI - Watch list - Making list status icons larger, fixing icon text 2026-09-09 18:31:50 +02:00
dgtlmoon f3c0efac95 UI - Groups - Fixing button alignment and using better form handling (#4409) 2026-09-09 18:26:10 +02:00
dgtlmoon 4cf1bbe7cd Rebuild translations 2026-09-09 17:14:56 +02:00
dgtlmoon 7d45bf104d UI - Watch stats - Adding a little extra debug to 'Stats' tab 2026-09-09 17:13:05 +02:00
dgtlmoon 6b75954061 UI - Edit watch - Moving notification error alert to 'error' tab, hiding less used Page Title and Link options under foldout (#4406)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-09 17:01:07 +02:00
dgtlmoon 62fd5eed8f UI - Mobile - Recheck/edit/etc buttons - Centered horizontally 2026-09-09 17:00:05 +02:00
dgtlmoon 30fc5d318a UI - Mobile - Tidy up row padding 2026-09-09 16:57:40 +02:00
dgtlmoon 9657ff4bf9 UI - Watch list - Dont hide buttons on mobile, set light opacity on desktop 2026-09-09 16:47:29 +02:00
dgtlmoonandClaude Opus 5 821b538ae6 Performance - (watched page lister) open link single eval (#4405)
* Watchlist/Notifications - Resolve 'Link to Open' once per row, don't leak 'DISABLED' into notification tokens

Follow-up to #4290.

`watch.open_link` was read three times per watch list row (two hrefs plus the
new title attribute). Each read re-runs `_resolve_link()`, which for a Jinja2
templated URL builds a fresh ImmutableSandboxedEnvironment - the cost the
comment in validate_url.py already warns about. Hoisted to a single
`{% set open_link = watch.open_link %}` alongside the other per-row lookups.

`watch_open_url` was set from `watch.open_link`, which returns the string
'DISABLED' when the URL fails validation, so the default RSS body template
(`RSS_TEMPLATE_HTML_DEFAULT`) rendered `<a href="DISABLED">` for those watches.
Now falls back to the raw URL, matching the neighbouring `watch_url` token.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Rebuild template

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 16:19:34 +02:00
dgtlmoon c104179ab5 Rebuild translations 2026-09-09 16:13:13 +02:00
Jeff Hedlund 2f60135cd0 feature: Implemented optional override link (#4290)
* Implemented optional link to override feature

* Added translation strings (untranslated)
2026-09-09 16:06:24 +02:00
dgtlmoon f6d710ca85 Apprise notifications - Updating to 1.13.1 (#4404) 2026-09-09 15:50:58 +02:00
dgtlmoon 7f3645f4a8 API - DELETE for Watch history #4397 (#4403) 2026-09-09 15:46:00 +02:00
455e0228ca Tests - Fix TestHistoryPathTraversal on macOS, and cover the containment check (#4390)
* Compare against a resolved data_dir in the history path-traversal test

Watch.history resolves entries with os.path.realpath, so
test_normal_snapshot_entry_is_accepted compared a resolved path against an
unresolved data_dir. On macOS the datastore lives under /tmp, which is a
symlink to /private/tmp, so the assertion fails for a path that is in fact
inside the directory. The guard is correct; the test was not.

Resolve both sides, matching what the production code does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Add a test that actually exercises the history containment check

Disabling the containment check in Watch.history left every test in
TestHistoryPathTraversal passing. os.path.basename() reduces both traversal
fixtures ('/etc/passwd', '../../etc/passwd') to 'passwd', so neither reaches
the check — they stop at the os.path.exists() test below it.

A bare '..' survives basename() and resolves to the parent of data_dir, which
exists, so the containment check is what rejects it. With that check disabled
this new test is the only one in the class that fails.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: GG5533 <285285461+GG5533@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-09 15:43:54 +02:00
dgtlmoon cd15016465 UI - Language selector menu-bar icon was cut off by a couple of pixels 2026-09-09 15:16:47 +02:00
dgtlmoon d6996b7a7e Translations - Rebuild catalog 2026-09-09 15:13:59 +02:00
dgtlmoon f1c681185f UI - 'Toast' notifications of actions was at the same height as menu tabs, moved down slightly. 2026-09-09 15:12:28 +02:00
dgtlmoon d901efc8e7 UI - Left menu folds out ONTOP of content, Improving VisualSelector & BrowserSteps width, Darkmode transitions off, Extra sidebar modes #4380 (#4401) 2026-09-09 15:09:27 +02:00
Hans Valerie 1dc7850d6c i18n: add Indonesian translation (#4402) 2026-09-09 15:08:40 +02:00
dgtlmoon 0f4b556af0 0.60.3
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
2026-09-04 14:12:00 +02:00
dgtlmoon 7a3bc2ab9e GHSA-56fq-63vj-9992 Add-watch-UI should be POST/CSRF protected @AmerMrkaljevic (#4375) 2026-09-04 12:39:36 +02:00
dgtlmoon 3a71777499 UI - Fixing lots of actions that should be a POST style action which led to a 404
Also solves some of GHSA-56fq-63vj-9992 Add-watch-UI should be POST/CSRF protected
2026-09-04 12:38:20 +02:00
dgtlmoon e0fb224d41 UI - Fixing lots of actions that should be a POST style action which led to a 404 (#4370) 2026-09-04 10:56:23 +02:00
dgtlmoon 050172129e Translations - support for translation overlays/local translations of existing strings (#4365)
Build and push containers / metadata (push) Canceled after 0s
Build and push containers / build-push-containers (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Canceled after 0s
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Canceled after 0s
ChangeDetection.io App Test / lint-code (push) Canceled after 0s
ChangeDetection.io App Test / lint-translations (push) Canceled after 0s
ChangeDetection.io App Test / lint-template-i18n (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Canceled after 0s
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-11 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-12 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-13 (push) Canceled after 0s
ChangeDetection.io App Test / test-application-3-14 (push) Canceled after 0s
* Translations - support for translation overlays/local translations of existing strings

* Dont forget translations_overlay for pip
2026-09-03 11:15:46 +02:00
200 changed files with 17442 additions and 3091 deletions
+13
View File
@@ -52,6 +52,19 @@ jobs:
git diff --stat changedetectionio/translations
exit 1
fi
- name: Check translation overlay
# Deliberately after extract_messages above, so overrides are validated against a freshly
# extracted messages.pot. An overlay entry is keyed on the exact upstream msgid: when a
# string is reworded upstream the override stops matching and silently reverts to upstream
# wording. This is the only thing that makes that visible.
# See changedetectionio/translations_overlay/README.md
if: hashFiles('changedetectionio/translations_overlay/**/*.po') != ''
run: |
find changedetectionio/translations_overlay -name "*.po" | while read f; do
echo "Checking $f"
msgfmt --check-format -o /dev/null "$f"
done
python changedetectionio/translations_overlay/manage.py check
lint-template-i18n:
runs-on: ubuntu-latest
@@ -62,6 +62,18 @@ jobs:
echo "---- Built for Python ${{ env.PYTHON_VERSION }} -----"
docker run test-changedetectionio bash -c 'pip list'
- name: License compliance - no strong copyleft in shipped image
run: |
# Runs against the built image, i.e. exactly what ships, with the entrypoint
# bypassed so EXTRA_PACKAGES cannot pull in the opt-in AGPL osint plugin.
# --partial-match is required: plain --fail-on is an exact string match and
# would silently miss declarations like "GPL-2.0-or-later".
docker run --rm --entrypoint /bin/bash test-changedetectionio -c '
pip install --quiet pip-licenses &&
pip-licenses --partial-match \
--fail-on="GPL-2.0;GPL-3.0;AGPL;GNU General Public License;GNU Affero General Public License"
'
- name: We should be Python ${{ env.PYTHON_VERSION }} ...
run: |
docker run test-changedetectionio bash -c 'python3 --version'
@@ -201,10 +213,20 @@ jobs:
- name: Playwright - Specific tests in built container
run: |
docker run --rm -e "FLASK_SERVER_NAME=cdio" -e "PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000" --network changedet-network --hostname=cdio test-changedetectionio bash -c 'cd changedetectionio;pytest -vv --capture=tee-sys --showlocals --tb=long --live-server-host=0.0.0.0 --live-server-port=5004 tests/fetchers/test_content.py'
docker run --rm -e "FLASK_SERVER_NAME=cdio" -e "PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000" --network changedet-network --hostname=cdio test-changedetectionio bash -c 'cd changedetectionio;pytest -vv --capture=tee-sys --showlocals --tb=long --live-server-host=0.0.0.0 --live-server-port=5004 tests/test_errorhandling.py'
docker run --rm -e "FLASK_SERVER_NAME=cdio" -e "PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000" --network changedet-network --hostname=cdio test-changedetectionio bash -c 'cd changedetectionio;pytest -vv --capture=tee-sys --showlocals --tb=long --live-server-host=0.0.0.0 --live-server-port=5004 tests/visualselector/test_fetch_data.py'
docker run --rm -e "FLASK_SERVER_NAME=cdio" -e "PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000" --network changedet-network --hostname=cdio test-changedetectionio bash -c 'cd changedetectionio;pytest -vv --capture=tee-sys --showlocals --tb=long --live-server-host=0.0.0.0 --live-server-port=5004 tests/fetchers/test_custom_js_before_content.py'
# Deliberately fail fast - the first failure is nearly always the real problem and it
# keeps the run short. Each file is wrapped in its own log group and the failing one is
# named explicitly, because everything after it is SKIPPED rather than run, and that is
# otherwise easy to misread as "the whole browser suite broke".
for t in tests/fetchers/test_content.py tests/test_errorhandling.py tests/visualselector/test_fetch_data.py tests/fetchers/test_custom_js_before_content.py tests/fetchers/test_renavigation.py; do
echo "::group::pytest $t"
if ! docker run --rm -e "FLASK_SERVER_NAME=cdio" -e "PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000" --network changedet-network --hostname=cdio test-changedetectionio \
bash -c "cd changedetectionio;pytest -vv --capture=tee-sys --showlocals --tb=long --live-server-host=0.0.0.0 --live-server-port=5004 $t"; then
echo "::endgroup::"
echo "::error::$t FAILED - stopping here. Any later test file and the remaining steps of this job were SKIPPED, not failed."
exit 1
fi
echo "::endgroup::"
done
- name: Playwright - Headers and requests
run: |
@@ -242,10 +264,17 @@ jobs:
- name: Pyppeteer - Specific tests in built container
run: |
docker run --rm -e "FLASK_SERVER_NAME=cdio" -e "FAST_PUPPETEER_CHROME_FETCHER=True" -e "PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000" --network changedet-network --hostname=cdio test-changedetectionio bash -c 'cd changedetectionio;pytest --live-server-host=0.0.0.0 --live-server-port=5004 tests/fetchers/test_content.py'
docker run --rm -e "FLASK_SERVER_NAME=cdio" -e "FAST_PUPPETEER_CHROME_FETCHER=True" -e "PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000" --network changedet-network --hostname=cdio test-changedetectionio bash -c 'cd changedetectionio;pytest --live-server-host=0.0.0.0 --live-server-port=5004 tests/test_errorhandling.py'
docker run --rm -e "FLASK_SERVER_NAME=cdio" -e "FAST_PUPPETEER_CHROME_FETCHER=True" -e "PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000" --network changedet-network --hostname=cdio test-changedetectionio bash -c 'cd changedetectionio;pytest --live-server-host=0.0.0.0 --live-server-port=5004 tests/visualselector/test_fetch_data.py'
docker run --rm -e "FLASK_SERVER_NAME=cdio" -e "FAST_PUPPETEER_CHROME_FETCHER=True" -e "PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000" --network changedet-network --hostname=cdio test-changedetectionio bash -c 'cd changedetectionio;pytest --live-server-host=0.0.0.0 --live-server-port=5004 tests/fetchers/test_custom_js_before_content.py'
# Fail fast, but name the file that failed - see the note in the playwright job above
for t in tests/fetchers/test_content.py tests/test_errorhandling.py tests/visualselector/test_fetch_data.py tests/fetchers/test_custom_js_before_content.py tests/fetchers/test_renavigation.py; do
echo "::group::pytest $t"
if ! docker run --rm -e "FLASK_SERVER_NAME=cdio" -e "FAST_PUPPETEER_CHROME_FETCHER=True" -e "PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000" --network changedet-network --hostname=cdio test-changedetectionio \
bash -c "cd changedetectionio;pytest --live-server-host=0.0.0.0 --live-server-port=5004 $t"; then
echo "::endgroup::"
echo "::error::$t FAILED - stopping here. Any later test file and the remaining steps of this job were SKIPPED, not failed."
exit 1
fi
echo "::endgroup::"
done
- name: Pyppeteer - Headers and requests checks
run: |
-54
View File
@@ -1,54 +0,0 @@
# Generally
In any commercial activity involving 'Hosting' (as defined herein), whether in part or in full, this license must be executed and adhered to.
# Commercial License Agreement
This Commercial License Agreement ("Agreement") is entered into by and between Web Technologies s.r.o. here-in ("Licensor") and (your company or personal name) _____________ ("Licensee"). This Agreement sets forth the terms and conditions under which Licensor provides its software ("Software") and services to Licensee for the purpose of reselling the software either in part or full, as part of any commercial activity where the activity involves a third party.
### Definition of Hosting
For the purposes of this Agreement, "hosting" means making the functionality of the Program or modified version available to third parties as a service. This includes, without limitation:
- Enabling third parties to interact with the functionality of the Program or modified version remotely through a computer network.
- Offering a service the value of which entirely or primarily derives from the value of the Program or modified version.
- Offering a service that accomplishes for users the primary purpose of the Program or modified version.
## 1. Grant of License
Subject to the terms and conditions of this Agreement, Licensor grants Licensee a non-exclusive, non-transferable license to install, use, and resell the Software. Licensee may:
- Resell the Software as part of a service offering or as a standalone product.
- Host the Software on a server and provide it as a hosted service (e.g., Software as a Service - SaaS).
- Integrate the Software into a larger product or service that is then sold or provided for commercial purposes, where the software is used either in part or full.
## 2. License Fees
Licensee agrees to pay Licensor the license fees specified in the ordering document. License fees are due and payable as specified in the ordering document. The fees may include initial licensing costs and recurring fees based on the number of end users, instances of the Software resold, or revenue generated from the resale activities.
## 3. Resale Conditions
Licensee must comply with the following conditions when reselling the Software, whether the software is resold in part or full:
- Provide end users with access to the source code under the same open-source license conditions as provided by Licensor.
- Clearly state in all marketing and sales materials that the Software is provided under a commercial license from Licensor, and provide a link back to https://changedetection.io.
- Ensure end users are aware of and agree to the terms of the commercial license prior to resale.
- Do not sublicense or transfer the Software to third parties except as part of an authorized resale activity.
## 4. Hosting and Provision of Services
Licensee may host the Software (either in part or full) on its servers and provide it as a hosted service to end users. The following conditions apply:
- Licensee must ensure that all hosted versions of the Software comply with the terms of this Agreement.
- Licensee must provide Licensor with regular reports detailing the number of end users and instances of the hosted service.
- Any modifications to the Software made by Licensee for hosting purposes must be made available to end users under the same open-source license conditions, unless agreed otherwise.
## 5. Services
Licensor will provide support and maintenance services as described in the support policy referenced in the ordering document should such an agreement be signed by all parties. Additional fees may apply for support services provided to end users resold by Licensee.
## 6. Reporting and Audits
Licensee agrees to provide Licensor with regular reports detailing the number of instances, end users, and revenue generated from the resale of the Software. Licensor reserves the right to audit Licensee’s records to ensure compliance with this Agreement.
## 7. Term and Termination
This Agreement shall commence on the effective date and continue for the period set forth in the ordering document unless terminated earlier in accordance with this Agreement. Either party may terminate this Agreement if the other party breaches any material term and fails to cure such breach within thirty (30) days after receipt of written notice.
## 8. Limitation of Liability and Disclaimer of Warranty
Executing this commercial license does not waive the Limitation of Liability or Disclaimer of Warranty as stated in the open-source LICENSE provided with the Software. The Software is provided "as is," without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the Software or the use or other dealings in the Software.
## 9. Governing Law
This Agreement shall be governed by and construed in accordance with the laws of the Czech Republic.
## Contact Information
For commercial licensing inquiries, please contact contact@changedetection.io and dgtlmoon@gmail.com.
+7
View File
@@ -9,3 +9,10 @@ Install the development and test dependencies with `pip install -r requirements-
Please be sure that all new functionality has a matching test!
Use `pytest` to validate/test, you can run the existing tests as `pytest tests/test_notification.py` for example
### New dependencies
Please check the licence before adding anything to `requirements.txt`. MIT, BSD,
Apache-2.0, LGPL and MPL are fine; GPL and AGPL are not, since they would
relicense the whole project. CI enforces this. There is usually a permissive
equivalent - ask in the PR if you're not sure.
+25
View File
@@ -101,6 +101,31 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libxrender-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
# Actually generate the locales. Installing the `locales` package above only
# ships /etc/locale.gen - it does not build any locale, so the image had just
# C, C.utf8 and POSIX. That made the `ENV LC_ALL=en_US.UTF-8` below unsatisfiable:
# locale.setlocale() in flask_app.py failed, fell back to C, and the
# format_number_locale / format_int_locale Jinja filters silently lost their
# thousands separators - 1234567.89 rendered as "1234567.89" rather than
# "1,234,567.89" in the restock/price overview, which is the very thing the
# `locales` package was added for.
#
# More than en_US is generated so that operators can override LC_ALL / LANG and
# get formatting for their own region (de_DE gives 1.234.567,89, fr_FR gives
# 1 234 567,89). Costs ~21MB and ~16s of build time.
#
# This list mirrors the UI translations in changedetectionio/translations - one
# glibc locale per language we ship a translation for, so any language a user
# can pick in the UI also has a working locale. Keep the two in sync when adding
# a translation. The territory for each bare language code comes from CLDR's
# likely-subtags (cs -> cs_CZ, ja -> ja_JP, ko -> ko_KR, uk -> uk_UA, zh ->
# zh_CN, zh_Hant_TW -> zh_TW), NOT from uppercasing the language code.
RUN for l in cs_CZ de_DE en_GB en_US es_ES fr_FR id_ID it_IT ja_JP ko_KR \
pl_PL pt_BR ru_RU tr_TR uk_UA zh_CN zh_TW; do \
sed -i "s/^# *${l}.UTF-8 UTF-8/${l}.UTF-8 UTF-8/" /etc/locale.gen; \
done \
&& locale-gen
# https://stackoverflow.com/questions/58701233/docker-logs-erroneously-appears-empty-until-container-stops
ENV PYTHONUNBUFFERED=1
+1 -1
View File
@@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2025 Web Technologies s.r.o.
Copyright (c) Leigh Morresi and the changedetection.io contributors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
+1
View File
@@ -13,6 +13,7 @@ recursive-include changedetectionio/store *
recursive-include changedetectionio/templates *
recursive-include changedetectionio/tests *
recursive-include changedetectionio/translations *
recursive-include changedetectionio/translations_overlay *
recursive-include changedetectionio/widgets *
prune changedetectionio/static/package-lock.json
prune changedetectionio/static/styles/node_modules
+5
View File
@@ -66,6 +66,11 @@ Easily organise and monitor prices for products from the dashboard, get alerts a
[<img src="docs/restock-overview.png" style="max-width:100%;" alt="Easily keep an eye on product price changes directly from the UI" title="Easily keep an eye on product price changes directly from the UI" />](https://changedetection.io?src=github)
#### Monitor price change and restock trends over time as a graph, identify patterns, buy at the right time!
[<img src="docs/price-tracking.jpeg" style="max-width:100%;" />](https://changedetection.io?src=github)
Set price change notification parameters, upper and lower price, price change percentage and more.
Always know when a product for sale drops in price.
+2 -2
View File
@@ -1,6 +1,6 @@
[python: **.py]
keywords = _ _l gettext pgettext:1c,2
keywords = _ _l gettext pgettext:1c,2 lazy_pgettext:1c,2
[jinja2: **/templates/**.html]
encoding = utf-8
keywords = _ _l gettext pgettext:1c,2
keywords = _ _l gettext pgettext:1c,2 lazy_pgettext:1c,2
+19 -1
View File
@@ -2,7 +2,7 @@
# Read more https://github.com/dgtlmoon/changedetection.io/wiki
# Semver means never use .01, or 00. Should be .1.
__version__ = '0.60.2'
__version__ = '0.60.7'
from changedetectionio.strtobool import strtobool
from json.decoder import JSONDecodeError
@@ -77,6 +77,24 @@ if 'MALLOC_ARENA_MAX' not in os.environ:
except Exception:
pass
def configure_litellm_pricing_source():
# On first import litellm fetches its model pricing/context-window map from GitHub's 'main'
# branch: an outbound request nobody asked for, up to a 5s stall on offline installs, and
# unpinned data. Default to the map bundled with the installed litellm instead.
# LITELLM_FETCH_PRICING_FROM_GITHUB=true opts back in; an explicit LITELLM_LOCAL_MODEL_COST_MAP
# (litellm's own switch) always wins. Must run before the first 'import litellm'.
try:
fetch_from_github = strtobool(os.getenv('LITELLM_FETCH_PRICING_FROM_GITHUB', 'false'))
except ValueError:
# Runs at package import, so a typo must not stop the app from starting
logger.warning("LITELLM_FETCH_PRICING_FROM_GITHUB is not a valid true/false value, using false")
fetch_from_github = False
os.environ.setdefault('LITELLM_LOCAL_MODEL_COST_MAP', 'False' if fetch_from_github else 'True')
configure_litellm_pricing_source()
# Set spawn as global default (safety net - all our code uses explicit contexts anyway)
# Skip in tests to avoid breaking pytest-flask's LiveServer fixture (uses unpicklable local functions)
if 'pytest' not in sys.modules:
+18 -1
View File
@@ -9,6 +9,10 @@ import json
# Number of URLs above which import switches to background processing
IMPORT_SWITCH_TO_BACKGROUND_THRESHOLD = 20
# Query params whose accepted values are wider than the `enum:` in the spec and are checked
# separately further down (plugin processors aren't listed in the static spec enum).
ENUM_VALIDATED_ELSEWHERE = {'processor'}
def default_content_type(content_type='text/plain'):
"""Decorator to set a default Content-Type header if none is provided."""
@@ -143,10 +147,23 @@ class Import(Resource):
# Convert to appropriate type based on schema
try:
converted_value = convert_query_param_to_type(param_value, schema_properties[param_name])
extras[param_name] = converted_value
except (ValueError, json.JSONDecodeError) as e:
return f"Invalid value for parameter '{param_name}': {str(e)}", 400
# Enforce any `enum:` declared in the OpenAPI spec (notification_format, method,
# conditions_match_logic ...). /api/v1/watch gets this for free because its JSON body is
# unmarshalled against the spec, but the import query params never were - so something
# like ?notification_format=Text used to be stored verbatim and then raise
# "Invalid notification format" at notification-send time, long after the import.
# `processor` is exempt: plugin processors are legal but aren't in the static spec enum,
# it has its own check against available_processors() below.
allowed_values = schema_properties[param_name].get('enum')
if allowed_values and param_name not in ENUM_VALIDATED_ELSEWHERE and converted_value not in allowed_values:
return (f"Invalid value for parameter '{param_name}': '{param_value}'. "
f"Must be one of: {', '.join(str(v) for v in allowed_values)}"), 400
extras[param_name] = converted_value
# Validate processor if provided
if 'processor' in extras:
from changedetectionio.processors import available_processors
+16
View File
@@ -141,6 +141,7 @@ class Watch(Resource):
watch['last_changed'] = watch_obj.last_changed
watch['viewed'] = watch_obj.viewed
watch['link'] = watch_obj.link
watch['open_link'] = watch_obj.open_link
# Resolved processor config: tag override wins over watch-level config (mirrors restock processor logic)
import json
@@ -293,6 +294,20 @@ class WatchHistory(Resource):
abort(404, message='No watch exists with the UUID of {}'.format(uuid))
return watch.history, 200
# Delete all history/snapshots for a watch, but keep the watch itself
# curl -X DELETE http://localhost:5000/api/v1/watch/<uuid_str:uuid>/history
@auth.check_token
@validate_openapi_request('deleteWatchHistory')
def delete(self, uuid):
"""Clear all snapshot history for a watch (the watch itself is kept)."""
if not self.datastore.data['watching'].get(uuid):
abort(404, message='No watch exists with the UUID of {}'.format(uuid))
# Same call as the UI "Clear history" button - wipes snapshots/screenshots and
# resets last_checked etc, while preserving the watch and its processor config
self.datastore.clear_watch_history(uuid)
return 'OK', 204
class WatchSingleHistory(Resource):
def __init__(self, **kwargs):
@@ -596,6 +611,7 @@ class CreateWatch(Resource):
'last_checked': watch['last_checked'],
'last_error': watch['last_error'],
'link': watch.link,
'open_link': watch.open_link,
'page_title': watch['page_title'],
'tags': [*tags], # Unpack dict keys to list (can't use list() since variable named 'list')
'title': watch['title'],
@@ -1,8 +1,10 @@
import os
from flask import Blueprint, render_template, request, jsonify, make_response, flash, redirect, url_for
from flask_babel import gettext
from loguru import logger
from changedetectionio import forms
from changedetectionio import forms, strtobool
from changedetectionio.auth_decorator import login_optionally_required
from . import browser_config
from changedetectionio.store import ChangeDetectionStore
@@ -34,6 +36,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
return render_template(
"add-watch-ui.html",
browser_backend_disabled=strtobool(os.getenv('ADD_WATCH_UI_BROWSER_BACKEND_DISABLED', 'False')),
form=form,
llm_configured=llm_configured,
llm_intent_watch_placeholder=LLM_INTENT_WATCH_PLACEHOLDER,
@@ -42,7 +45,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
system_default_browser=browser_config.system_default_description(datastore),
)
@add_watch_ui_blueprint.route("/snapshot", methods=['GET'])
@add_watch_ui_blueprint.route("/snapshot", methods=['POST'])
@login_optionally_required
def add_watch_ui_snapshot():
"""One-shot live fetch of an arbitrary URL for the Add Watch visual selector.
@@ -52,6 +55,10 @@ def construct_blueprint(datastore: ChangeDetectionStore):
connect, "Goto site", grab the screenshot + xpath element data, then tear
the browser down again. Element selection then happens client-side on the
returned data, exactly like the watch Edit page's visual selector.
POST-only and CSRF protected on purpose: this drives a real browser fetch and
writes a temporary watch dir, so as a GET it could be triggered cross-origin
(or by any tag/link that issues a GET) without the operator's consent.
"""
import base64
from changedetectionio.blueprint.browser_steps import (
@@ -71,7 +78,10 @@ def construct_blueprint(datastore: ChangeDetectionStore):
# backslash/parser-differential rejection of GHSA-rph4-96w6-q594 (GHSA-56fq-63vj-9992).
# Note this fetch never reaches difference_detection_processor.call_browser(), so it gets
# no gating from there - it has to validate for itself.
url = (request.args.get('url') or '').strip()
url = (request.form.get('url') or '').strip()
if not url.lower().startswith('http'):
url='https://' + url
ok, reason = is_fetch_url_allowed(url)
if not ok:
logger.warning(f"Add-watch snapshot: refused '{url}' - {reason}")
@@ -82,7 +92,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
# Either way it has to be able to render a preview - the plain HTTP client
# produces no screenshot and no element data, so previewing with it is pointless
# (and it used to be the silent default here, see the system-default bug).
fetcher_name = (request.args.get('fetch_backend') or '').strip() or browser_config.default_visual_browser(datastore)
fetcher_name = (request.form.get('fetch_backend') or '').strip() or browser_config.default_visual_browser(datastore)
if not fetcher_name or not browser_config.is_visual_capable(fetcher_name, datastore):
logger.warning(f"Add-watch snapshot: refused browser '{fetcher_name}' for '{url}'")
return make_response('No interactive browser available that can render a live preview '
@@ -59,9 +59,18 @@ $(document).ready(() => {
$.ajax({
url: add_watch_snapshot_url,
// POST, never GET - this makes the server-side browser fetch a URL of our
// choosing, so it must not be triggerable cross-origin. csrf.js adds the
// X-CSRFToken header to every non-GET ajax call; the CSRF field on the form
// is sent too so it works even if that handler hasn't run yet.
method: 'POST',
// Preview with the browser picked in the list - that same browser is what
// gets saved on the watch, so what you see here is what it will check with.
data: {url: url, fetch_backend: $('input[name="fetch_backend"]:checked').val() || ''},
data: {
url: url,
fetch_backend: $('input[name="fetch_backend"]:checked').val() || '',
csrf_token: $('#new-watch-form input[name="csrf_token"]').val() || '',
},
dataType: 'json',
}).done((data) => {
showState('ready');
@@ -15,7 +15,7 @@
<!-- TOP : enter the URL and fetch a live preview -->
<div id="add-watch-url-row">
{{ render_nolabel_field(form.url, placeholder="https://...", required=true, class="pure-input-1") }}
{{ render_nolabel_field(form.url, placeholder="www.website.com/page.html", required=true, class="pure-input-1") }}
<button type="button" id="add-watch-go" class="pure-button pure-button-primary">{{ _('Go') }}</button>
</div>
@@ -51,14 +51,14 @@
<!-- RIGHT : options -->
<div id="add-watch-options-pane">
<div class="add-watch-option-group" id="quick-watch-processor-type">
{{ render_simple_field(form.processor) }}
{{ render_simple_field(form.processor, label=_('What do you want to achieve?')) }}
</div>
{%- if not browser_backend_disabled -%}
<div class="add-watch-option-group" id="quick-watch-fetch-backend">
{# Rendered by hand rather than with render_field(): the system-default entry
is listed for explanation and has to render disabled, and WTForms can't
carry a per-option disabled attribute through a RadioField's choices. #}
<span class="label"><label>{{ form.fetch_backend.label.text }}</label></span>
<label>{{ _('Select a browser') }}</label>
<ul>
{%- for browser in form.fetch_backend -%}
{%- set unusable = browser.data in unusable_browsers -%}
@@ -68,7 +68,7 @@
{%- endfor -%}
</ul>
</div>
{%- endif -%}
<div class="add-watch-option-group" id="by-element-toggle-group">
{%- if llm_configured -%}
<label class="pure-checkbox" for="by-element-toggle">
@@ -78,7 +78,6 @@
{# No LLM intent available? Then narrowing by element is the only thing this
page can do - selection is always on, so nothing is labelled or offered
here, just the hint on how to use the preview. #}
<span class="pure-form-message-inline">{{ _('Hover & click the preview to watch just one part of the page.') }}</span>
<a id="clear-selector" class="pure-button button-secondary button-xsmall" style="display: none;">{{ _('Clear selection') }}</a>
</div>
@@ -98,7 +98,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
backups_blueprint.register_blueprint(construct_restore_blueprint(datastore))
backup_threads = []
@backups_blueprint.route("/request-backup", methods=['GET'])
@backups_blueprint.route("/request-backup", methods=['POST'])
@login_optionally_required
def request_backup():
if any(thread.is_alive() for thread in backup_threads):
@@ -35,8 +35,10 @@
</p>
{% endif %}
<a class="pure-button pure-button-primary"
href="{{ url_for('backups.request_backup') }}">{{ _('Create backup') }}</a>
<form method="POST" action="{{ url_for('backups.request_backup') }}" style="display: inline;">
<input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
<button type="submit" class="pure-button pure-button-primary">{{ _('Create backup') }}</button>
</form>
{% if available_backups %}
{# POST + CSRF token: this permanently deletes every backup archive, so it must
not be reachable from a bare GET (an <img src=...> on any page the operator
@@ -296,7 +296,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
return browsersteps_start_session
@browser_steps_blueprint.route("/browsersteps_start_session", methods=['GET'])
@browser_steps_blueprint.route("/browsersteps_start_session", methods=['POST'])
@login_optionally_required
def browsersteps_start_session():
# A new session was requested, return sessionID
@@ -100,7 +100,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
results = _recalc_check_status(uuid=uuid)
return results
@check_proxies_blueprint.route("/<uuid_str:uuid>/start", methods=['GET'])
@check_proxies_blueprint.route("/<uuid_str:uuid>/start", methods=['POST'])
@login_optionally_required
def start_check(uuid):
@@ -19,6 +19,7 @@
<div class="tab-pane-inner" id="url-list">
<p>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('Restoring changedetection.io backups is in the') }}<a href="{{ url_for('backups.restore.restore') }}"> {{ _('backups section') }}</a>.
<br>
</p>
@@ -90,6 +91,7 @@
</fieldset>
<div class="pure-control-group">
<span class="pure-form-message-inline">
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('Table of custom column and data types mapping for the') }} <strong>{{ _('Custom mapping') }}</strong> {{ _('File mapping type.') }}
</span>
<table style="border: 1px solid #aaa; padding: 0.5rem; border-radius: 4px;">
+16
View File
@@ -0,0 +1,16 @@
"""Left-rail (action sidebar) display modes.
A leaf module on purpose: forms.py, flask_app.py and model/App.py all need these, and
model/App.py seeding its default from forms.py would drag the whole form stack (~550
modules) into the model layer.
"""
from flask_babel import lazy_gettext as _l
# The complete set of left-rail modes - flask_app.get_sidebar_mode_class() maps these
# (and only these) onto body classes, so a new mode here needs a new mapping there.
MENU_SIDEBAR_ACTIONMODES = [
('expandable', _l('Expand on hover')), # Slim icon rail that expands on hover/focus
('pinned-expanded', _l('Always expanded')), # Always expanded, never collapses
('minimal', _l('Stays minimal')), # Always small, never expands
]
MENU_SIDEBAR_ACTIONMODES_DEFAULT = 'expandable'
@@ -14,7 +14,7 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q: PriorityQueue
price_data_follower_blueprint = Blueprint('price_data_follower', __name__)
@price_data_follower_blueprint.route("/<uuid_str:uuid>/accept", methods=['GET'])
@price_data_follower_blueprint.route("/<uuid_str:uuid>/accept", methods=['POST'])
@login_optionally_required
def accept(uuid):
datastore.data['watching'][uuid]['track_ldjson_price_data'] = PRICE_DATA_TRACK_ACCEPT
@@ -24,7 +24,7 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q: PriorityQueue
worker_pool.queue_item_async_safe(update_q, queuedWatchMetaData.PrioritizedItem(priority=1, item={'uuid': uuid}))
return redirect(url_for("watchlist.index"))
@price_data_follower_blueprint.route("/<uuid_str:uuid>/reject", methods=['GET'])
@price_data_follower_blueprint.route("/<uuid_str:uuid>/reject", methods=['POST'])
@login_optionally_required
def reject(uuid):
datastore.data['watching'][uuid]['track_ldjson_price_data'] = PRICE_DATA_TRACK_REJECT
+121
View File
@@ -0,0 +1,121 @@
"""Progressive Web App support - manifest, service worker, Android share target.
Both routes here are registered at the app ROOT (url_prefix=''), and that is load-bearing
rather than a style choice:
- a manifest's scope defaults to its own directory, and the relative URLs it uses so that
sub-path installs work resolve against the manifest's own URL. Served from anywhere but
the root, the whole app would be scoped to that subdirectory,
- a service worker can only control paths at or below its own URL, so from /pwa/sw.js its
scope would be /pwa/ and it would control nothing - no WebAPK, and therefore no entry in
the Android share sheet.
The third piece, the share target itself, lives in the watch list blueprint: the manifest
points share_target at the watch list so a shared link just prefills the quick-add form that
was already there. See service.pwa_preset_url().
"""
import os
from flask import Blueprint, make_response, render_template, request, send_from_directory, g
from changedetectionio.auth_decorator import login_optionally_required
from . import service
HERE = os.path.dirname(os.path.abspath(__file__))
def construct_blueprint():
pwa_blueprint = Blueprint('pwa', __name__, template_folder="templates")
@pwa_blueprint.app_template_global('pwa_https_mode')
def pwa_https_mode():
"""Template-side gate for anything that invites a phone install.
Registered app-wide (not just on this blueprint) so the Settings page - and a future
welcome wizard - can ask the same question and get the same answer.
"""
return service.https_mode(request.is_secure)
@pwa_blueprint.route("/site.webmanifest", methods=['GET'])
def site_webmanifest():
"""The PWA manifest.
Not a static file, where it sat as a leftover of the 2022 favicon-generator import
(3a8a41a3f), because: the scope problem above; Python's mimetypes has no entry for
.webmanifest so send_from_directory hands it out as application/octet-stream; and
the name has to vary per instance.
"""
name, short_name = service.instance_names(
forwarded_prefix=request.headers.get('X-Forwarded-Prefix'),
script_root=request.script_root,
)
response = make_response(render_template(
"manifest.json",
name=name,
short_name=short_name,
# Deliberately not translated: this response is cached publicly and shared by
# every visitor of the instance, so it can't carry one user's session locale.
description=service.DESCRIPTION,
))
response.headers['Content-Type'] = 'application/manifest+json'
# Short TTL so a rename propagates, but long enough that the periodic re-fetch an
# installed PWA performs isn't a per-navigation cost.
response.headers['Cache-Control'] = 'public, max-age=3600'
# Identical for every visitor of this instance, so let
# PublicStaticAssetSessionInterface drop the "Vary: Cookie" that would otherwise key
# it on the caller's session cookie.
g.public_static_asset = True
return response
@pwa_blueprint.route("/sw.js", methods=['GET'])
def service_worker():
response = make_response(send_from_directory(os.path.join(HERE, "static"), path="sw.js"))
response.headers['Content-Type'] = 'text/javascript'
# The worker script itself must not be cached hard, or a fix to it can't be shipped:
# browsers re-fetch it to detect updates and will honour a stale cache entry.
response.headers['Cache-Control'] = 'no-cache, must-revalidate'
g.public_static_asset = True
return response
@pwa_blueprint.route("/pwa/install-qrcode.svg", methods=['GET'])
@login_optionally_required
def install_qrcode():
"""QR code pointing at this instance, for installing the PWA on a phone.
Getting the app onto a phone otherwise means typing a URL like
https://example.com/my-instance/ on a phone keyboard, sub-path and all. That is the
actual barrier to the share-sheet feature existing at all, so it is worth removing.
Unlike the manifest and the worker, this one is NOT public - it is only rendered
inside Settings, and it would tell an anonymous caller the instance's external
address. The blueprint-wide exemption in check_authentication() deliberately lists
endpoints rather than the whole blueprint, so adding a route here doesn't publish it.
The path carries an explicit /pwa/ even though this blueprint is mounted at the
root: only the manifest and the worker actually need to sit at the top level.
SVG rather than PNG: a few hundred bytes, crisp at any size, no image library. Black
on white regardless of theme - a QR needs its quiet zone and contrast, and an <img>
can't inherit the page's dark mode anyway.
"""
import io as _io
import segno
# request.url_root, NOT url_for(_external=True): url_for builds from the configured
# SERVER_NAME, so behind a reverse proxy it emits the internal address and the QR
# sends the phone somewhere it can't reach. url_root is scheme + host + sub-path
# exactly as the browser asked for them.
buffer = _io.BytesIO()
segno.make(request.url_root, error='m').save(buffer, kind='svg', scale=4, border=2,
dark='#000000', light='#ffffff')
response = make_response(buffer.getvalue())
response.headers['Content-Type'] = 'image/svg+xml'
response.headers['Cache-Control'] = 'no-store'
return response
return pwa_blueprint
+119
View File
@@ -0,0 +1,119 @@
"""Pure helpers behind the PWA blueprint.
Kept free of Flask request state so they can be unit tested directly - the routes and the
watch list pass the request values in.
"""
import os
DEFAULT_NAME = 'ChangeDetection.io'
DESCRIPTION = 'Web page change detection and monitoring'
# Set to force the mobile-install UI on (or off) regardless of the detected scheme
HTTPS_OVERRIDE_ENV = 'FORCE_MOBILE_HTTPS_STYLE_MANIFEST'
def https_mode(is_secure):
"""Should this instance offer the phone-install UI?
A PWA can only be installed over HTTPS, so anything that invites someone to install
(today the Settings QR code, later an Install button) has to be hidden otherwise - it
would send them to a page with no Install option and no explanation why.
Detection is request.is_secure, which reads X-Forwarded-Proto via ProxyFix - but only
when USE_X_SETTINGS is set. Terminating TLS at a proxy that doesn't send that header, or
running without USE_X_SETTINGS, makes a perfectly installable instance look like plain
http, so FORCE_MOBILE_HTTPS_STYLE_MANIFEST overrides the guess in either direction.
"""
forced = os.getenv(HTTPS_OVERRIDE_ENV, '').strip()
if forced:
from changedetectionio import strtobool
return bool(strtobool(forced))
return bool(is_secure)
def instance_names(forwarded_prefix='', script_root=''):
"""Work out what this instance should be called on a home screen.
Co-tenanted instances (same host, different sub-path, as in
https://example.com/my-instance/) install as separate PWAs with separate icons and
separate Android share-sheet entries. Sharing one name makes them indistinguishable at
the point of use, so allow an override and otherwise derive something human out of the
sub-path: /my-instance -> "My Instance".
Returns (name, short_name).
"""
name = os.getenv('PWA_NAME', '').strip()
if not name:
prefix = (forwarded_prefix or script_root or '').strip('/')
name = prefix.replace('-', ' ').replace('_', ' ').title() if prefix else DEFAULT_NAME
# short_name is the home screen label, and there is no honest short form of
# "ChangeDetection.io". Truncating it here invented "ChangeDetect" - a second name for
# the product, on the most visible surface there is, when the trademark is the whole
# protective mechanism (see CLAUDE.md). Better to ship the real name and let the
# launcher ellipsise it like it does for every other long app name. PWA_SHORT_NAME is
# there for anyone who wants something shorter.
short_name = os.getenv('PWA_SHORT_NAME', '').strip() or name
return name, short_name
MAX_URL_LENGTH = 2000
def pwa_preset_url(args):
r"""Pull the URL out of an Android Web Share Target hand-off.
The manifest's share_target action is the watch list itself, so a share arrives as
GET /?pwa_preset_url=... and all this does is prefill the quick-add form's URL field -
the processor radios and LLM intent box already sitting under it then do the rest. No
watch is created here; that still takes the form's POST, so a share can't add anything
on its own.
The url param is only populated when the sharing app sets Intent.EXTRA_TEXT to a bare
link. Plenty of Android apps instead share prose with the link buried in it ("Check this
out https://..."), which arrives as text/title, so fall back to digging it out.
Extraction is linkify-it-py (already a dependency, also used by the notification
handler) rather than a regex, purely because it is better at this one job - the real
URL validation happens on the form's POST, not here. A naive https?://\S+ swallows the
full stop in "...have a look at https://example.com/page." and the closing bracket in
"(see https://example.com/x)", while still breaking
https://en.wikipedia.org/wiki/Foo_(bar) where the bracket belongs to the URL. A URL
prefilled with a trailing full stop is a failed first share.
"""
explicit = (args.get('pwa_preset_url') or '').strip()
# An app that populated the url field gave us a real URL, so take it verbatim - the
# linkifier would trim a trailing character that may well be significant here.
if explicit.lower().startswith(('http://', 'https://')):
return explicit[:MAX_URL_LENGTH]
from linkify_it import LinkifyIt
linkify = LinkifyIt()
for candidate in (explicit, args.get('pwa_preset_text'), args.get('pwa_preset_title')):
if not candidate:
continue
for match in linkify.match(candidate) or []:
# The linkifier also resolves mailto:, ftp: and protocol-relative //host links.
# Prefilling any of those is just noise in a box that only ever wants a web page.
if match.url.lower().startswith(('http://', 'https://')):
return match.url[:MAX_URL_LENGTH]
return ''
SHARE_ARGS = ('pwa_preset_url', 'pwa_preset_text', 'pwa_preset_title')
def is_share(args):
"""True when this request came in through the share sheet rather than the app icon.
start_url and share_target.action are the same URL ("./"), so the only thing telling a
share apart from a normal launch is the presence of these params. Tapping the home
screen icon must not produce share-related UI.
"""
return any(args.get(k) for k in SHARE_ARGS)
@@ -0,0 +1,26 @@
// changedetection.io service worker.
//
// This exists for ONE reason: Android only registers a Web Share Target for a real WebAPK,
// and Chrome's criteria for offering "Install" (rather than the shortcut that isn't a WebAPK
// and never reaches the share sheet) have wanted a service worker with a fetch handler. A
// user typically tries sharing exactly once, so the cost of being wrong here is the whole
// feature, and ten lines of no-op worker is cheap insurance.
//
// It deliberately CACHES NOTHING. Static assets already get fingerprinted ?v= URLs with
// immutable Cache-Control, so a caching layer here would add no speed - only the risk of
// serving a stale page with a dead CSRF token, from a cache the operator can't clear
// remotely. See test_cache_control_headers.py for what that failure looks like.
//
// Only navigations are intercepted, and only to pass them straight through. Leaving assets,
// range requests, Socket.IO and EventSource untouched keeps navigation preload and streaming
// working exactly as they do without a worker.
self.addEventListener('install', () => self.skipWaiting());
self.addEventListener('activate', (event) => event.waitUntil(self.clients.claim()));
self.addEventListener('fetch', (event) => {
if (event.request.mode === 'navigate') {
event.respondWith(fetch(event.request));
}
});
@@ -0,0 +1,76 @@
{#
PWA manifest. Served by site_webmanifest() in the pwa blueprint, NOT as a static file.
Every URL below is deliberately RELATIVE. Relative URLs in a manifest resolve against the
manifest's own URL, so an instance reverse-proxied at https://example.com/my-instance/ gets
a scope and start_url of /my-instance/ without ever being told its own prefix. That is why
these are not url_for(): url_for only produces the prefix when USE_X_SETTINGS is on AND the
proxy actually sends X-Forwarded-Prefix, whereas the URL the browser used to fetch this
file is always right.
"id" pins the app identity. It resolves to the same value that the old manifest's implicit
id (its start_url of "/") had on a root-hosted instance, so an already-installed PWA updates
in place rather than appearing as a second icon on the home screen.
#}
{
"id": "./",
"name": {{ name|tojson }},
"short_name": {{ short_name|tojson }},
"description": {{ description|tojson }},
"icons": [
{
"src": "static/favicons/android-chrome-192x192.png",
"sizes": "192x192",
"type": "image/png",
"purpose": "any"
},
{
"src": "static/favicons/android-chrome-256x256.png",
"sizes": "256x256",
"type": "image/png",
"purpose": "any"
},
{#
A separate file rather than "any maskable" on the two above. Android crops maskable
icons to whatever shape the launcher uses and only guarantees a circle of 80% of the
icon's width; the full-bleed art puts the white logo ring at 80.5% and 5px below centre,
so the ring was being clipped. This variant is recentred and inset to clear the safe
zone. One file cannot serve both purposes - "any" wants full-bleed, "maskable" wants
padding - which is what the old "any maskable" on a single entry was claiming.
#}
{
"src": "static/favicons/android-chrome-256x256-maskable.png",
"sizes": "256x256",
"type": "image/png",
"purpose": "maskable"
}
],
{#
Android share sheet integration. "action" is the watch list itself, so a shared link just
prefills the quick-add form's URL field and the processor radios already under it handle
picking a mode - no separate confirm screen to keep in sync with the real one.
method GET on purpose: a POST share target requires a service worker to intercept it,
whereas GET is a plain navigation. It also means nothing is created by the share itself -
adding the watch is still the form's POST, with its CSRF token.
Not supported by Safari on iOS at any version; the equivalent there is an Apple Shortcut
pointed at this same URL.
#}
"share_target": {
"action": "./",
"method": "GET",
"params": {
"url": "pwa_preset_url",
"text": "pwa_preset_text",
"title": "pwa_preset_title"
}
},
"scope": "./",
"start_url": "./",
"display": "standalone",
"theme_color": "#5bbad5",
"background_color": "#ffffff",
"categories": ["utilities", "productivity"],
"orientation": "any"
}
+1 -1
View File
@@ -24,4 +24,4 @@ RSS_TEMPLATE_PLAINTEXT_DEFAULT = "<pre>{{watch_label}} had a change.\n\n{{diff}}
# @todo add some [edit]/[history]/[goto] etc links
# @todo need {{watch_edit_link}} + delete + history link token
RSS_TEMPLATE_HTML_DEFAULT = "<html><body>\n<h4><a href=\"{{watch_url}}\">{{watch_label}}</a></h4>\n<p>{{diff}}</p>\n</body></html>\n"
RSS_TEMPLATE_HTML_DEFAULT = "<html><body>\n<h4><a href=\"{{watch_open_url}}\">{{watch_label}}</a></h4>\n<p>{{diff}}</p>\n</body></html>\n"
@@ -105,7 +105,7 @@ def construct_single_watch_routes(rss_blueprint, datastore):
fe = fg.add_entry()
title_suffix = f"Change @ {res['original_context']['change_datetime']}"
populate_feed_entry(fe, watch, res.get('body', ''), guid, timestamp_to,
link={'href': watch.get('url')}, title_suffix=title_suffix)
link={'href': watch.open_link_override or watch.get('url')}, title_suffix=title_suffix)
add_watch_categories(fe, watch, datastore)
response = make_response(fg.rss_str())
@@ -238,6 +238,15 @@ def construct_blueprint(datastore: ChangeDetectionStore):
)
llm_config = _get_llm_cfg(datastore) or {}
llm_env_configured = llm_configured_via_env()
# Once a key is stored, the provider and the key itself are locked and can only be
# cleared with "Remove provider". The key field renders blank so that an untouched
# save preserves it, which makes any stray value in it - a browser autofill, a bad
# paste - silently overwrite a working key on the next Save, with no copy kept
# anywhere to recover from. Disabling the input takes it out of the POST entirely;
# the form is seeded with data=default, so an absent field falls back to the stored
# value and the merge below is a no-op for it.
llm_provider_locked = bool(llm_config.get('api_key')) and not llm_env_configured
llm_stored = datastore.data['settings']['application'].get('llm') or {}
llm_token_budget_month = get_global_token_budget_month(datastore)
llm_token_budget_month_env = get_global_token_budget_month() # env var only, for readonly logic
@@ -253,6 +262,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
api_key=datastore.data['settings']['application'].get('api_access_token'),
llm_config=llm_config,
llm_env_configured=llm_env_configured,
llm_provider_locked=llm_provider_locked,
llm_stored=llm_stored,
llm_token_budget_month=llm_token_budget_month,
llm_token_budget_month_env=llm_token_budget_month_env,
@@ -278,7 +288,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
return output
@settings_blueprint.route("/reset-api-key", methods=['GET'])
@settings_blueprint.route("/reset-api-key", methods=['POST'])
@login_optionally_required
def settings_reset_api_key():
secret = secrets.token_hex(16)
@@ -295,7 +305,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
logs=notification_debug_log if len(notification_debug_log) else ["Notification logs are empty - no notifications sent yet."])
return output
@settings_blueprint.route("/toggle-all-paused", methods=['GET'])
@settings_blueprint.route("/toggle-all-paused", methods=['POST'])
@login_optionally_required
def toggle_all_paused():
current_state = datastore.data['settings']['application'].get('all_paused', False)
@@ -309,7 +319,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
return redirect(url_for('watchlist.index'))
@settings_blueprint.route("/toggle-all-muted", methods=['GET'])
@settings_blueprint.route("/toggle-all-muted", methods=['POST'])
@login_optionally_required
def toggle_all_muted():
current_state = datastore.data['settings']['application'].get('all_muted', False)
+1 -1
View File
@@ -130,7 +130,7 @@ def construct_llm_blueprint(datastore: ChangeDetectionStore):
logger.exception("LLM model list full traceback:")
return jsonify({'models': [], 'error': str(e)}), 400
@llm_blueprint.route("/test", methods=['GET'])
@llm_blueprint.route("/test", methods=['POST'])
@login_optionally_required
def llm_test():
from flask import request
@@ -31,6 +31,7 @@
<div class="pure-control-group" id="notification-base-url">
{{ render_field(form.base_url, class="m-d") }}
<span class="pure-form-message-inline">
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('Base URL used for the') }} <code>{{ '{{ base_url }}' }}</code> {{ _('token in notification links.') }}<br>
{{ _('Default value is the system environment variable') }} '<code>BASE_URL</code>' - <a href="https://github.com/dgtlmoon/changedetection.io/wiki/Configurable-BASE_URL-setting">{{ _('read more here') }}</a>.
</span>
@@ -9,6 +9,10 @@
<script src="{{url_for('static_content', group='js', filename='vis.js')}}" defer></script>
<script src="{{url_for('static_content', group='js', filename='global-settings.js')}}" defer></script>
<script src="{{url_for('static_content', group='js', filename='scheduler.js')}}" defer></script>
{# PWA related #}
{%- set menu_icon -%}&nbsp;<i data-feather="more-vertical"></i>{% endset -%}
{%- set install_action -%}<strong>{{ _('Install and create shortcut - then "install"') }}</strong>{% endset -%}
<div class="edit-form">
<div class="tabs collapsable">
<ul>
@@ -40,6 +44,7 @@
<div class="pure-control-group">
{{ render_field(form.requests.form.time_between_check, class="time-check-widget") }}
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
<span class="pure-form-message-inline">{{ _('Default recheck time for all watches, current system minimum is') }} <i>{{min_system_recheck_seconds}}</i> {{ _('seconds') }} (<a href="https://github.com/dgtlmoon/changedetection.io/wiki/Misc-system-settings#enviroment-variables">{{ _('more info') }}</a>).</span>
<div id="time-between-check-schedule">
<!-- Start Time and End Time {{ timezone_default_config }} -->
@@ -76,7 +81,7 @@
{% endif %}
</div>
<div class="pure-control-group">
<a href="{{url_for('ui.clear_all_history')}}" class="pure-button button-error">{{ _('Clear Snapshot History') }}</a>
<a href="{{url_for('ui.clear_all_history')}}" class="cdio-btn button-error">{{ _('Clear Snapshot History') }}</a>
</div>
<div class="pure-control-group">
{{ render_checkbox_field(form.application.form.shared_diff_access, class="shared_diff_access") }}
@@ -86,6 +91,35 @@
{{ render_checkbox_field(form.application.form.empty_pages_are_a_change) }}
<span class="pure-form-message-inline">{{ _('When a request returns no content, or the HTML does not contain any text, is this considered a change?') }}</span>
</div>
{#- Installing on a phone otherwise means typing this instance's URL, sub-path and
all, on a phone keyboard. Only shown over HTTPS because that is a hard
requirement for installing a PWA - offering it over plain http would just
send people to a page with no Install option and no explanation why.
FORCE_MOBILE_HTTPS_STYLE_MANIFEST overrides the detection. -#}
{% if pwa_https_mode() %}
<div class="pure-control-group" id="pwa-install">
<img id="pwa-install-qr"
src="{{ url_for('pwa.install_qrcode') }}"
alt="{{ _('QR code linking to this changedetection.io instance') }}"
width="104" height="104">
<div id="pwa-install-text">
<strong><i data-feather="smartphone"></i> {{ _('Install on your phone') }}</strong>
{#- Icon and bolded button label are passed in as placeholders rather than
baked into the msgid: it keeps raw HTML out of the .po files, and lets a
translator move the menu glyph to wherever the verb lands in their
language. |safe is needed for the substituted markup - both values are
ours, neither is user input. -#}
{#- The &nbsp; lives in the placeholder and the msgid has no space before
%(icon)s, so the glyph can't be orphaned onto the next line away from
the word it belongs to. -#}
<span class="pure-form-message-inline">
{{ _('Scan this, then use your phone browser\'s menu%(icon)s and choose "%(action)s".', icon=menu_icon, action=install_action)|safe }}<br>
{{ _('On Android you can then share any web page straight to this install from the share sheet.') }}<br>
{{ _('On iPhone the app installs but Safari has no share sheet support.') }}
</span>
</div>
</div>
{% endif %}
{% if form.requests.proxy %}
<div>
<br>
@@ -121,6 +155,7 @@
{{ render_field(form.requests.form.workers) }}
{% set worker_info = get_worker_status_info() %}
<span class="pure-form-message-inline">{{ _('Number of concurrent workers to process watches. More workers = faster processing but higher memory usage.') }}<br>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('Currently running:') }} <strong>{{ worker_info.count }}</strong> {{ _('operational') }} {{ worker_info.type }} {{ _('workers') }}{% if worker_info.active_workers > 0 %} ({{ worker_info.active_workers }} {{ _('actively processing') }}){% endif %}.</span>
</div>
<div class="pure-control-group">
@@ -135,6 +170,7 @@
{{ render_field(form.requests.form.default_ua) }}
<span class="pure-form-message-inline">
{{ _('Applied to all requests.') }}<br><br>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('Note: Simply changing the User-Agent often does not defeat anti-robot technologies, it\'s important to consider') }} <a href="https://changedetection.io/tutorial/what-are-main-types-anti-robot-mechanisms">{{ _('all of the ways that the browser is detected') }}</a>.
</span>
</div>
@@ -202,13 +238,14 @@ nav
<div class="pure-control-group">
{{ render_checkbox_field(form.application.form.api_access_token_enabled) }}
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
<div class="pure-form-message-inline">{{ _('Restrict API access limit by using') }} <code>x-api-key</code> {{ _('header - required for the Chrome Extension to work') }}</div><br>
<div class="pure-form-message-inline"><br>{{ _('API Key') }} <span id="api-key">{{api_key}}</span>
<span style="display:none;" id="api-key-copy" >{{ _('copy') }}</span>
</div>
</div>
<div class="pure-control-group">
<a href="{{url_for('settings.settings_reset_api_key')}}" class="pure-button button-small button-cancel">{{ _('Regenerate API key') }}</a>
<button type="submit" formmethod="post" formnovalidate formaction="{{url_for('settings.settings_reset_api_key')}}" class="cdio-btn button-small button-cancel">{{ _('Regenerate API key') }}</button>
</div>
<div class="pure-control-group">
<h4>{{ _('Chrome Extension') }}</h4>
@@ -224,6 +261,31 @@ nav
</a>
</p>
</div>
{% if pwa_https_mode() %}
<div class="pure-control-group" id="pwa-install">
<img id="pwa-install-qr"
src="{{ url_for('pwa.install_qrcode') }}"
alt="{{ _('QR code linking to this changedetection.io instance') }}"
width="104" height="104">
<div id="pwa-install-text">
<strong><i data-feather="smartphone"></i> {{ _('Install on your phone') }}</strong>
{#- Icon and bolded button label are passed in as placeholders rather than
baked into the msgid: it keeps raw HTML out of the .po files, and lets a
translator move the menu glyph to wherever the verb lands in their
language. |safe is needed for the substituted markup - both values are
ours, neither is user input. -#}
{#- The &nbsp; lives in the placeholder and the msgid has no space before
%(icon)s, so the glyph can't be orphaned onto the next line away from
the word it belongs to. -#}
<span class="pure-form-message-inline">
{{ _('Scan this, then use your phone browser\'s menu%(icon)s and choose "%(action)s".', icon=menu_icon, action=install_action)|safe }}<br>
{{ _('On Android you can then share any web page straight to this install from the share sheet.') }}<br>
{{ _('On iPhone the app installs but Safari has no share sheet support.') }}
</span>
</div>
</div>
{% endif %}
</div>
<div class="tab-pane-inner" id="rss">
<div class="pure-control-group">
@@ -281,11 +343,16 @@ nav
</div>
<div class="pure-control-group">
{{ render_checkbox_field(form.application.form.ui.form.favicons_enabled, class="") }}
<span class="pure-form-message-inline">{{ _('Enable or Disable Favicons next to the watch list') }}</span>
<span class="pure-form-message-inline">{{ _('Enable or Disable Favicons next to the watch list') }}</span><br>
<span class="pure-form-message-inline">{{ _('Disabling this can speed up your page fetches because the favicon does not need to be fetched.') }}</span>
</div>
<div class="pure-control-group">
{{ render_checkbox_field(form.application.form.ui.use_page_title_in_list) }}
</div>
<div class="pure-control-group">
{{ render_checkbox_field(form.application.form.ui.use_share_watch) }}
<span class="pure-form-message-inline">{{ _('This copies your watch to a temporary hosted database and returns a link to make sharing/copying the watch to other people') }}</span>
</div>
<div class="pure-control-group">
{{ render_field(form.application.form.pager_size) }}
<span class="pure-form-message-inline">{{ _('Number of items per page in the watch overview list, 0 to disable.') }}</span>
@@ -57,8 +57,13 @@
<div class="stab-overview-cta">
{% if llm_config and llm_config.get('model') %}
<span class="stab-configured-badge">&#10003; {{ _('AI / LLM configured:') }} {{ llm_config.get('model') }}</span>
{% if not llm_env_configured %}
<button type="button" class="cdio-btn" data-stab-goto="provider">
<i data-feather="settings"></i> {{ _('Edit provider') }} &rarr;
</button>
{% endif %}
{% else %}
<button type="button" class="pure-button pure-button-primary" data-stab-goto="provider">
<button type="button" class="cdio-btn pure-button-primary" data-stab-goto="provider">
<i data-feather="settings"></i> {{ _('Configure AI Provider') }} &rarr;
</button>
{% endif %}
@@ -113,7 +118,7 @@
<div class="pure-control-group">
<label for="llm-provider">{{ _('Provider') }}</label>
<select id="llm-provider" onchange="llmOnProviderChange(this.value)">
<select id="llm-provider" onchange="llmOnProviderChange(this.value)"{% if llm_provider_locked %} disabled{% endif %}>
<option value="">— {{ _('select a provider') }} —</option>
<option value="anthropic">Anthropic</option>
<option value="gemini">Google (Gemini)</option>
@@ -125,8 +130,13 @@
</div>
<div class="pure-control-group">
{{ render_field(form.llm.form.api_key) }}
<span class="pure-form-message-inline" id="llm-key-hint"></span>
{#- Disabled once a key is stored: a disabled input is not submitted at all, so there is
no way for an autofilled or mistyped value to reach the save path and overwrite a
working key. "Remove provider" is the only way to change it. -#}
{{ render_field(form.llm.form.api_key, disabled=llm_provider_locked) }}
<span class="pure-form-message-inline" id="llm-key-hint">
{%- if llm_provider_locked %}{{ _('Saved. Use "Remove provider" below to change the provider or key.') }}{% endif -%}
</span>
</div>
<div class="pure-control-group" id="llm-base-group" style="display:none">
{{ render_field(form.llm.form.api_base) }}
@@ -148,7 +158,7 @@
<div class="pure-control-group" id="llm-fetch-group" style="display:none">
<label></label>
<button type="button" id="llm-fetch-btn" class="pure-button button-xsmall" onclick="llmFetchModels()"
<button type="button" id="llm-fetch-btn" class="cdio-btn button-xsmall" onclick="llmFetchModels()"
style="background:#27ae60;color:#fff;border:none;">
&#8635; {{ _('Load available models') }}
</button>
@@ -164,7 +174,11 @@
<div class="pure-control-group">
{{ render_field(form.llm.form.model,
placeholder=_("Enter API key and click 'Load available models'")) }}
placeholder=_("Type a model name or choose from available models"),
aria_describedby="llm-model-help") }}
<span class="pure-form-message-inline" id="llm-model-help">
{{ _('You can edit the model name directly or load available models above. Click Save to apply your change. Leave the API key blank to keep your saved key.') }}
</span>
</div>
{% if llm_config and llm_config.get('model') %}
@@ -174,12 +188,17 @@
&#10003; {{ _('AI / LLM configured:') }} {{ llm_config.get('model') }}
</span>
&nbsp;
<button type="button" class="cdio-btn button-xsmall"
onclick="const field = document.getElementById('{{ form.llm.form.model.id }}'); field.focus(); field.select();">
{{ _('Change model') }}
</button>
&nbsp;
{# data-method="POST" tells modal.js to POST with the CSRF token instead of
navigating — GET previously allowed <img>-based CSRF wipe (GHSA-g36r-fm2p-87xm).
Stays as <a> because we're inside the outer settings <form> — nested forms are
invalid HTML, so modal.js builds a body-level hidden form for the POST. #}
<a href="{{ url_for('settings.llm.llm_clear') }}"
class="pure-button button-xsmall"
class="cdio-btn button-xsmall"
style="background:#c0392b;color:#fff;"
data-method="POST"
data-requires-confirm
@@ -188,10 +207,10 @@
data-confirm-message="<p>{{ _('This will remove your saved AI provider, model, and API key.') }}</p>"
data-confirm-button="{{ _('Remove') }}"
data-cancel-button="{{ _('Cancel') }}">
&#10005; {{ _('Remove') }}
&#10005; {{ _('Remove provider') }}
</a>
&nbsp;
<button type="button" id="llm-test-btn" class="pure-button button-xsmall" onclick="llmRunTest()"
<button type="button" id="llm-test-btn" class="cdio-btn button-xsmall" onclick="llmRunTest()"
style="background:#2980b9;color:#fff;border:none;">
&#9654; {{ _('Test connection') }}
</button>
@@ -207,7 +226,7 @@
<label style="color:#888; font-size:0.85em;">{{ _('Cache') }}</label>
{# See comment above on data-method="POST"+modal.js (GHSA-g36r-fm2p-87xm). #}
<a href="{{ url_for('settings.llm.llm_clear_summary_cache') }}"
class="pure-button button-xsmall"
class="cdio-btn button-xsmall"
style="background:#7f8c8d;color:#fff;"
data-method="POST"
data-requires-confirm
@@ -246,7 +265,7 @@
{{ render_field(form.llm.form.change_summary_default) }}
<span class="pure-form-message-inline">
{{ _('Used for all watches unless overridden by the watch or its tag/group.') }}
&nbsp;<a href="#" class="pure-button button-small" onclick="var t=document.getElementById('llm-change_summary_default'); if(!t.value && t.placeholder) t.value=t.placeholder; return false;">{{ _('Modify default prompt') }}</a>
&nbsp;<a href="#" class="cdio-btn button-small" onclick="var t=document.getElementById('llm-change_summary_default'); if(!t.value && t.placeholder) t.value=t.placeholder; return false;">{{ _('Modify default prompt') }}</a>
</span>
</div>
@@ -577,7 +596,10 @@
if (mult.trim()) params.set('local_token_multiplier', mult.trim());
try {
const resp = await fetch('{{ url_for("settings.llm.llm_test") }}?' + params);
const resp = await fetch('{{ url_for("settings.llm.llm_test") }}?' + params, {
method: 'POST',
headers: {'X-CSRFToken': csrftoken}
});
const data = await resp.json();
if (data.ok) {
result.style.cssText = 'display:block; background:rgba(39,174,96,0.08); border:1px solid rgba(39,174,96,0.3); border-radius:5px; padding:0.6em 0.85em; font-size:0.88em; line-height:1.45;';
+3 -2
View File
@@ -6,7 +6,7 @@ from loguru import logger
from changedetectionio.blueprint.tags.colour import safe_css_colour
from changedetectionio.store import ChangeDetectionStore
from changedetectionio.flask_app import login_optionally_required
from changedetectionio.llm.evaluator import get_llm_config as _get_llm_config
from changedetectionio.llm.evaluator import get_llm_config as _get_llm_config, is_llm_features_disabled
def construct_blueprint(datastore: ChangeDetectionStore):
@@ -62,7 +62,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
return redirect(url_for('tags.tags_overview_page'))
@tags_blueprint.route("/mute/<uuid_str:uuid>", methods=['GET'])
@tags_blueprint.route("/mute/<uuid_str:uuid>", methods=['POST'])
@login_optionally_required
def mute(uuid):
tag = datastore.data['settings']['application']['tags'].get(uuid)
@@ -230,6 +230,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
output = render_template("edit-tag.html",
extra_form_content=included_content,
extra_tab_content=form.extra_tab_content() if form.extra_tab_content() else None,
llm_features_disabled=is_llm_features_disabled(),
matching_watches=matching_watches,
settings_application=datastore.data['settings']['application'],
**template_args
@@ -29,7 +29,9 @@
<li class="tab" id=""><a href="#general">{{ _('General') }}</a></li>
{# Always shown, like the watch edit page: with no provider configured the pane
explains how to set one up (and carries the AI switches through on save). #}
{%- if not llm_features_disabled %}
<li class="tab"><a href="#ai-llm">{{ _('AI / LLM') }}</a></li>
{%- endif -%}
<li class="tab"><a href="#filters-and-triggers">{{ _('Filters & Triggers') }}</a></li>
{% if extra_tab_content %}
<li class="tab"><a href="#extras_tab">{{ extra_tab_content }}</a></li>
@@ -91,11 +93,11 @@
</script>
</fieldset>
</div>
{%- if not llm_features_disabled %}
<div class="tab-pane-inner" id="ai-llm">
{% include "edit/include_llm_intent.html" %}
</div>
{%- endif -%}
<div class="tab-pane-inner" id="filters-and-triggers">
{# TRANSLATORS: CJK fonts lack native italics; allow substitution with conventional local styling. dennis-ignore: W303 #}
<p>{{ _('These settings are <strong><i>added</i></strong> to any existing watch configurations.')|safe }}</p>
@@ -30,13 +30,13 @@ html[data-darkmode="true"] .watch-tag-list.tag-{{ class_name }} {
<form class="pure-form" action="{{ url_for('tags.form_tag_add') }}" method="POST" id="new-watch-form">
<input type="hidden" name="csrf_token" value="{{ csrf_token() }}" >
<fieldset>
<legend>{{ _('Add a new organisational tag') }}</legend>
<legend>{{ _('Add a new organisational tag / group name.') }}</legend>
<div id="watch-add-wrapper-zone">
<div>
{{ render_simple_field(form.name, placeholder=_("Watch group / tag")) }}
</div>
<div>
{{ render_simple_field(form.save_button, title=_("Save") ) }}
{{ render_simple_field(form.save_button, title=_("Save"), label=false ) }}
</div>
</div>
<br>
@@ -64,20 +64,18 @@ html[data-darkmode="true"] .watch-tag-list.tag-{{ class_name }} {
<td colspan="3">{{ _('No website organisational tags/groups configured') }}</td>
</tr>
{% endif %}
<form method="POST" style="display: inline;">
<input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
{% for uuid, tag in available_tags %}
{#-{{ loop.cycle('pure-table-odd', 'pure-table-even') }}-#}
<tr id="{{ uuid }}" class="">
<td class="watch-controls">
<a class="link-mute state-{{'on' if tag.notification_muted else 'off'}}" href="{{url_for('tags.mute', uuid=tag.uuid)}}" aria-label="{{ _('Mute notifications') }}" title="{{ _('Mute notifications') }}"><i data-feather="{{ 'bell-off' if tag.notification_muted else 'bell' }}" class="icon icon-mute"></i></a>
<button formaction="{{url_for('tags.mute', uuid=tag.uuid)}}" type="submit" class="bare-btn link-mute state-{{'on' if tag.notification_muted else 'off'}}" aria-label="{{ _('Mute notifications') }}" title="{{ _('Mute notifications') }}"><i data-feather="{{ 'bell-off' if tag.notification_muted else 'bell' }}" class="icon icon-mute"></i></button>
</td>
<td class="watch-count">{{ "{:,}".format(tag_count[uuid]) if uuid in tag_count else 0 }}</td>
<td class="title-col inline"> <a href="{{url_for('watchlist.index', tag=uuid) }}" class="watch-tag-list tag-{{ tag.title|sanitize_tag_class }}">{{ tag.title }}</a></td>
<td>
<button formaction="{{ url_for('ui.form_watch_checknow', tag=uuid) }}" type="submit" class="cdio-btn"><i data-feather="refresh-cw"></i>{{ _('Recheck') }}</button>
<a class="cdio-btn" href="{{ url_for('tags.form_tag_edit', uuid=uuid) }}">{{ _('Edit') }}</a>
<form method="POST" action="{{ url_for('ui.form_watch_checknow', tag=uuid) }}" style="display: inline;">
<input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
<button type="submit" class="cdio-btn"><i data-feather="refresh-cw"></i>{{ _('Recheck') }}</button>
</form>
<a class="cdio-btn cdio-btn--danger"
href="{{ url_for('tags.delete', uuid=uuid) }}"
data-method="POST"
@@ -100,6 +98,7 @@ html[data-darkmode="true"] .watch-tag-list.tag-{{ class_name }} {
</td>
</tr>
{% endfor %}
</form>
</tbody>
</table>
</div>
+6 -4
View File
@@ -1,7 +1,7 @@
import time
import threading
from blinker import signal
from flask import Blueprint, request, redirect, url_for, flash, render_template, session, current_app
from flask import Blueprint, request, redirect, url_for, flash, render_template, session, current_app, abort
from flask_babel import gettext
from loguru import logger
@@ -406,10 +406,12 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, worker_pool,
return redirect(url_for('watchlist.index'))
@ui_blueprint.route("/share-url/<uuid_str:uuid>", methods=['GET'])
@ui_blueprint.route("/share-url/<uuid_str:uuid>", methods=['POST'])
@login_optionally_required
def form_share_put_watch(uuid):
if not datastore.data['settings']['application']['ui'].get('use_share_watch'):
abort(403, description="Access denied")
"""Given a watch UUID, upload the info and return a share-link
the share-link can be imported/added"""
import requests
@@ -455,7 +457,7 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, worker_pool,
return redirect(url_for('watchlist.index'))
@ui_blueprint.route("/language/auto-detect", methods=['GET'])
@ui_blueprint.route("/language/auto-detect", methods=['POST'])
def delete_locale_language_session_var_if_it_exists():
"""Clear the session locale preference to auto-detect from browser Accept-Language header"""
if 'locale' in session:
+266 -65
View File
@@ -172,29 +172,45 @@ def construct_blueprint(datastore: ChangeDetectionStore):
prompt = ''
return jsonify({'prompt': prompt})
@diff_blueprint.route("/diff/<uuid_str:uuid>/llm-summary", methods=['GET'])
@login_optionally_required
def diff_llm_summary(uuid):
# ── AI change summary ────────────────────────────────────────────────────────────────────
# Generation is slow (an LLM round-trip), so it does not happen in the request thread:
#
# POST /diff/<uuid>/llm-summary start it (or return an already-cached summary)
# GET /diff/<uuid>/llm-summary poll for the result
#
# POST is deliberate: starting a generation spends tokens and marks the watch viewed, which
# must not be reachable from a bare cross-site GET (an <img src=...> would do it). Flask-WTF's
# CSRFProtect covers every non-GET route, and static/js/csrf.js puts the token on the header
# of every jQuery non-GET call, so the browser side needs nothing extra.
#
# The GET poll stays side-effect free: it reads the cache and the in-flight registry and never
# starts work, which is what makes it safe to leave as a GET.
def _summary_request_context(uuid):
"""Resolve everything both routes need, without reading snapshots or building a diff.
Returns (context_dict, None) or (None, (json_body, http_status)) when the request cannot
be served at all. Kept cheap so the poll route stays cheap: the cache key does not depend
on the diff text, only on the version pair, the effective prompt, the diff prefs and the
model - so a poll never has to re-run difflib over two snapshots.
"""
Generate (or return cached) an AI summary of the diff between two snapshots.
Called via AJAX from the diff page when no cached summary exists.
Returns JSON: {"summary": "...", "error": null} or {"summary": null, "error": "..."}
"""
import difflib
from flask import jsonify
from changedetectionio.llm.evaluator import (
DiffPrefs, build_summary_cache_prompt, get_effective_summary_prompt, get_llm_settings,
resolve_llm_timeout,
)
try:
watch = datastore.data['watching'][uuid]
except KeyError:
return jsonify({'summary': None, 'error': 'Watch not found'}), 404
return None, ({'summary': None, 'error': 'Watch not found', 'status': 'error'}, 404)
llm_cfg = datastore.data.get('settings', {}).get('application', {}).get('llm', {})
if not llm_cfg.get('model'):
return jsonify({'summary': None, 'error': 'LLM not configured'}), 400
return None, ({'summary': None, 'error': 'LLM not configured', 'status': 'error'}, 400)
dates = list(watch.history.keys())
if len(dates) < 2:
return jsonify({'summary': None, 'error': 'Not enough history'}), 400
return None, ({'summary': None, 'error': 'Not enough history', 'status': 'error'}, 400)
# Default baseline for the watchlist "Summary" link (when no explicit from_version
# is requested) is configurable at Settings > AI. Default 'second_last_version'
@@ -204,18 +220,50 @@ def construct_blueprint(datastore: ChangeDetectionStore):
default_from = best_from if best_from else dates[-2]
else:
default_from = dates[-2]
from_version = request.args.get('from_version', default_from)
to_version = request.args.get('to_version', dates[-1])
from changedetectionio.llm.evaluator import DiffPrefs
prefs = DiffPrefs.from_request_args(request.args)
all_changes = prefs.all_changes
ignore_whitespace = prefs.ignore_whitespace
show_removed = prefs.show_removed
show_added = prefs.show_added
prefs = DiffPrefs.from_request_args(request.args)
from_version = request.args.get('from_version', default_from)
to_version = request.args.get('to_version', dates[-1])
settings = get_llm_settings(datastore)
# Diff-pref flags + system prompt + active model are part of the cache key
# so prompt or model changes bust the cache.
cache_prompt = build_summary_cache_prompt(
effective_prompt=get_effective_summary_prompt(watch, datastore),
max_summary_tokens=settings.max_summary_tokens,
prefs=prefs,
model=settings.model,
)
return {
'watch': watch,
'dates': dates,
'prefs': prefs,
'from_version': from_version,
'to_version': to_version,
'cache_prompt': cache_prompt,
# Same granularity as the on-disk cache filename (version pair + prompt hash), so an
# in-flight job is deduplicated exactly when it would write to the same cache entry.
'job_key': f"{uuid}|{from_version}|{to_version}|"
f"{watch._llm_summary_prompt_hash(cache_prompt)}",
# How long this deployment is prepared to wait for the model on one call. The
# browser must not give up before this, or it reports a failure for a job that is
# still running and will still write its summary - a local Ollama/vLLM endpoint
# gets 1800s here against the client's old fixed 180s. See _pending_reply().
'llm_timeout': resolve_llm_timeout(llm_cfg),
}, None
def _build_summary_diff(ctx):
"""Build the diff text to summarise. The expensive half - POST only."""
import difflib
watch = ctx['watch']
prefs = ctx['prefs']
from_version, to_version = ctx['from_version'], ctx['to_version']
def _prep(text):
"""Optionally normalise whitespace on each line before diffing."""
if not ignore_whitespace:
if not prefs.ignore_whitespace:
return text.splitlines()
return [' '.join(line.split()) for line in text.splitlines()]
@@ -225,13 +273,13 @@ def construct_blueprint(datastore: ChangeDetectionStore):
def _apply_filters(diff_text):
"""Strip +/- lines the user has hidden in the UI so the LLM matches what they see."""
if show_removed and show_added:
if prefs.show_removed and prefs.show_added:
return diff_text
out = []
for line in diff_text.splitlines():
if line.startswith('-') and not show_removed:
if line.startswith('-') and not prefs.show_removed:
continue
if line.startswith('+') and not show_added:
if line.startswith('+') and not prefs.show_added:
continue
out.append(line)
return '\n'.join(out)
@@ -240,15 +288,16 @@ def construct_blueprint(datastore: ChangeDetectionStore):
from_text = watch.get_history_snapshot(timestamp=from_version)
to_text = watch.get_history_snapshot(timestamp=to_version)
except Exception as e:
return jsonify({'summary': None, 'error': f'Could not read snapshots: {e}'}), 500
return None, None, ({'summary': None, 'error': f'Could not read snapshots: {e}',
'status': 'error'}, 500)
if all_changes:
if prefs.all_changes:
# Build sequential diffs for every intermediate snapshot between from and to
# so the LLM sees the full timeline of changes, not just start→end
sorted_dates = sorted(dates)
sorted_dates = sorted(ctx['dates'])
try:
start_idx = sorted_dates.index(from_version)
end_idx = sorted_dates.index(to_version)
end_idx = sorted_dates.index(to_version)
except ValueError:
start_idx, end_idx = 0, len(sorted_dates) - 1
@@ -269,34 +318,94 @@ def construct_blueprint(datastore: ChangeDetectionStore):
else:
diff_text = _apply_filters(_make_unified_diff(from_text, to_text))
if not diff_text.strip():
return jsonify({'summary': None, 'error': 'No differences found'})
return diff_text, to_text, None
from changedetectionio.llm.evaluator import (
summarise_change, get_effective_summary_prompt, build_summary_cache_prompt,
is_global_token_budget_exceeded, get_global_token_budget_month,
LLMInputTooLargeError,
)
def _mark_viewed(uuid):
import time
datastore.set_last_viewed(uuid, int(time.time()))
# Diff-pref flags + system prompt + active model are part of the cache key
# so prompt or model changes bust the cache.
from changedetectionio.llm.evaluator import get_llm_settings
_ls = get_llm_settings(datastore)
_max_summary_tokens = _ls.max_summary_tokens
_llm_model = _ls.model
cache_prompt = build_summary_cache_prompt(
effective_prompt=get_effective_summary_prompt(watch, datastore),
max_summary_tokens=_max_summary_tokens,
prefs=prefs,
model=_llm_model,
)
# Slack on top of the model's own deadline: the worker still has to come back through
# litellm and write the summary to the on-disk cache after the last token.
SUMMARY_DEADLINE_MARGIN = 15
def _pending_reply(ctx):
"""The 202 body, carrying the point in time after which the summary is not coming.
The client cannot work this out for itself. It knows neither the configured timeout
(300s for a cloud provider, 1800s for a local endpoint) nor when the job started - which
is often long before *this* browser asked, because another tab, or this same tab before
a reload, already kicked it off. So the deadline is anchored to the job's real start and
is the same answer no matter who polls or when.
Two forms of the same instant, because neither alone is enough:
timeout_at absolute epoch seconds - stable across polls, and what to display.
expires_in seconds from now - what to actually count down with, since a browser clock
that is minutes off would otherwise expire the wait immediately.
A job still queued behind another generation has not started its timeout yet, so it
measures from now and its deadline keeps moving until a worker picks it up.
"""
import time
from changedetectionio.llm.summary_jobs import summary_jobs
started = summary_jobs.run_started_at(ctx['job_key'])
now = time.time()
timeout_at = int((started or now) + ctx['llm_timeout'] + SUMMARY_DEADLINE_MARGIN)
return {
'summary': None,
'error': None,
'status': 'pending',
'timeout_at': timeout_at,
'expires_in': max(0, timeout_at - int(now)),
}
@diff_blueprint.route("/diff/<uuid_str:uuid>/llm-summary", methods=['POST'])
@login_optionally_required
def diff_llm_summary(uuid):
"""
Start (or serve from cache) an AI summary of the diff between two snapshots.
Returns JSON:
200 {"summary": "...", "cached": true, "status": "done"} already generated
202 {"summary": null, "status": "pending", generating, poll the GET
"timeout_at": epoch, "expires_in": secs} (when to stop waiting)
4xx/5xx {"summary": null, "error": "...", "status": "error"} cannot be generated
"""
from flask import jsonify
ctx, err = _summary_request_context(uuid)
if err:
body, code = err
return jsonify(body), code
watch = ctx['watch']
from_version, to_version = ctx['from_version'], ctx['to_version']
cache_prompt, job_key = ctx['cache_prompt'], ctx['job_key']
# Check cache — keyed by version pair + prompt hash (invalidates if prompt changes)
cached = watch.get_llm_diff_summary(from_version, to_version, prompt=cache_prompt)
if cached:
import time
datastore.set_last_viewed(uuid, int(time.time()))
return jsonify({'summary': cached, 'error': None, 'cached': True})
_mark_viewed(uuid)
return jsonify({'summary': cached, 'error': None, 'cached': True, 'status': 'done'})
from changedetectionio.llm.summary_jobs import SummaryJobFailed, summary_jobs
# A failure from a previous attempt that no poll collected (e.g. the user navigated away).
# Deliver it once rather than silently starting a fresh attempt on the same broken config.
stored = summary_jobs.take_error(job_key)
if stored:
code, message = stored
return jsonify({'summary': None, 'error': message, 'status': 'error'}), code
if summary_jobs.is_pending(job_key):
logger.info(f"AI summary already in progress for {uuid} ({from_version}->{to_version}), "
f"returning pending - not starting a second generation")
_mark_viewed(uuid)
return jsonify(_pending_reply(ctx)), 202
from changedetectionio.llm.evaluator import (
LLMInputTooLargeError, get_global_token_budget_month,
is_global_token_budget_exceeded, summarise_change,
)
# Check global monthly token budget before making an LLM call
if is_global_token_budget_exceeded(datastore):
@@ -311,27 +420,119 @@ def construct_blueprint(datastore: ChangeDetectionStore):
used=f'{used:,}',
),
'budget_exceeded': True,
'status': 'error',
}), 429
try:
summary = summarise_change(watch, datastore, diff=diff_text, current_snapshot=to_text)
except LLMInputTooLargeError as e:
return jsonify({'summary': None, 'error': str(e)}), 400
except Exception as e:
logger.error(f"LLM summary generation failed for {uuid}: {e}")
return jsonify({'summary': None, 'error': _clean_litellm_error(e)}), 500
diff_text, to_text, err = _build_summary_diff(ctx)
if err:
body, code = err
return jsonify(body), code
if not summary:
return jsonify({'summary': None, 'error': 'LLM returned empty summary'})
if not diff_text.strip():
return jsonify({'summary': None, 'error': 'No differences found', 'status': 'error'})
try:
def _job():
"""Runs on the summary_jobs pool - no request context available in here."""
import time as _time
_started = _time.time()
logger.info(f"AI summary generation started for {uuid} ({from_version}->{to_version}), "
f"{len(diff_text)} chars of diff")
try:
summary = summarise_change(watch, datastore, diff=diff_text, current_snapshot=to_text)
except LLMInputTooLargeError as e:
raise SummaryJobFailed(str(e), http_status=400)
except Exception as e:
logger.error(f"LLM summary generation failed for {uuid}: {e}")
raise SummaryJobFailed(_clean_litellm_error(e), http_status=500)
if not summary:
raise SummaryJobFailed('LLM returned empty summary', http_status=200)
# Persisted before the job is marked done, so a poll can never see "nothing running,
# nothing cached" and start paying for the same summary a second time.
watch.save_llm_diff_summary(summary, from_version, to_version, prompt=cache_prompt)
except Exception as e:
logger.warning(f"Could not cache llm summary for {uuid}: {e}")
logger.info(f"AI summary generation finished for {uuid} in "
f"{_time.time() - _started:.1f}s ({len(summary)} chars)")
import time
datastore.set_last_viewed(uuid, int(time.time()))
return jsonify({'summary': summary, 'error': None, 'cached': False})
# Re-read the cache. Between the read above and here we have read two snapshots and run
# difflib, which is long enough for a job started by another tab to have finished and
# written the summary - and submitting now would pay the LLM for it a second time.
cached = watch.get_llm_diff_summary(from_version, to_version, prompt=cache_prompt)
if cached:
return jsonify({'summary': cached, 'error': None, 'cached': True, 'status': 'done'})
def _announce(_key):
"""Tell any connected browser the job is done so it does not have to poll.
A blinker signal rather than a direct emit: the socket server subscribes to it when
realtime is enabled, and when it is disabled nothing is listening and this is a no-op.
The payload is only an identifier - the client fetches the summary through the normal
authenticated poll route, so the text never goes out over a broadcast channel.
"""
from blinker import signal
signal('llm_summary_ready').send(
watch_uuid=uuid, from_version=from_version, to_version=to_version,
)
if not summary_jobs.submit(job_key, _job, on_settled=_announce):
# Another request submitted the same key while we were building the diff. The registry
# refused ours, so nothing was sent to the LLM twice.
logger.info(f"AI summary for {uuid} was already started by a concurrent request, "
f"returning pending")
summary_jobs.submit(job_key, _job, on_settled=_announce)
_mark_viewed(uuid)
return jsonify(_pending_reply(ctx)), 202
@diff_blueprint.route("/diff/<uuid_str:uuid>/llm-summary", methods=['GET'])
@login_optionally_required
def diff_llm_summary_poll(uuid):
"""
Poll for a summary started by the POST above. Read-only: never calls the LLM.
Returns JSON:
200 {"summary": "...", "cached": true, "status": "done"} ready
202 {"summary": null, "status": "pending", still generating
"timeout_at": epoch, "expires_in": secs}
200 {"summary": null, "status": "idle"} nothing running - re-POST
4xx/5xx {"summary": null, "error": "...", "status": "error"} the job failed
"""
from flask import jsonify
ctx, err = _summary_request_context(uuid)
if err:
body, code = err
return jsonify(body), code
cached = ctx['watch'].get_llm_diff_summary(
ctx['from_version'], ctx['to_version'], prompt=ctx['cache_prompt']
)
if cached:
_mark_viewed(uuid)
return jsonify({'summary': cached, 'error': None, 'cached': True, 'status': 'done'})
from changedetectionio.llm.summary_jobs import summary_jobs
stored = summary_jobs.take_error(ctx['job_key'])
if stored:
code, message = stored
return jsonify({'summary': None, 'error': message, 'status': 'error'}), code
if summary_jobs.is_pending(ctx['job_key']):
return jsonify(_pending_reply(ctx)), 202
# Re-read the cache before giving up. The job can finish in the window between the read
# above and the is_pending check - it writes the summary and *then* clears the flag - so
# 'idle' here would send the client back to POST for a summary already sitting on disk.
cached = ctx['watch'].get_llm_diff_summary(
ctx['from_version'], ctx['to_version'], prompt=ctx['cache_prompt']
)
if cached:
_mark_viewed(uuid)
return jsonify({'summary': cached, 'error': None, 'cached': True, 'status': 'done'})
# Genuinely nothing: the process restarted mid-generation. Tell the client to start again
# rather than poll forever.
return jsonify({'summary': None, 'error': None, 'status': 'idle'})
@diff_blueprint.route("/diff/<uuid_str:uuid>/processor-data", methods=['GET'])
@login_optionally_required
@@ -70,6 +70,8 @@ def construct_blueprint(datastore: ChangeDetectionStore):
# use the same as when it is triggered, but then override it with the form test values
n_object = NotificationContextData({
'watch_url': request.form.get('window_url', "https://changedetection.io"),
# Falls back to watch_url when the watch has no 'Link to Open' set
'watch_open_url': watch.open_link_override or request.form.get('window_url', "https://changedetection.io"),
'notification_urls': notification_urls
})
+2 -1
View File
@@ -104,7 +104,8 @@ def construct_blueprint(datastore: ChangeDetectionStore):
output = render_template("preview.html",
capabilities=capabilities,
content=content,
current_diff_url=watch['url'],
current_diff_label=watch.label,
current_diff_url=watch.open_link,
current_version=timestamp,
extra_stylesheets=extra_stylesheets,
extra_title=f" - {gettext('Diff')} - {watch.label} @ {timestamp}",
@@ -10,6 +10,7 @@
<fieldset>
<div class="pure-control-group">
{{ _('This will remove version history (snapshots) for ALL watches, but keep your list of URLs!') }} <br />
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('You may like to use the') }} <strong>{{ _('BACKUP') }}</strong> {{ _('link first.') }}<br />
</div>
<br />
@@ -24,6 +25,7 @@
size="10"
/>
<span class="pure-form-message-inline"
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
>{{ _('Type in the word') }} <strong>{{ _('clear') }}</strong> {{ _('to confirm that you understand.') }}</span
>
</div>
@@ -8,7 +8,7 @@
{% endif %}
const highlight_submit_ignore_url="{{url_for('ui.ui_edit.highlight_submit_ignore_url', uuid=uuid)}}";
const watch_url= {{watch_a.link|tojson}};
const watch_url= {{watch_a.open_link|tojson}};
// Initial scroll position: if set, scroll to this line number in #difference on page load
const initialScrollToLineNumber = {{ initial_scroll_line_number|default('null') }};
@@ -18,8 +18,10 @@
<script src="https://cdn.jsdelivr.net/npm/piexifjs@1.0.6/piexif.min.js"></script>
<script src="{{url_for('static_content', group='js', filename='snippet-to-image.js')}}"></script>
<script src="{{url_for('static_content', group='js', filename='diff-overview.js')}}" defer></script>
<script src="{{url_for('static_content', group='js', filename='llm-summary.js')}}" defer></script>
<div id="diff-header">
<div id="settings">
<form class="pure-form " action="{{ url_for("ui.ui_diff.diff_history_page", uuid=uuid) }}" method="GET" id="diff-form">
<fieldset class="diff-fieldset">
@@ -49,6 +51,10 @@
{#<button type="submit" class="pure-button pure-button-primary reset-margin">Go</button>#}
{% endif %}
</fieldset>
{# Collapses the diff options below into a popover. Hidden until
diff-overview.js takes over the fieldset, so with scripting off the
options stay inline and reachable. #}
<button type="button" id="diff-filters-toggle" class="pure-button" aria-expanded="false" aria-controls="diff-style">{{ _('Filters') }} &#x25be;</button>
<fieldset id="diff-style">
<span>
<label for="diffWords" class="pure-checkbox">
@@ -66,17 +72,20 @@
<label for="changesOnly" class="pure-checkbox" id="label-diff-changes">
<input type="checkbox" id="changesOnly" name="changesOnly" {% if diff_prefs.changesOnly %}checked=""{% endif %}> {{ _('Same/non-changed') }}</label>
</span>
{# TRANSLATORS: 'Removed' is a noun label for a diff display filter (toggles removed lines), not a verb phrase or status badge #}
<span>
<label for="removed" class="pure-checkbox" id="label-diff-removed">
<input type="checkbox" id="removed" name="removed" {% if diff_prefs.removed %}checked=""{% endif %}> {{ _('Removed') }}</label>
<input type="checkbox" id="removed" name="removed" {% if diff_prefs.removed %}checked=""{% endif %}> {{ pgettext('diff filter', 'Removed') }}</label>
</span>
{# TRANSLATORS: 'Added' is a noun label for a diff display filter (toggles added lines), not a verb phrase or status badge #}
<span>
<label for="added" class="pure-checkbox" id="label-diff-added">
<input type="checkbox" id="added" name="added" {% if diff_prefs.added %}checked=""{% endif %}> {{ _('Added') }}</label>
<input type="checkbox" id="added" name="added" {% if diff_prefs.added %}checked=""{% endif %}> {{ pgettext('diff filter', 'Added') }}</label>
</span>
{# TRANSLATORS: 'Replaced' is a noun label for a diff display filter (toggles replaced lines), not a verb phrase or status badge #}
<span>
<label for="replaced" class="pure-checkbox" id="label-diff-replaced">
<input type="checkbox" id="replaced" name="replaced" {% if diff_prefs.replaced %}checked=""{% endif %}> {{ _('Replaced') }}</label>
<input type="checkbox" id="replaced" name="replaced" {% if diff_prefs.replaced %}checked=""{% endif %}> {{ pgettext('diff filter', 'Replaced') }}</label>
</span>
{%- if llm_configured -%}
<span>
@@ -87,9 +96,13 @@
</fieldset>
{%- if versions|length >= 2 -%}
<div id="keyboard-nav">
{# In the sticky bar the prompt and the two words are hidden and
only the arrows show, so each link repeats its label as the
accessible name and the tooltip - without them a screen reader
is left reading the arrow glyph itself. #}
<strong>{{ _('Keyboard:') }} </strong>
<a href="" class="pure-button pure-button-primary" id="btn-previous"> &larr; {{ _('Previous') }}</a>
&nbsp; <a class="pure-button pure-button-primary" id="btn-next" href=""> &rarr; {{ _('Next') }}</a>
<a href="" class="pure-button pure-button-primary" id="btn-previous" title="{{ _('Previous') }}" aria-label="{{ _('Previous') }}"> &larr; <span class="keyboard-nav-label">{{ _('Previous') }}</span></a>
<a class="pure-button pure-button-primary" id="btn-next" href="" title="{{ _('Next') }}" aria-label="{{ _('Next') }}"> &rarr; <span class="keyboard-nav-label">{{ _('Next') }}</span></a>
</div>
{%- endif -%}
</form>
@@ -105,11 +118,13 @@
{% if last_error_text %}<li class="tab" id="error-text-tab"><a href="#error-text">{{ _('Error Text') }}</a></li> {% endif %}
{% if last_error_screenshot %}<li class="tab" id="error-screenshot-tab"><a href="#error-screenshot">{{ _('Error Screenshot') }}</a></li> {% endif %}
<li class="tab" id="text-tab"><a href="#text">{{ _('Text') }}</a></li>
<li class="tab" id="screenshot-tab"><a href="#screenshot">{{ _('Current screenshot') }}</a></li>
<li class="tab" id="screenshot-tab"><a href="#screenshot">{{ _('Screenshot') }}</a></li>
<li class="tab" id="extract-tab"><a href="{{ url_for('ui.ui_diff.diff_history_page_extract_GET', uuid=uuid)}}">{{ _('Extract Data') }}</a></li>
</ul>
</div>
</div>{# /diff-header #}
<div id="diff-ui">
<div class="tab-pane-inner" id="error-text">
<div class="snapshot-age error">{{watch_a.error_text_ctime|format_seconds_ago}} {{ _('seconds ago.') }}</div>
@@ -206,34 +221,26 @@ $(function () {
var ignoreWhitespace = $('#ignoreWhitespace').is(':checked') ? 1 : 0;
var showRemoved = $('#removed').is(':checked') ? 1 : 0;
var showAdded = $('#added').is(':checked') ? 1 : 0;
$.getJSON(summaryUrl, {
from_version: fromVersion,
to_version: toVersion,
all_changes: llmAllChanges,
ignore_whitespace: ignoreWhitespace,
removed: showRemoved,
added: showAdded,
})
.done(function (data) {
if (data.summary) {
$area.find('.llm-diff-summary-text')
.removeClass('llm-diff-summary-loading')
.text(data.summary);
$area.removeAttr('data-pending');
} else if (data.error) {
showLlmError(data.error);
} else {
$area.remove();
}
})
.fail(function (xhr) {
var resp = xhr.responseJSON;
if (resp && resp.error) {
showLlmError(resp.error);
} else {
showLlmError('AI summary request failed (HTTP ' + xhr.status + ').');
}
});
window.llmSummary.fetch({
url: summaryUrl,
uuid: "{{ uuid }}",
params: {
from_version: fromVersion,
to_version: toVersion,
all_changes: llmAllChanges,
ignore_whitespace: ignoreWhitespace,
removed: showRemoved,
added: showAdded,
},
onDone: function (summary) {
$area.find('.llm-diff-summary-text')
.removeClass('llm-diff-summary-loading')
.text(summary);
$area.find('.llm-diff-summary-prompt').remove();
$area.removeAttr('data-pending');
},
onError: showLlmError,
});
});
</script>
{% endif %}
@@ -78,6 +78,7 @@
<fieldset>
<div class="pure-control-group">
{{ render_field(form.url, placeholder="https://...", required=true, class="m-d") }}
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
<div class="pure-form-message">{{ _('Some sites use JavaScript to create the content, for this you should') }} <a href="https://github.com/dgtlmoon/changedetection.io/wiki/Fetching-pages-with-WebDriver">{{ _('use the Chrome/WebDriver Fetcher') }}</a></div>
<div class="pure-form-message">{{ _('Variables are supported in the URL') }} (<a href="https://github.com/dgtlmoon/changedetection.io/wiki/Handling-variables-in-the-watched-URL">{{ _('help and examples here') }}</a>).</div>
</div>
@@ -100,6 +101,23 @@
{{ render_field(form.title, class="m-d", placeholder=watch.label) }}
<span class="pure-form-message-inline">{{ _('Automatically uses the page title if found, you can also use your own title/description here') }}</span>
</div>
<div class="pure-control-group">
<details class="form-wrapper">
<summary>{{ _('Extra Page Title and Link options') }}</summary>
<br>
<div class=" border-fieldset">
<div class="pure-control-group">
{{ render_field(form.link_to_open, placeholder="https://...", class="m-d") }}
<span class="pure-form-message-inline">{{ _('Optional - links in the list, history and notifications open this URL instead. Does not change the URL being checked.') }}</span>
</div>
<div class="pure-control-group">
{{ render_ternary_field(form.use_page_title_in_list) }}
</div>
</div>
</details>
</div>
<div class="pure-control-group time-between-check border-fieldset">
{{ render_checkbox_field(form.time_between_check_use_default, class="use-default-timecheck") }}
@@ -119,13 +137,6 @@
</div>
<br>
</div>
<div class="pure-control-group">
{{ render_checkbox_field(form.filter_failure_notification_send) }}
<span class="pure-form-message-inline">
{{ _('Sends a notification when the filter can no longer be seen on the page, good for knowing when the page changed and your filter will not work anymore.') }}
</span>
</div>
<div class="pure-control-group">
{{ render_field(form.history_snapshot_max_length, class="history_snapshot_max_length") }}
<span class="pure-form-message-inline">{{ _('Limit collection of history snapshots for each watch to this number of history items.') }}
@@ -133,9 +144,7 @@
{{ _('Set to empty to use system settings default') }}
</span>
</div>
<div class="pure-control-group">
{{ render_ternary_field(form.use_page_title_in_list) }}
</div>
</fieldset>
</div>
@@ -250,7 +259,7 @@ Math: {{ 1 + 1 }}") }}
<div class="flex-wrapper" >
<div id="browser-steps-ui" class="noselect">
<div class="noselect" id="browsersteps-selector-wrapper" style="width: 100%">
<div class="noselect" id="browsersteps-selector-wrapper">
<span class="loader" >
<span id="browsersteps-click-start">
<h2 >{{ _('Click here to Start') }}</h2>
@@ -276,6 +285,7 @@ Math: {{ 1 + 1 }}") }}
{% else %}
<p>
<strong>{{ _('Sorry, this functionality only works with fetchers that support interactive Javascript (so far only Playwright based fetchers)') }}<br>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('You need to') }} <a href="#request">{{ _('Set the fetch method') }}</a> {{ _('to one that supports interactive Javascript.') }}</strong>
</p>
{% endif %}
@@ -295,6 +305,12 @@ Math: {{ 1 + 1 }}") }}
</span>
</div>
{% endif %}
<div class="pure-control-group">
{{ render_checkbox_field(form.filter_failure_notification_send) }}
<span class="pure-form-message-inline">
{{ _('Sends a notification when the filter can no longer be seen on the page, good for knowing when the page changed and your filter will not work anymore.') }}
</span>
</div>
<div class="field-group" id="notification-field-group">
{% if has_default_notification_urls %}
<div class="inline-warning">
@@ -319,6 +335,7 @@ Math: {{ 1 + 1 }}") }}
<div class="pure-form-message-inline">
<p id="verify-state-text">{{ _('Use the verify (✓) button to test if a condition passes against the current snapshot.') }}</p>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('Read a quick tutorial about') }} <a href="https://changedetection.io/tutorial/conditional-actions-web-page-changes">{{ _('using conditional web page changes here') }}</a>.<br>
</div>
</div>
@@ -342,6 +359,7 @@ Math: {{ 1 + 1 }}") }}
{{ _('Use the preview page to see your filters and triggers highlighted.') }}
</li>
<li>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('Some sites use JavaScript to create the content, for this you should') }} <a href="https://github.com/dgtlmoon/changedetection.io/wiki/Fetching-pages-with-WebDriver">{{ _('use the Chrome/WebDriver Fetcher') }}</a>
</li>
</ul>
@@ -356,7 +374,9 @@ Math: {{ 1 + 1 }}") }}
{{ render_checkbox_field(form.filter_text_replaced) }}
{{ render_checkbox_field(form.filter_text_removed) }}
<span class="pure-form-message-inline">{{ _('Note: Depending on the length and similarity of the text on each line, the algorithm may consider an <strong>addition</strong> instead of <strong>replacement</strong> for example.')|safe }}</span><br>
<span class="pure-form-message-inline">&nbsp;{{ _('So it\'s always better to select') }} <strong>{{ _('Added') }}</strong>+<strong>{{ _('Replaced') }}</strong> {{ _('when you\'re interested in new content.') }}</span><br>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
<span class="pure-form-message-inline">&nbsp;{{ _('So it\'s always better to select') }} <strong>{{ pgettext('diff filter', 'Added') }}</strong>+<strong>{{ pgettext('diff filter', 'Replaced') }}</strong> {{ _('when you\'re interested in new content.') }}</span><br>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
<span class="pure-form-message-inline">&nbsp;{{ _('When content is merely moved in a list, it will also trigger an') }} <strong>{{ _('addition') }}</strong>, {{ _('consider enabling') }} <code><strong>{{ _('Only trigger when unique lines appear') }}</strong></code></span>
</fieldset>
<fieldset class="pure-control-group">
@@ -429,6 +449,7 @@ Math: {{ 1 + 1 }}") }}
{% if capabilities.supports_screenshots and capabilities.supports_xpath_element_data %}
{% if visual_selector_data_ready %}
<span class="pure-form-message-inline" id="visual-selector-heading">
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('The Visual Selector tool lets you select the') }} <i>{{ _('text') }}</i> {{ _('elements that will be used for the change detection. It automatically fills-in the filters in the "CSS/JSONPath/JQ/XPath Filters" box of the') }} <a href="#filters-and-triggers">{{ _('Filters & Triggers') }}</a> {{ _('tab. Use') }} <strong>{{ _('Shift+Click') }}</strong> {{ _('to select multiple items.') }}
</span>
@@ -467,6 +488,7 @@ Math: {{ 1 + 1 }}") }}
{% else %}
<p>
<strong>{{ _('Sorry, this functionality only works with fetchers that support Javascript and screenshots (such as playwright etc).') }}<br>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('You need to') }} <a href="#request">{{ _('Set the fetch method') }}</a> {{ _('to one that supports Javascript and screenshots.') }}</strong>
</p>
{% endif %}
@@ -507,6 +529,26 @@ Math: {{ 1 + 1 }}") }}
<td>{{ _('Server type reply') }}</td>
<td>{{ watch.get('remote_server_reply') }}</td>
</tr>
<tr>
<td><code>capabilities.supports_xpath_element_data</code></td>
<td>{{ _('Yes') if capabilities.supports_xpath_element_data else _('No') }}</td>
</tr>
<tr>
<td><code>capabilities.supports_screenshots</code></td>
<td>{{ _('Yes') if capabilities.supports_screenshots else _('No') }}</td>
</tr>
<tr>
<td><code>capabilities.supports_visual_selector</code></td>
<td>{{ _('Yes') if capabilities.supports_visual_selector else _('No') }}</td>
</tr>
<tr>
<td><code>supports_browser_steps</code></td>
<td>{{ _('Yes') if capabilities.supports_browser_steps else _('No') }}</td>
</tr>
<tr>
<td><code>visual_selector_data_ready</code></td>
<td>{{ _('Yes') if visual_selector_data_ready else _('No') }}</td>
</tr>
{% if not llm_features_disabled and settings_application.get('llm', {}).get('model') %}
<tr>
<td>{{ _('AI tokens (last check)') }}</td>
@@ -14,10 +14,30 @@ from changedetectionio.auth_decorator import login_optionally_required
# bulk actions so a filtered view and the actions taken on it always agree.
from changedetectionio.blueprint.watchlist import filters as wl_filters
from changedetectionio.blueprint.watchlist.row_context import watch_row_context
# The manifest points share_target at this page, so an Android share lands in index()
from changedetectionio.blueprint.pwa.service import is_share, pwa_preset_url
def construct_blueprint(datastore: ChangeDetectionStore, update_q, queuedWatchMetaData):
watchlist_blueprint = Blueprint('watchlist', __name__, template_folder="templates")
@watchlist_blueprint.route("/toggle", methods=['POST'])
@login_optionally_required
def toggle():
op = request.args.get('op')
uuid = request.args.get('uuid')
watch = datastore.data['watching'].get(uuid)
if not watch:
flash(_('Watch not found'), 'error')
else:
if op == 'pause':
watch.toggle_pause()
elif op == 'mute':
watch.toggle_mute()
watch.commit()
return redirect(url_for('watchlist.index', tag=request.args.get('tag')))
@watchlist_blueprint.route("/", methods=['GET'])
@login_optionally_required
def index():
@@ -36,17 +56,6 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, queuedWatchMe
if request.args.get('rss'):
return redirect(url_for('rss.feed', tag=active_tag_uuid))
op = request.args.get('op')
if op:
uuid = request.args.get('uuid')
if op == 'pause':
datastore.data['watching'][uuid].toggle_pause()
elif op == 'mute':
datastore.data['watching'][uuid].toggle_mute()
datastore.data['watching'][uuid].commit()
return redirect(url_for('watchlist.index', tag = active_tag_uuid))
# Sort by last_changed and add the uuid which is usually the key..
sorted_watches = []
active_processor = request.args.get('processor', '').strip()
@@ -85,6 +94,17 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, queuedWatchMe
sorted_watches.append(watch)
form = forms.quickWatchForm(request.form)
# A share from the Android share sheet lands here (see pwa_preset_url). Prefill only -
# the operator still picks a mode and presses Watch.
if not form.url.data and is_share(request.args):
form.url.data = pwa_preset_url(request.args)
if not form.url.data:
# Sharing a photo, or a note with no link in it, otherwise just opens an
# ordinary-looking empty page and leaves the user wondering. Gated on
# is_share so tapping the home screen icon (same URL, no params) says nothing.
flash(_('Nothing shareable was found - no web page link in what you shared.'), 'notice')
page = request.args.get(get_page_parameter(), type=int, default=1)
total_count = len(sorted_watches)
@@ -30,10 +30,10 @@ def watch_row_context(datastore, active_tag_uuid=None, queued_uuids=None):
return {
'active_tag_uuid': active_tag_uuid,
# Kept 0 (rather than any_watches_have_processor_by_name) while the price column is
# disabled — it also decides cols_required on the page, so page and row must agree or
# a pushed row ends up with a different <td> count than the table header.
'any_has_restock_price_processor': 0,
# Also decides cols_required on the page, so page and row must agree or a pushed row
# ends up with a different <td> count than the table header — which is why it is
# computed here and not passed separately by the page.
'any_has_restock_price_processor': datastore.any_watches_have_processor_by_name("restock_diff"),
'datastore': datastore,
'has_proxies': datastore.proxy_list,
'processor_descriptions': processors.get_processor_descriptions(),
@@ -11,8 +11,12 @@
{%- set checking_now = is_checking_now(watch) -%}
{%- set history_n = watch.history_n -%}
{%- set favicon = watch.get_favicon_filename() -%}
{# Resolved once per row: each read re-validates the URL and, for a Jinja2 templated
URL, builds a fresh sandboxed environment - see validate_url.py #}
{%- set open_link = watch.open_link -%}
{%- set error_texts = watch.compile_error_texts(has_proxies=has_proxies) -%}
{%- set system_use_url_watchlist = datastore.data['settings']['application']['ui'].get('use_page_title_in_list') -%}
{%- set system_use_share = datastore.data['settings']['application']['ui'].get('use_share_watch') -%}
{# Class settings mirrored in changedetectionio/static/js/realtime.js for the frontend #}
{# loop.cycle('pure-table-odd', 'pure-table-even'),#}
{%- set row_classes = [
@@ -35,18 +39,25 @@
<td class="inline checkbox-uuid" ><div><input name="uuids" type="checkbox" value="{{ watch.uuid}} " >{# <span class="counter-i">{{ loop.index+pagination.skip }}</span>#}</div></td>
<td class="inline watch-controls">
<div>
<a class="ajax-op state-off pause-toggle" data-op="pause" aria-label="{{ _('Pause checks') }}" title="{{ _('Pause checks') }}" href="{{url_for('watchlist.index', op='pause', uuid=watch.uuid, tag=active_tag_uuid)}}"><i data-feather="pause" class="icon icon-pause"></i></a>
<a class="ajax-op state-on pause-toggle" data-op="pause" style="display: none" aria-label="{{ _('UnPause checks') }}" title="{{ _('UnPause checks') }}" href="{{url_for('watchlist.index', op='pause', uuid=watch.uuid, tag=active_tag_uuid)}}"><i data-feather="play" class="icon icon-unpause"></i></a>
<a class="ajax-op state-off mute-toggle" data-op="mute" aria-label="{{ _('Mute notification') }}" title="{{ _('Mute notification') }}" href="{{url_for('watchlist.index', op='mute', uuid=watch.uuid, tag=active_tag_uuid)}}"><i data-feather="bell" class="icon icon-mute"></i></a>
<a class="ajax-op state-on mute-toggle" data-op="mute" style="display: none" aria-label="{{ _('UnMute notification') }}" title="{{ _('UnMute notification') }}" href="{{url_for('watchlist.index', op='mute', uuid=watch.uuid, tag=active_tag_uuid)}}"><i data-feather="bell-off" class="icon icon-mute"></i></a>
{%- set pause_action = url_for('watchlist.toggle', op='pause', uuid=watch.uuid, tag=active_tag_uuid) -%}
{%- set mute_action = url_for('watchlist.toggle', op='mute', uuid=watch.uuid, tag=active_tag_uuid) -%}
<button type="submit" formmethod="post" formaction="{{ pause_action }}" class="bare-btn ajax-op state-off pause-toggle" data-op="pause" aria-label="{{ _('Pause checks') }}" title="{{ _('Pause checks') }}"><i data-feather="pause" class="icon icon-pause"></i></button>
<button type="submit" formmethod="post" formaction="{{ pause_action }}" class="bare-btn ajax-op state-on pause-toggle" data-op="pause" style="display: none" aria-label="{{ _('UnPause checks') }}" title="{{ _('UnPause checks') }}"><i data-feather="play" class="icon icon-unpause"></i></button>
<button type="submit" formmethod="post" formaction="{{ mute_action }}" class="bare-btn ajax-op state-off mute-toggle" data-op="mute" aria-label="{{ _('Mute notification') }}" title="{{ _('Mute notification') }}"><i data-feather="bell" class="icon icon-mute"></i></button>
<button type="submit" formmethod="post" formaction="{{ mute_action }}" class="bare-btn ajax-op state-on mute-toggle" data-op="mute" style="display: none" aria-label="{{ _('UnMute notification') }}" title="{{ _('UnMute notification') }}"><i data-feather="bell-off" class="icon icon-mute"></i></button>
</div>
</td>
<td class="watch-processor">
{%- if watch['processor'] and watch['processor'] in processor_badge_texts -%}
<a href="{{ url_for('watchlist.index', tag=active_tag_uuid, processor=watch['processor']) }}" class="processor-badge processor-badge-{{ watch['processor'] }}{{ ' active' if active_processor == watch['processor'] else '' }}"
title="{{ processor_descriptions.get(watch['processor'], watch['processor']) }}">{{ processor_badge_texts[watch['processor']] }}</a>
{%- endif -%}
</td>
<td class="title-col inline">
<div class="grid-wrapper">
{% if 'favicons_enabled' not in ui_settings or ui_settings['favicons_enabled'] %}
<div class="favicon">
<a target="_blank" rel="noopener" href="{{ watch.link.replace('source:','') }}">
<a target="_blank" rel="noopener" href="{{ open_link }}">
{# Intersection Observer lazy loading: store real URL in data-src, load only when visible in viewport #}
<img alt="Favicon thumbnail"
class="favicon lazy-favicon"
@@ -63,16 +74,13 @@
</div>
{% endif %}
<div class="watch-text-info">
{%- if watch['processor'] and watch['processor'] in processor_badge_texts -%}
<a href="{{ url_for('watchlist.index', tag=active_tag_uuid, processor=watch['processor']) }}" class="processor-badge processor-badge-{{ watch['processor'] }}{{ ' active' if active_processor == watch['processor'] else '' }}" title="{{ processor_descriptions.get(watch['processor'], watch['processor']) }}">{{ processor_badge_texts[watch['processor']] }}</a>
{%- endif -%}
<span class="watch-title">
{% if system_use_url_watchlist or watch.get('use_page_title_in_list') %}
{{ watch.label }}
{% else %}
{{ watch.get('title') or watch.link }}
{% endif %}
<a class="external" target="_blank" rel="noopener" href="{{ watch.link.replace('source:','') }}">&nbsp;</a>
<a class="external" target="_blank" rel="noopener" href="{{ open_link }}" title="{{ open_link }}"><i data-feather="external-link"></i></a>
</span>
{%- for watch_tag_uuid, watch_tag in datastore.get_all_tags_for_watch(watch['uuid']).items() -%}
@@ -81,20 +89,27 @@
<div class="error-text" style="display:none;">{{ error_texts|safe }}</div>
{%- if watch['processor'] == 'text_json_diff' -%}
{%- if watch['has_ldjson_price_data'] and not watch['track_ldjson_price_data'] -%}
<div class="ldjson-price-track-offer">Switch to Restock & Price watch mode? <a href="{{url_for('price_data_follower.accept', uuid=watch.uuid)}}" class="pure-button button-xsmall">Yes</a> <a href="{{url_for('price_data_follower.reject', uuid=watch.uuid)}}" class="">No</a></div>
<div class="ldjson-price-track-offer">Switch to Restock & Price watch mode? <button type="submit" formmethod="post" formaction="{{url_for('price_data_follower.accept', uuid=watch.uuid)}}" class="pure-button button-xsmall">Yes</button> <button type="submit" formmethod="post" formaction="{{url_for('price_data_follower.reject', uuid=watch.uuid)}}" class="bare-btn bare-btn--link">No</button></div>
{%- endif -%}
{%- endif -%}
</div>
<div class="status-icons">
<a class="link-spread" href="{{url_for('ui.form_share_put_watch', uuid=watch.uuid)}}"><img src="{{url_for('static_content', group='images', filename='spread.svg')}}" class="status-icon icon icon-spread" title="{{ _('Create a link to share watch config with others') }}" ></a>
{%- if system_use_share -%} <button type="submit" formmethod="post" formaction="{{url_for('ui.form_share_put_watch', uuid=watch.uuid)}}" class="bare-btn link-spread"><img src="{{url_for('static_content', group='images', filename='spread.svg')}}" class="status-icon icon icon-spread" title="{{ _('Create a link to share watch config with others') }}" ></button>{%- endif -%}
{%- set effective_fetcher = watch.get_fetch_backend if watch.get_fetch_backend != "system" else system_default_fetcher -%}
{%- if effective_fetcher and ("html_webdriver" in effective_fetcher or "html_" in effective_fetcher or "extra_browser_" in effective_fetcher) -%}
{{ effective_fetcher|fetcher_status_icons }}
{%- endif -%}
{%- if watch.is_pdf -%}<img class="status-icon" src="{{url_for('static_content', group='images', filename='pdf-icon.svg')}}" alt="Converting PDF to text" >{%- endif -%}
{%- if watch.is_pdf -%}<img class="status-icon" src="{{url_for('static_content', group='images', filename='pdf-icon.svg')}}" title="Converting PDF to text" alt="Converting PDF to text" >{%- endif -%}
{%- if watch.has_browser_steps -%}<img class="status-icon status-browsersteps" src="{{url_for('static_content', group='images', filename='steps.svg')}}" alt="Browser Steps is enabled" >{%- endif -%}
</div>
</div>
</td>
{%- if any_has_restock_price_processor -%}
<td class="restock-and-price col-restock-price">
{%- if watch['processor'] == 'restock_diff' -%}
{#- @todo - this could be injected somehow watch.extra_row_info or something -#}
<div class="restock-info-wrap">
@@ -135,23 +150,17 @@
{%- endif -%}
</div>
{%- endif -%}
</div>
</td>
{#
{%- if any_has_restock_price_processor -%}
<td class="restock-and-price">
</td>
{%- endif -%}
#}
{#last_checked becomes fetch-start-time#}
<td class="last-checked" data-timestamp="{{ watch.last_checked }}" data-fetchduration={{ watch.fetch_time }} data-eta_complete="{{ watch.last_checked+watch.fetch_time }}" data-label="{{ _('Checked') }}">
<td class="last-checked" data-timestamp="{{ watch.last_checked }}" data-fetchduration={{ watch.fetch_time }} data-eta_complete="{{ watch.last_checked+watch.fetch_time }}" data-label="{{ pgettext('timestamp', 'Checked') }}">
<div class="spinner-wrapper" style="display:none;" >
<span class="spinner"></span><span class="status-text">&nbsp;{{ watch['__check_status'] or _('Checking now') }}</span>
</div>
<span class="innertext">{{watch|format_last_checked_time|safe}}</span>
</td>
<td class="last-changed" data-timestamp="{{ watch.last_changed }}" data-label="{{ _('Changed') }}">
<td class="last-changed" data-timestamp="{{ watch.last_changed }}" data-label="{{ pgettext('timestamp', 'Changed') }}">
<span class="innertext">{%- if watch.history_n >=2 and watch.last_changed >0 -%}
{{watch.last_changed|format_timestamp_timeago}}
{%- else -%}
@@ -163,7 +172,9 @@
<div>
{%- set target_attr = ' target="' ~ watch.uuid ~ '"' if datastore.data['settings']['application']['ui'].get('open_diff_in_new_tab') else '' -%}
<a href="" class="already-in-queue-button recheck cdio-btn cdio-btn--primary cdio-btn--sm" style="display: none;" disabled="disabled"><i data-feather="clock"></i>{{ _('Queued') }}</a>
<a href="{{ url_for('ui.form_watch_checknow', uuid=watch.uuid, tag=request.args.get('tag')) }}" data-op='recheck' class="ajax-op recheck cdio-btn cdio-btn--primary cdio-btn--sm"><i data-feather="refresh-cw"></i>{{ _('Recheck') }}</a>
<button type="submit" formmethod="post"
formaction="{{ url_for('ui.form_watch_checknow', uuid=watch.uuid, tag=request.args.get('tag')) }}"
data-op='recheck' class="ajax-op recheck cdio-btn cdio-btn--primary cdio-btn--sm"><i data-feather="refresh-cw"></i>{{ _('Recheck') }}</button>
<a href="{{ url_for('ui.ui_edit.edit_page', uuid=watch.uuid, tag=active_tag_uuid)}}#general" class="cdio-btn cdio-btn--primary cdio-btn--sm">{{ _('Edit') }}</a>
<a href="{{ url_for('ui.ui_diff.diff_history_page', uuid=watch.uuid)}}" {{target_attr}} class="cdio-btn cdio-btn--primary cdio-btn--sm history-link ai-history-btn" style="display: none;" data-uuid="{{ watch.uuid }}" data-summary-url="{{ url_for('ui.ui_diff.diff_llm_summary', uuid=watch.uuid) }}" data-processor-data-url="{{ url_for('ui.ui_diff.diff_history_page_processor_data', uuid=watch.uuid) }}"><span class="btn-label-history">{{ _('History') }}</span><span class="btn-label-summary">&#x2728; {{ _('Summary') }}</span></a>
<a href="{{ url_for('ui.ui_preview.preview_page', uuid=watch.uuid)}}" {{target_attr}} class="cdio-btn cdio-btn--primary cdio-btn--sm preview-link" style="display: none;">{{ _('Preview') }}</a>
@@ -6,6 +6,7 @@
] -%}
{%- from '_helpers.html' import render_simple_field, render_field, render_nolabel_field, sort_by_title -%}
<script src="{{url_for('static_content', group='js', filename='jquery-3.6.0.min.js')}}"></script>
<script src="{{url_for('static_content', group='js', filename='llm-summary.js')}}" defer></script>
<script src="{{url_for('static_content', group='js', filename='watch-overview.js')}}" defer></script>
<script src="{{url_for('static_content', group='js', filename='restock-price-graph.js')}}" defer></script>
<script src="{{url_for('static_content', group='js', filename='modal.js')}}"></script>
@@ -140,8 +141,10 @@ window.watchOverviewI18n = {
priceLow: {{ _('Currently low')|tojson }},
priceTypical: {{ _('Currently typical')|tojson }},
priceHigh: {{ _('Currently high')|tojson }},
cheaperThan: {{ _('cheaper than %s% of tracked prices')|tojson }},
pricierThan: {{ _('more expensive than %s% of tracked prices')|tojson }},
{# TRANSLATORS: {pct} is a percentage including the % sign, e.g. "80%" #}
cheaperThan: {{ _('cheaper than {pct} of tracked prices')|tojson }},
{# TRANSLATORS: {pct} is a percentage including the % sign, e.g. "80%" #}
pricierThan: {{ _('more expensive than {pct} of tracked prices')|tojson }},
typicalNote: {{ _('around the usual price')|tojson }},
avgLabel: {{ _('avg')|tojson }}
};
@@ -153,8 +156,8 @@ window.watchOverviewI18n = {
<form class="pure-form" action="{{ url_for('ui.ui_views.form_quick_watch_add') }}" method="POST" id="new-watch-form">
<div id="add-watch-url-row">
<input type="hidden" name="csrf_token" value="{{ csrf_token() }}">
<label for="url"><strong>{{ _('Web page URL') }}</strong></label><br>
{{ render_nolabel_field(form.url, placeholder="https://...", required=true, class="pure-input-1") }}
<label for="url"><strong>{{ _('Web page URL') }}</strong></label>
{{ render_nolabel_field(form.url, placeholder="www.website.com/page.html", required=true, class="pure-input-1", span_wrap=false) }}
<button type="submit" id="add-watch-go" class="pure-button pure-button-primary">{{ _('Watch') }}</button>
</div>
{% if llm_configured %}
@@ -193,13 +196,6 @@ window.watchOverviewI18n = {
{% endfor %}
</template>
{% endif %}
{# "Select all matching" banner — shown by watch-overview.js when the whole
visible page is selected but more matching rows exist on other pages. #}
<div id="select-all-banner" class="select-all-banner" style="display: none;"
data-select-all-tmpl="{{ _('All %(page)s on this page are selected.') }}"
data-select-all-action="{{ _('Select all %(total)s matching') }}"
data-all-selected-tmpl="{{ _('All %(total)s matching are selected.') }}"
data-clear-action="{{ _('Clear selection') }}"></div>
<div id="stats_row">
<div class="left">{%- if watches|length >= pagination.per_page -%}{{ pagination.info }}{%- endif -%}
@@ -208,15 +204,11 @@ window.watchOverviewI18n = {
</div>
</div>
{%- if search_q -%}<div id="search-result-info">{{ _('Searching') }} "<strong><i>{{search_q}}</i></strong>"</div>{%- endif -%}
{%- set sort_order = sort_order or 'asc' -%}
{%- set sort_attribute = sort_attribute or 'last_changed' -%}
{%- set pagination_page = request.args.get('page', 0) -%}
{%- set cols_required = 6 -%}
{%- set cols_required = 7 -%}
{%- if any_has_restock_price_processor -%}
{%- set cols_required = cols_required + 1 -%}
{%- endif -%}
@@ -287,6 +279,7 @@ window.watchOverviewI18n = {
</div>
<div id="checkbox-operations">
<button type="button" class="pure-button button-xsmall" id="check-cancel" style="background: var(--color-background-button-cancel); color: #333;"><i data-feather="x" ></i>&nbsp;{{ _('Close') }}</button>
<button type="button" class="pure-button button-secondary button-xsmall" id="check-invert"><i data-feather="repeat" ></i>&nbsp;{{ _('Invert') }}</button>
<button class="pure-button button-secondary button-xsmall" name="op" value="pause"><i data-feather="pause" ></i>&nbsp;{{ _('Pause') }}</button>
<button class="pure-button button-secondary button-xsmall" name="op" value="unpause"><i data-feather="play" ></i>&nbsp;{{ _('UnPause') }}</button>
@@ -321,9 +314,13 @@ window.watchOverviewI18n = {
data-confirm-title="{{ _('Delete Watches?') }}"
data-confirm-message="{{ _('<p><strong>Are you sure you want to delete the selected watches?</strong></p><p>This action cannot be undone.</p>') }}"
data-confirm-button="{{ _('Delete') }}"><i data-feather="trash" ></i>&nbsp;{{ _('Delete') }}</button>
<button type="button" class="pure-button button-xsmall" id="check-cancel" style="background: var(--color-background-button-cancel); color: #333;"><i data-feather="x" ></i>&nbsp;{{ _('Close') }}</button>
</div>
<div id="select-all-banner" class="select-all-banner" style="display: none;"
data-select-all-tmpl="{{ _('All %(page)s on this page are selected.') }}"
data-select-all-action="{{ _('Select all %(total)s matching') }}"
data-all-selected-tmpl="{{ _('All %(total)s matching are selected.') }}"
data-clear-action="{{ _('Clear selection') }}">
</div>
{%- set table_classes = [
'favicon-enabled' if 'favicons_enabled' not in ui_settings or ui_settings['favicons_enabled'] else 'favicon-not-enabled',
] -%}
@@ -340,28 +337,44 @@ window.watchOverviewI18n = {
<a class="{{ 'active '+link_order if sort_attribute == 'notification_muted' else 'inactive' }}" href="{{url_for('watchlist.index', sort='notification_muted', order=link_order, **active_filters)}}"><i data-feather="volume-2" style="width: 16px; height: 16px;"></i><span class='arrow {{link_order}}'></span></a>
</div>
</th>
<th><!-- processor/mode --></th>
<th id="th-webpage"><a class="{{ 'active '+link_order if sort_attribute == 'label' else 'inactive' }}" href="{{url_for('watchlist.index', sort='label', order=link_order, **active_filters)}}">{{ _('Web page') }} <span class='arrow {{link_order}}'></span></a></th>
{%- if any_has_restock_price_processor -%}
<th>{{ _('Restock & Price') }}</th>
<th class="col-restock-price">{{ _('Restock & Price') }}</th>
{%- endif -%}
<th id="h-lastchecked"><a class="{{ 'active '+link_order if sort_attribute == 'last_checked' else 'inactive' }}" href="{{url_for('watchlist.index', sort='last_checked', order=link_order, **active_filters)}}"><span class="hide-on-mobile">{{ _('Checked') }}</span><span class="hide-on-desktop">{{ _('Checked') }}</span> <span class='arrow {{link_order}}'></span></a></th>
<th id="h-lastchanged"><a class="{{ 'active '+link_order if sort_attribute == 'last_changed' else 'inactive' }}" href="{{url_for('watchlist.index', sort='last_changed', order=link_order, **active_filters)}}"><span class="hide-on-mobile">{{ _('Changed') }}</span><span class="hide-on-desktop">{{ _('Changed') }}</span> <span class='arrow {{link_order}}'></span></a></th>
{# TRANSLATORS: 'Checked' labels the last-checked time column in the watch overview list, not a checkbox state #}
<th id="h-lastchecked"><a class="{{ 'active '+link_order if sort_attribute == 'last_checked' else 'inactive' }}" href="{{url_for('watchlist.index', sort='last_checked', order=link_order, **active_filters)}}"><span class="hide-on-mobile">{{ pgettext('timestamp', 'Checked') }}</span><span class="hide-on-desktop">{{ pgettext('timestamp', 'Checked') }}</span> <span class='arrow {{link_order}}'></span></a></th>
{# TRANSLATORS: 'Changed' labels the last-changed time column in the watch overview list, not a status badge #}
<th id="h-lastchanged"><a class="{{ 'active '+link_order if sort_attribute == 'last_changed' else 'inactive' }}" href="{{url_for('watchlist.index', sort='last_changed', order=link_order, **active_filters)}}"><span class="hide-on-mobile">{{ pgettext('timestamp', 'Changed') }}</span><span class="hide-on-desktop">{{ pgettext('timestamp', 'Changed') }}</span> <span class='arrow {{link_order}}'></span></a></th>
<th class="empty-cell"></th>
</tr>
</thead>
<tbody>
{%- if not watches|length -%}
<tr>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
<td colspan="{{ cols_required }}" style="text-wrap: wrap;">{{ _('No web page change detection watches configured, please add a URL in the box above, or') }} <a href="{{ url_for('imports.import_page')}}" >{{ _('import a list') }}</a>.</td>
</tr>
{%- endif -%}
{%- set ns = namespace(any_in_page_had_restock=false) -%}
{%- for watch in (watches|sort(attribute=sort_attribute, reverse=sort_order == 'asc'))|pagination_slice(skip=pagination.skip) -%}
{%- include "watch-overview-single-row.html" -%}
{%- if watch['processor'] == 'restock_diff' -%}
{%- set ns.any_in_page_had_restock = true -%}
{%- endif -%}
{%- endfor -%}
</tbody>
</table>
</div>
{%- if not ns.any_in_page_had_restock %}
<style>
.col-restock-price {
display: none;
}
</style>
{%- endif %}
{{ pagination.links }}
</div>
</form>
@@ -9,6 +9,39 @@ from changedetectionio.content_fetchers.base import get_playwright_bypass_csp, m
from changedetectionio.jinja2_custom import render as jinja_render
from changedetectionio.validate_url import validate_fetch_url_async
def track_latest_navigation_response(page):
"""Record the latest main-frame document response seen on this page, and return the holder.
Idempotent on purpose - every navigation would otherwise add another 'response' listener, and
that event fires once per HTTP response (hundreds on a heavy page), so the callbacks are worth
not duplicating. One tracker per page is installed and then shared by the fetcher and by every
action_goto_url() call on it.
Returns a dict that holds {'response': <latest main-frame document response>}, or None if the
page does not support event listeners (the unit test stubs, mainly).
"""
if not hasattr(page, 'on'):
return None
existing = getattr(page, '_cdio_latest_navigation_response', None)
if existing is not None:
return existing
latest = {}
def _keep(response):
try:
if response.frame == page.main_frame and response.request.is_navigation_request():
latest['response'] = response
except Exception as e:
# Never let a bookkeeping listener break a fetch
logger.debug(f"Could not record navigation response: {e}")
page.on("response", _keep)
page._cdio_latest_navigation_response = latest
return latest
def browser_steps_get_valid_steps(browser_steps: list):
if browser_steps is not None and len(browser_steps):
valid_steps = list(filter(
@@ -145,8 +178,23 @@ class steppable_browser_interface():
# and private-IP SSRF possible via a browser step (GHSA-hm22-wg2m-35v4).
await validate_fetch_url_async(value)
# Chrome 153+ refuses to commit a navigation when an error status arrives with a
# zero-length body, so page.goto() raises net::ERR_HTTP_RESPONSE_CODE_FAILURE instead of
# handing back the response. The response was received fine, we just never get it as a
# return value, so fall back to the page's navigation-response tracker and hand that back -
# callers then report a real "Error - 404" instead of a raw net:: string.
navigation_response = track_latest_navigation_response(self.page)
now = time.time()
response = await self.page.goto(value, timeout=0, wait_until='load')
try:
response = await self.page.goto(value, timeout=0, wait_until='load')
except Exception as e:
if 'ERR_HTTP_RESPONSE_CODE_FAILURE' not in str(e) or not navigation_response:
raise
response = navigation_response['response']
logger.debug(f"Navigation was aborted by the browser (empty body on an error status), "
f"recovered status {response.status} from the response event")
logger.debug(f"Time to goto URL {time.time()-now:.2f}s")
return response
@@ -14,7 +14,11 @@ global_hookimpl = pluggy.HookimplMarker("changedetectionio")
def levenshtein_ratio_recent_history(watch, incoming_text=None):
try:
from Levenshtein import ratio, distance
# rapidfuzz (MIT) instead of Levenshtein (GPL-2.0-or-later), to keep the
# shipped deps free of strong copyleft. Indel.normalized_similarity is the
# exact equivalent of Levenshtein.ratio - Levenshtein.normalized_similarity is not.
from rapidfuzz.distance.Levenshtein import distance
from rapidfuzz.distance.Indel import normalized_similarity as ratio
k = list(watch.history.keys())
a = None
b = None
@@ -90,6 +90,7 @@ class Fetcher():
screenshot_format = None
status_code = None
webdriver_js_execute_code = None
worker_id = None
xpath_data = None
xpath_element_js = ""
@@ -120,6 +121,11 @@ class Fetcher():
if kwargs and 'lock_viewport_elements' in kwargs:
self.lock_viewport_elements = kwargs.get('lock_viewport_elements')
# Which async worker is driving this fetch, subclasses use it to keep per-worker browser
# state (profile dirs etc) apart, stays None when we're not called from a worker
if kwargs and 'worker_id' in kwargs:
self.worker_id = kwargs.get('worker_id')
@classmethod
def get_status_icon_data(cls):
@@ -296,6 +296,20 @@ class fetcher(Fetcher):
self.page = await context.new_page()
# Track the LATEST main-frame document response for the whole fetch, not just the one
# goto() returns. This app compares the text of the page the browser ends up on, and a
# site that gates with an interstitial (503/429 + meta-refresh) navigates to the real
# page *during* the extra_wait below - judging the fetch on the first response fails a
# watch whose content is present and fine. Same for plain client-side redirects.
# Shared with action_goto_url() so only one 'response' listener exists on the page.
from changedetectionio.browser_steps.browser_steps import track_latest_navigation_response
# Must be an identity check - the tracker hands back the same (initially empty, so
# falsy) dict the listener writes into, and `or {}` would quietly swap in a different
# one that never gets updated.
latest_navigation_response = track_latest_navigation_response(self.page)
if latest_navigation_response is None:
latest_navigation_response = {}
# Listen for all console events and handle errors
self.page.on("console", lambda msg: logger.debug(f"Playwright console: Watch URL: {url} {msg.type}: {msg.text} {msg.args}"))
@@ -336,6 +350,25 @@ class fetcher(Fetcher):
extra_wait = int(os.getenv("WEBDRIVER_DELAY_BEFORE_CONTENT_READY", 5)) + self.render_extract_delay
await self.page.wait_for_timeout(extra_wait * 1000)
# A meta-refresh or client-side redirect usually lands during that wait, so judge the
# fetch on the document we are actually about to extract rather than the first one.
latest = latest_navigation_response.get('response')
if latest is not None and latest is not response:
logger.debug(f"Page navigated again while waiting, judging the fetch on {latest.url} "
f"(status {latest.status}) instead of the first response for {url}")
response = latest
try:
self.headers = await response.all_headers()
except Exception as e:
logger.debug(f"Could not refresh headers from the final document: {e}")
# Don't extract while a navigation is mid-flight, that is what produces
# "Execution context was destroyed, most likely because of a navigation"
try:
await self.page.wait_for_load_state('load', timeout=extra_wait * 1000)
except Exception as e:
logger.debug(f"Page did not reach a settled load state, continuing anyway: {e}")
try:
self.status_code = response.status
except Exception as e:
+191 -47
View File
@@ -1,5 +1,4 @@
import asyncio
import gc
import json
import os
import websockets.exceptions
@@ -10,6 +9,7 @@ from loguru import logger
from changedetectionio.content_fetchers import SCREENSHOT_MAX_HEIGHT_DEFAULT, visualselector_xpath_selectors, \
SCREENSHOT_SIZE_STITCH_THRESHOLD, SCREENSHOT_DEFAULT_QUALITY, XPATH_ELEMENT_JS, INSTOCK_DATA_JS, \
SCREENSHOT_MAX_TOTAL_HEIGHT, FAVICON_FETCHER_JS
from changedetectionio import gc_debounce
from changedetectionio.content_fetchers.base import Fetcher, get_playwright_bypass_csp, manage_user_agent
from changedetectionio.content_fetchers.exceptions import PageUnloadable, Non200ErrorCodeReceived, EmptyReply, BrowserFetchTimedOut, \
BrowserConnectError
@@ -236,6 +236,7 @@ class fetcher(Fetcher):
async def quit(self, watch=None):
watch_uuid = watch.get('uuid') if watch else 'unknown'
closed_something = bool(getattr(self, 'page', None) or getattr(self, 'browser', None))
# Close page
try:
@@ -263,8 +264,16 @@ class fetcher(Fetcher):
logger.info(f"[{watch_uuid}] Cleanup puppeteer complete")
# Force garbage collection to release resources
gc.collect()
# Only collect if this call actually closed something.
#
# quit() runs twice per check - from run()'s finally, then again from the worker's
# safety net - and it sets self.page/self.browser to None in its own finally
# blocks. The second call therefore closes nothing, creates no garbage and breaks
# no cycles, but still paid for a full stop-the-world collection: measured at 88
# calls across 51 checks, roughly half of them reclaiming nothing. The pyppeteer
# page/connection/session graph is genuinely cyclic, so the first call still runs.
if closed_something:
gc_debounce.collect('puppeteer.quit')
async def fetch_page(self,
current_include_filters,
@@ -399,56 +408,192 @@ class fetcher(Fetcher):
# Enable Network domain to detect when first bytes arrive
await self.page._client.send('Network.enable')
# Now set up the frame navigation handlers
async def handle_frame_navigation(event=None):
# Wait n seconds after the frameStartedLoading, not from any frameStartedLoading/frameStartedNavigating
logger.debug(f"Frame navigated: {event}")
w = extra_wait - 2 if extra_wait > 4 else 2
logger.debug(f"Waiting {w} seconds before calling Page.stopLoading...")
await asyncio.sleep(w)
# Navigate (bounded), then wait the configured "wait n seconds before extracting text"
# delay, then stop whatever is still loading, then extract. The delay is measured from
# when navigation finished, not from when it started, because the point of it is to let
# JS-rendered content appear *after* load - anchoring it to the start would quietly give a
# slow-loading page almost no settle time.
#
# Only that delay is a user-facing setting. The navigation bound above is a safety net with
# a sane default, not a tuning knob, so there is still one number for users to think about.
#
# There is no way to know a page is "finished" - plenty of sites navigate as part of their
# normal design, and some sit forever on a subresource that never answers. So the delay
# restarts if the MAIN frame replaces its document (a redirect or interstitial gets the
# same settle time the first document got), iframes do not restart it, and it is capped so
# a page that re-navigates in a loop cannot extend it indefinitely.
max_content_ready_resets = int(os.getenv("BROWSER_CONTENT_READY_MAX_RESETS", 2))
# Check if page still exists (might have been closed due to error during sleep)
if not self.page or not hasattr(self.page, '_client'):
logger.debug("Page already closed, skipping stopLoading")
return
async def wait_for_content_ready_then_stop_loading():
main_frame_id = self.page.mainFrame._id
renavigated = asyncio.Event()
logger.debug("Issuing stopLoading command...")
await self.page._client.send('Page.stopLoading')
logger.debug("stopLoading command sent!")
def _on_main_frame_navigation(event):
if event.get('frameId') == main_frame_id:
renavigated.set()
async def setup_frame_handlers_on_first_response(event):
# Only trigger for the main document response
if event.get('type') == 'Document':
logger.debug("First response received, setting up frame handlers for forced page stop load.")
self.page._client.on('Page.frameStartedNavigating', lambda e: asyncio.create_task(handle_frame_navigation(e)))
self.page._client.on('Page.frameStartedLoading', lambda e: asyncio.create_task(handle_frame_navigation(e)))
self.page._client.on('Page.frameStoppedLoading', lambda e: logger.debug(f"Frame stopped loading: {e}"))
logger.debug("First response received, setting up frame handlers for forced page stop load DONE SETUP")
# De-register this listener - we only need it once
self.page._client.remove_listener('Network.responseReceived', setup_frame_handlers_on_first_response)
self.page._client.on('Page.frameStartedLoading', _on_main_frame_navigation)
self.page._client.on('Page.frameStoppedLoading', lambda e: logger.debug(f"Frame stopped loading: {e}"))
try:
resets = 0
while True:
renavigated.clear()
try:
await asyncio.wait_for(renavigated.wait(), timeout=extra_wait)
# Main frame started a new document
resets += 1
if resets > max_content_ready_resets:
# The cap is there to stop a page that re-navigates in a loop from
# extending the fetch forever - it is NOT permission to extract
# immediately. Breaking straight out here landed on whatever document
# happened to be mid-flight, with zero settle time: measured against a
# page that hops every 500ms, the fetch ended after 2.7s holding 130
# bytes of an intermediate hop, no final document and no JS-rendered
# content, while logging "content-ready wait of 12s elapsed".
#
# So spend the delay one last time, just without arming another reset.
# Total stays bounded at (max_resets + 2) * extra_wait, and whatever we
# extract has had the same settle time every other fetch gets.
logger.debug(f"Main frame re-navigated {resets} times (cap "
f"{max_content_ready_resets}), waiting {extra_wait}s once "
f"more without restarting, then extracting regardless")
await asyncio.sleep(extra_wait)
break
logger.debug(f"Main frame started a new document, restarting the {extra_wait}s "
f"content-ready wait ({resets}/{max_content_ready_resets})")
except asyncio.TimeoutError:
# Quiet for the whole delay - the page is as ready as it is going to get
break
finally:
self.page._client.remove_listener('Page.frameStartedLoading', _on_main_frame_navigation)
# Listen for first response to trigger frame handler setup
self.page._client.on('Network.responseReceived', setup_frame_handlers_on_first_response)
# Stop whatever is still in flight so the DOM and screenshot come from what rendered,
# rather than waiting on a subresource that may never answer
try:
logger.debug(f"Content-ready wait finished, issuing Page.stopLoading before extracting")
await self.page._client.send('Page.stopLoading')
logger.debug("stopLoading command sent!")
# stopLoading stops the network, not script execution. A page whose JS has pegged
# the renderer's main thread (a runaway loop, a rAF that never settles) holds that
# thread indefinitely, and every CDP call that needs to run script then queues
# behind it and never returns - page.content, the xPath scraper, the favicon
# fetcher. The fetch dies at PUPPETEER_MAX_PROCESSING_TIMEOUT_SECONDS having
# extracted nothing, with a core spinning the entire time.
#
# Nothing else recovers this. Runtime.evaluate's own `timeout` parameter bounds an
# evaluation once it starts, not time spent queued behind the running task, and
# wrapping the call in asyncio.wait_for is worse than useless: cancelling a
# pyppeteer request mid-flight leaves the connection unusable ("Target closed" on
# everything after it). Terminating execution is what releases the thread -
# measured against a deliberately spinning page, extraction went from timing out
# to returning the full DOM in 0.0s and the renderer dropped from 1.00 to 0.08
# cores. Safe here because stopLoading has already declared "give me what
# rendered", and the content-ready wait above has already had its chance to let
# late JS-rendered content appear.
await self.page._client.send('Runtime.terminateExecution')
logger.debug("Runtime.terminateExecution sent, any runaway page script is stopped")
except Exception as e:
logger.debug(f"Page.stopLoading/Runtime.terminateExecution skipped, page is most "
f"likely already gone: {e}")
# Track the LATEST main-frame document response for the whole fetch, not just the one that
# goto() happens to return. This app compares the text of the page the browser ends up on,
# and plenty of sites navigate again after the first response:
# - an interstitial answering 503/429 with a meta-refresh into the real 200 page, where
# judging the first response fails a watch whose content is sitting right there
# - a plain client-side redirect to another host (slated.com -> get.slated.com)
# It also covers Chrome 153+, which refuses to commit a navigation when an error status
# arrives with a zero-length body: goto() raises net::ERR_HTTP_RESPONSE_CODE_FAILURE rather
# than returning the response, but the response itself still arrives on this event.
navigation_response = {}
def _keep_navigation_response(response):
# Note pyppeteer exposes these as properties, unlike playwright where they are methods
if response.frame == self.page.mainFrame and response.request.isNavigationRequest:
navigation_response['response'] = response
self.page.on('response', _keep_navigation_response)
# pyppeteer's navigation watcher is bound to the loaderId of the navigation it started. If
# the page replaces that document (redirect/interstitial) the 'load' it waits for never
# arrives for that loaderId, so goto() never returns - and with timeout=0 it would block
# until the hard PUPPETEER_MAX_PROCESSING_TIMEOUT_SECONDS kill, burning a worker slot for
# minutes on a page that is fully loaded. Bound it, then fall back to the document we can
# see. Verified against slated.com and getastra.com, which hang indefinitely otherwise.
nav_timeout = int(os.getenv("BROWSER_NAVIGATION_TIMEOUT_SECONDS", 30))
response = None
attempt=0
while not response:
logger.debug(f"Attempting page fetch {url} attempt {attempt}")
asyncio.create_task(handle_frame_navigation())
response = await self.page.goto(url, timeout=0)
await asyncio.sleep(1 + extra_wait)
# Check if page still exists before sending command
if self.page and hasattr(self.page, '_client'):
await self.page._client.send('Page.stopLoading')
try:
while not response:
logger.debug(f"Attempting page fetch {url} attempt {attempt}")
# Race goto() against the main frame actually firing 'load'. In the re-navigation
# case goto() can never resolve, but the replacement document does fire 'load' -
# usually within a few seconds - so this returns then instead of sitting out the
# whole nav_timeout. Whichever arrives first means "the document is loaded".
main_frame_loaded = asyncio.Event()
main_frame_id = self.page.mainFrame._id
if response:
break
if not response:
logger.warning("Page did not fetch! trying again!")
if response is None and attempt>=2:
logger.warning(f"Content Fetcher > Response object was none (as in, the response from the browser was empty, not just the content) exiting attempt {attempt}")
raise EmptyReply(url=url, status_code=None)
attempt+=1
def _on_lifecycle(event):
if event.get('name') == 'load' and event.get('frameId') == main_frame_id:
main_frame_loaded.set()
self.page._client.on('Page.lifecycleEvent', _on_lifecycle)
goto_task = asyncio.ensure_future(self.page.goto(url, timeout=0))
load_task = asyncio.ensure_future(main_frame_loaded.wait())
try:
done, _pending = await asyncio.wait({goto_task, load_task},
timeout=nav_timeout,
return_when=asyncio.FIRST_COMPLETED)
if goto_task in done:
try:
response = goto_task.result()
except Exception as e:
if 'ERR_HTTP_RESPONSE_CODE_FAILURE' not in str(e) or not navigation_response:
raise
response = navigation_response['response']
logger.debug(f"Navigation was aborted by the browser (empty body on an error status), "
f"recovered status {response.status} from the response event")
else:
# Either the replacement document loaded, or we ran out of patience
response = navigation_response.get('response')
if not response:
raise BrowserFetchTimedOut(msg=f"Browser did not finish navigating to {url} within "
f"{nav_timeout}s and no main-frame response was seen.")
why = ("the page replaced the document it started on" if load_task in done
else f"navigation did not settle within {nav_timeout}s")
logger.warning(f"Continuing with the document actually loaded ({why}) - "
f"status {response.status} for {response.url}")
finally:
self.page._client.remove_listener('Page.lifecycleEvent', _on_lifecycle)
for t in (goto_task, load_task):
if not t.done():
t.cancel()
if response:
break
if not response:
logger.warning("Page did not fetch! trying again!")
if response is None and attempt>=2:
logger.warning(f"Content Fetcher > Response object was none (as in, the response from the browser was empty, not just the content) exiting attempt {attempt}")
raise EmptyReply(url=url, status_code=None)
attempt+=1
# Navigation is done; now honour "wait n seconds before extracting text" and then
# force-stop whatever is still loading, so extraction always gets what rendered.
# Awaited inline rather than fired off as a task, so nothing can outlive the fetch.
await wait_for_content_ready_then_stop_loading()
# That wait is where a meta-refresh interstitial typically swaps in the real page, so
# re-check which document we are actually on before judging the status code.
latest = navigation_response.get('response')
if latest is not None and latest is not response:
logger.debug(f"Page navigated again while waiting, judging the fetch on {latest.url} "
f"(status {latest.status}) instead of {response.url} (status {response.status})")
response = latest
finally:
self.page.remove_listener('response', _keep_navigation_response)
self.headers = response.headers
@@ -509,8 +654,7 @@ class fetcher(Fetcher):
self.screenshot = await capture_full_page(page=self.page, screenshot_format=self.screenshot_format, watch_uuid=watch_uuid, lock_viewport_elements=self.lock_viewport_elements)
# Force garbage collection - pyppeteer base64 decode creates temporary buffers
import gc
gc.collect()
gc_debounce.collect('puppeteer.after_screenshot')
self.xpath_data = await self.page.evaluate(XPATH_ELEMENT_JS, {
"visualselector_xpath_selectors": visualselector_xpath_selectors,
"max_height": MAX_TOTAL_HEIGHT
@@ -54,57 +54,98 @@
return 0;
});
const timeoutMs = 2000;
// All candidates are fetched concurrently under one shared deadline.
//
// Sequentially, each icon got its own fresh 2s AbortController, so a site declaring
// five <link rel="icon"> variants spent 10s+ here - inside the page, holding a browser
// and a worker the whole time. Simply capping the total made it worse: giving up early
// returns no icon, nothing gets saved, favicon_is_expired() stays true and the cost is
// paid again on the very next check, forever. Fetching in parallel bounds the wall time
// *and* still finds a working icon, so it saves and the watch stops asking.
//
// Measured against a page with five hanging icons, one 404 and one good one:
// sequential, 2s each : 10.1s, found the icon
// total cap only : 3.0s, found nothing (then repeats every check)
// parallel + deadline : ~3s, found the icon
const TOTAL_BUDGET_MS = 3000;
// 1 MB — matches the server-side limit in bump_favicon()
const MAX_BYTES = 1 * 1024 * 1024;
for (const icon of icons) {
const toBase64 = (blob) => new Promise(resolve => {
// Always resolves. The previous version resolved only from onloadend and read
// reader.result unguarded, so a FileReader failure threw inside the callback and left
// the promise permanently pending - the whole favicon fetch then hung with nothing
// bounding it, because clearTimeout had already fired.
try {
const reader = new FileReader();
reader.onerror = () => resolve(null);
reader.onloadend = () => {
try {
const result = reader.result;
resolve(result ? String(result).split(',')[1] : null);
} catch (e) {
resolve(null);
}
};
reader.readAsDataURL(blob);
} catch (e) {
resolve(null);
}
});
const controller = new AbortController();
const budget = setTimeout(() => controller.abort(), TOTAL_BUDGET_MS);
const fetchOne = async (icon) => {
try {
// Inline data URI — no network fetch needed, data is already here
if (icon.href.startsWith('data:')) {
const match = icon.href.match(/^data:([^;]+);base64,([A-Za-z0-9+/=]+)$/);
if (!match) continue;
if (!match) return null;
const mime_type = match[1];
const base64 = match[2];
// Rough size check: base64 is ~4/3 the binary size
if (base64.length * 0.75 > MAX_BYTES) continue;
if (base64.length * 0.75 > MAX_BYTES) return null;
return { url: icon.href, mime_type, base64 };
}
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), timeoutMs);
const resp = await fetch(icon.href, {
signal: controller.signal,
redirect: 'follow'
});
clearTimeout(timeout);
if (!resp.ok) return null;
if (!resp.ok) {
continue;
}
// Skip an oversized icon before pulling its body down the wire, where the server
// tells us the size up front.
const declared = parseInt(resp.headers.get('content-length') || '0', 10);
if (declared > MAX_BYTES) return null;
// Still covered by the shared signal: aborting errors the body stream too. The
// previous version cleared its timer before this line, leaving a slow or
// never-ending body read completely unguarded.
const blob = await resp.blob();
if (blob.size > MAX_BYTES) return null;
if (blob.size > MAX_BYTES) continue;
// Convert blob to base64
const reader = new FileReader();
return await new Promise(resolve => {
reader.onloadend = () => {
resolve({
url: icon.href,
mime_type: blob.type,
base64: reader.result.split(",")[1]
});
};
reader.readAsDataURL(blob);
});
const base64 = await toBase64(blob);
if (!base64) return null;
return { url: icon.href, mime_type: blob.type, base64 };
} catch (e) {
continue;
return null;
}
};
try {
const settled = await Promise.all(icons.map(fetchOne));
// icons[] is already in preference order (largest, then apple-touch-icon), so the
// first success in that order is the one we want - not merely the fastest to answer.
const best = settled.find(r => r);
if (best) return best;
} catch (e) {
// fall through to "nothing found"
} finally {
clearTimeout(budget);
}
// nothing found
@@ -71,6 +71,7 @@ async () => {
'out of stock',
'out-of-stock',
'plus disponible',
'producto sin stock',
'prodotto esaurito',
'produkt niedostępny',
'rupture',
File diff suppressed because it is too large Load Diff
+44 -8
View File
@@ -2,8 +2,9 @@ import os
import re
from loguru import logger
from wtforms.widgets.core import TimeInput
from flask_babel import lazy_gettext as _l, gettext
from flask_babel import gettext, lazy_gettext as _l, lazy_pgettext
from changedetectionio.blueprint.menu_modes import MENU_SIDEBAR_ACTIONMODES, MENU_SIDEBAR_ACTIONMODES_DEFAULT
from changedetectionio.blueprint.rss import RSS_FORMAT_TYPES, RSS_TEMPLATE_TYPE_OPTIONS, RSS_TEMPLATE_HTML_DEFAULT
from changedetectionio.llm.ui_strings import LLM_INTENT_WATCH_PLACEHOLDER
from changedetectionio.llm.evaluator import (
@@ -696,14 +697,18 @@ class ValidateCSSJSONXPATHInput(object):
raise ValidationError("XPath not permitted in this field!")
from lxml import etree, html
import elementpath
from changedetectionio.html_tools import get_safe_xpath3_parser, lxml_guard, lxml_html_parser
from changedetectionio.html_tools import get_safe_xpath3_parser, lxml_guard, lxml_html_parser, \
XPATH_CODEPOINT_COLLATION
line = line.replace('xpath:', '')
try:
# Runs on a Flask request thread - must share the worker's lxml lock.
with lxml_guard():
tree = html.fromstring("<html></html>", parser=lxml_html_parser())
elementpath.select(tree, line.strip(), parser=get_safe_xpath3_parser())
# Same collation the filter will actually run under, so validation
# cannot accept an expression that then behaves differently at check time.
elementpath.select(tree, line.strip(), parser=get_safe_xpath3_parser(),
default_collation=XPATH_CODEPOINT_COLLATION)
except elementpath.ElementPathError as e:
message = field.gettext('\'%(expression)s\' is not a valid XPath expression. (%(error)s)')
raise ValidationError(message % {'expression': line, 'error': str(e)})
@@ -939,6 +944,7 @@ class SingleBrowserStep(Form):
class processor_text_json_diff_form(commonSettingsForm):
url = StringField(_l('Web Page URL'), validators=[validateURL()])
link_to_open = StringField(_l('Open Link Override'), validators=[validators.Optional(), validateURL()], default='')
tags = StringTagUUID(_l('Group Tag'), [validators.Optional()], default='')
time_between_check = EnhancedFormField(
@@ -1009,7 +1015,8 @@ class processor_text_json_diff_form(commonSettingsForm):
notification_muted = TernaryNoneBooleanField(_l('Notifications'), default=None, yes_text=_l("Muted"), no_text=_l("On"))
notification_screenshot = BooleanField(_l('Attach screenshot to notification (where possible)'), default=False)
conditions_match_logic = RadioField(_l('Match'), choices=[('ALL', _l('Match all of the following')),('ANY', _l('Match any of the following'))], default='ALL')
# TRANSLATORS: Label for the setting combining condition rules with AND/OR. Noun; not the verb 'to match'.
conditions_match_logic = RadioField(lazy_pgettext('condition', 'Match'), choices=[('ALL', _l('Match all of the following')),('ANY', _l('Match any of the following'))], default='ALL')
conditions = FieldList(FormField(ConditionFormRow), min_entries=1) # Add rule logic here
use_page_title_in_list = TernaryNoneBooleanField(_l('Use page <title> in list'), default=None)
@@ -1046,6 +1053,19 @@ class processor_text_json_diff_form(commonSettingsForm):
self.url.errors.append(gettext('Invalid template syntax: %(error)s') % {'error': e})
result = False
# Attempt to validate jinja2 templates in the optional "Link to Open"
if self.link_to_open.data and self.link_to_open.data.strip():
try:
jinja_render(template_str=self.link_to_open.data)
except ModuleNotFoundError as e:
logger.error(e)
self.link_to_open.errors.append(gettext('Invalid template syntax configuration: %(error)s') % {'error': e})
result = False
except Exception as e:
logger.error(e)
self.link_to_open.errors.append(gettext('Invalid template syntax: %(error)s') % {'error': e})
result = False
# Attempt to validate jinja2 templates in the body
if self.body.data and self.body.data.strip():
try:
@@ -1160,13 +1180,13 @@ class globalSettingsApplicationUIForm(Form):
socket_io_enabled = BooleanField(_l('Realtime UI Updates Enabled'), default=True, validators=[validators.Optional()])
favicons_enabled = BooleanField(_l('Favicons Enabled'), default=True, validators=[validators.Optional()])
use_page_title_in_list = BooleanField(_l('Use page <title> in watch overview list')) #BooleanField=True
use_share_watch = BooleanField(_l('Enable watch "sharing"'))
timeago_format = SelectField(_l('Relative time format'),
choices=[('long', _l('Long (1 minute ago)')), ('short', _l('Short (1m ago)'))],
default='long', validators=[validators.Optional()])
sidebar_mode = SelectField(_l('Navigation sidebar'),
choices=[('collapsed', _l('Collapsed icon rail (expands on hover)')),
('pinned', _l('Always expanded'))],
default='collapsed', validators=[validators.Optional()])
choices=MENU_SIDEBAR_ACTIONMODES,
default=MENU_SIDEBAR_ACTIONMODES_DEFAULT, validators=[validators.Optional()])
# datastore.data['settings']['application']..
class globalSettingsApplicationForm(commonSettingsForm):
@@ -1246,7 +1266,23 @@ class globalSettingsLLMForm(Form):
_l('API Key'),
validators=[validators.Optional()],
render_kw={
"autocomplete": "off",
# NOT "off": Chrome deliberately ignores autocomplete="off" on type=password,
# so the browser's saved site password was being prefilled here. This field
# renders blank precisely so that submitting it untouched PRESERVES the stored
# key - a prefilled value therefore silently overwrote a working API key on the
# next Save. "new-password" is the value Chrome honours; the data-* attributes
# ask 1Password and LastPass to keep out of it too.
"autocomplete": "new-password",
"data-1p-ignore": "true",
"data-lpignore": "true",
"data-form-type": "other",
# Belt and braces, for when no key is stored yet and the field is editable
# (once one IS stored the template disables it outright). Chrome will not
# autofill a readonly input, and global-settings.js drops the attribute the
# moment the field is focused or clicked. readonly rather than disabled here,
# because a disabled input is not submitted and the key could never be set.
"readonly": True,
"data-unlock-on-interact": "1",
"style": "width: 24em;",
},
)
+73
View File
@@ -0,0 +1,73 @@
"""
Debounced explicit garbage collection for the watch-check hot path.
Several places call gc.collect() after a check to keep C-level memory (pyppeteer buffers,
libxml2 documents, PIL, brotli) from accumulating. Individually each is reasonable. Run
concurrently by many fetch workers they become a storm: with FETCH_WORKERS=50 and roughly
five call sites per check, the process spends most of its time stopped in the collector,
because every gc.collect() is a full stop-the-world pass that walks the entire heap while
holding the GIL.
Measured on 153 real puppeteer checks of a live site, FETCH_WORKERS=50, an `//div` filter
so the lxml document tree is realistic:
collects objects freed gc time checks/sec CPU/check RSS plateau
one per call site 790 2,594,098 91.3s 0.766 1.373s 279.6MB
debounced to 1s 48 2,219,010 7.3s 1.433 0.731s 275.3MB
none at all 0 0 0.0s 1.503 0.685s 287.7MB
Debouncing keeps 86% of the reclamation for 6% of the collections, and resident memory
ends up LOWER than collecting every time. It works because the collector is process-wide:
any worker's collection breaks every other worker's cycles too, so with many workers the
calls are overwhelmingly redundant duplicates rather than independently necessary.
Removing them entirely was also measured. It is slightly faster still, but it was the only
configuration whose RSS had not plateaued by the end of the run, so it is not offered.
Collecting a younger generation was measured and rejected: gen 0 freed 1,136 objects
against the full pass's 2,594,098, because objects that survive a 10-30 second fetch have
already been promoted out of gen 0. It is cheap because it does almost nothing.
Environment:
EXPLICIT_GC_MIN_INTERVAL seconds between explicit collections, process-wide.
Default 1.0. Set 0 to collect at every call site as before.
EXPLICIT_GC_COLLECT set false to disable explicit collection entirely. A
measurement switch for attributing a slowdown, not a
recommended setting - expect resident memory to drift.
"""
import gc
import os
import threading
import time
from changedetectionio.strtobool import strtobool
ENABLED = strtobool(os.getenv('EXPLICIT_GC_COLLECT', 'true'))
MIN_INTERVAL = float(os.getenv('EXPLICIT_GC_MIN_INTERVAL', '1.0') or 0)
_last_collect = 0.0
_lock = threading.Lock()
def collect(where=None):
"""Explicit collection for the per-check hot path, rate-limited process-wide.
`where` is a short label for the call site, kept so callers read clearly and so a
future caller can log it. Returns the number of objects collected, or 0 when the call
was debounced or disabled - matching gc.collect()'s return, so this is a drop-in
replacement for it.
"""
global _last_collect
if not ENABLED:
return 0
if MIN_INTERVAL > 0:
now = time.monotonic()
with _lock:
if now - _last_collect < MIN_INTERVAL:
return 0
_last_collect = now
return gc.collect()
+41 -8
View File
@@ -95,7 +95,13 @@ TEXT_FILTER_LIST_LINE_SUFFIX = "<br>"
TRANSLATE_WHITESPACE_TABLE = str.maketrans('', '', '\r\n\t ')
PERL_STYLE_REGEX = r'^/(.*?)/([a-z]*)?$'
TITLE_RE = re.compile(r"<title[^>]*>(.*?)</title>", re.I | re.S)
# Whitespace is allowed after the tag name ("</title >") but not after "<" - "< title>" is text, not a tag
TITLE_RE = re.compile(r"<title[^>]*>(.*?)</title\s*>", re.I | re.S)
# Locate the tag with pos/endpos instead of data.lower(), which copies the whole document (67MB seen
# for one page) and, for str, can change its length ('İ' lowers to 2 chars) so the offset is wrong.
TITLE_TAG_STR_RE = re.compile(r"<title", re.I | re.A)
TITLE_TAG_BYTES_RE = re.compile(rb"<title", re.I)
TITLE_TAG_UTF32_RE = re.compile(rb"<\x00\x00\x00[tT]\x00\x00\x00")
META_CS = re.compile(r'<meta[^>]+charset=["\']?\s*([a-z0-9_\-:+.]+)', re.I)
# jq builtins that can leak sensitive data or cause harm when user-supplied expressions are executed.
@@ -164,6 +170,25 @@ _DEFAULT_UNSAFE_XPATH3_FUNCTIONS = [
]
# XPath 3.1 says the default collation is Unicode codepoint collation. elementpath instead leaves
# its collation functions pointing at locale.strxfrm / locale.strcoll, so the process-wide
# LC_COLLATE decides what contains() means:
#
# def contains(self, a, b): return self.strxfrm(b) in self.strxfrm(a)
#
# With LC_COLLATE=C, strxfrm() is the identity and that is an ordinary substring test. With
# LC_COLLATE=en_US.UTF-8 it returns a binary collation key, and a substring of a collation key is
# not the collation key of the substring - so contains(), starts-with(), ends-with() and
# substring-before/after() return false for every input, and a filter that matches 67 elements
# matches 0 (#4437). Name tests, axes and '=' are unaffected, which is what made it look like the
# page had changed layout.
#
# flask_app.py keeps LC_COLLATE in "C" for this reason, but a filter must not depend on a distant
# module's locale bookkeeping, nor on what an operator puts in LANG/LC_ALL. Pinning the collation
# per evaluation makes the filter mean the same thing in every deployment.
XPATH_CODEPOINT_COLLATION = 'http://www.w3.org/2005/xpath-functions/collation/codepoint'
def get_safe_xpath3_parser():
"""Return an XPath3Parser subclass with filesystem/environment access functions removed.
@@ -383,7 +408,9 @@ def xpath_filter(xpath_filter, html_content, append_pretty_line_formatting=False
# This allows //title to match elements in the default namespace
namespaces[''] = tree.nsmap[None]
r = elementpath.select(tree, xpath_filter.strip(), namespaces=namespaces, parser=get_safe_xpath3_parser())
r = elementpath.select(tree, xpath_filter.strip(), namespaces=namespaces,
parser=get_safe_xpath3_parser(),
default_collation=XPATH_CODEPOINT_COLLATION)
#@note: //title/text() now works with default namespaces (fixed by registering '' prefix)
#@note: //title/text() wont work where <title>CDATA.. (use cdata_in_document_to_text first)
@@ -897,14 +924,18 @@ def extract_title(data: bytes | str, sniff_bytes: int = 2048, scan_chars: int =
# rare but possible. We read up to 128 KiB from the tag onwards to handle
# even pathological cases without scanning the whole document.
_TITLE_WINDOW = 131072
# Only look for the tag in the first 1 MiB (chars, or bytes for 8-bit). Well past any real <head>,
# and a huge page is never scanned end to end.
_TITLE_SEARCH_LIMIT = 1024 * 1024
try:
match data:
case bytes() if data.startswith((b"\xff\xfe\x00\x00", b"\x00\x00\xfe\xff")):
# UTF-32: locate the tag in the raw bytes, then decode the window.
tag_pos = data.lower().find(b"<\x00\x00\x00t\x00\x00\x00")
if tag_pos == -1:
tag_m = TITLE_TAG_UTF32_RE.search(data, 0, _TITLE_SEARCH_LIMIT * 4)
if not tag_m:
return None
tag_pos = tag_m.start()
chunk = data[tag_pos: tag_pos + _TITLE_WINDOW * 4].decode("utf-32", errors="replace")
prefix = chunk
case bytes() if data.startswith((b"\xff\xfe", b"\xfe\xff")):
@@ -913,9 +944,10 @@ def extract_title(data: bytes | str, sniff_bytes: int = 2048, scan_chars: int =
prefix = data[: max(scan_chars * 2, _TITLE_WINDOW)].decode("utf-16", errors="replace")
case bytes():
# UTF-8 / legacy 8-bit: find the tag cheaply in raw bytes.
tag_pos = data.lower().find(b"<title")
if tag_pos == -1:
tag_m = TITLE_TAG_BYTES_RE.search(data, 0, _TITLE_SEARCH_LIMIT)
if not tag_m:
return None
tag_pos = tag_m.start()
raw_chunk = data[tag_pos: tag_pos + _TITLE_WINDOW]
try:
chunk = raw_chunk.decode("utf-8")
@@ -932,9 +964,10 @@ def extract_title(data: bytes | str, sniff_bytes: int = 2048, scan_chars: int =
return None
prefix = chunk
case str():
tag_pos = data.lower().find("<title")
if tag_pos == -1:
tag_m = TITLE_TAG_STR_RE.search(data, 0, _TITLE_SEARCH_LIMIT)
if not tag_m:
return None
tag_pos = tag_m.start()
prefix = data[tag_pos: tag_pos + _TITLE_WINDOW]
case _:
logger.error(f"Title extraction received unsupported data type: {type(data)}")
+3
View File
@@ -19,6 +19,7 @@ def get_timeago_locale(flask_locale, short=False):
- Swedish: Flask uses 'sv', timeago uses 'sv_SE'
- Norwegian: Flask uses 'no', timeago uses 'nb_NO' or 'nn_NO'
- Hindi: Flask uses 'hi', timeago uses 'in_HI'
- Indonesian: Flask uses 'id', timeago uses 'in_ID'
- Czech: Flask uses 'cs', but timeago doesn't support Czech - fallback to English
Args:
@@ -46,6 +47,7 @@ def get_timeago_locale(flask_locale, short=False):
'sv': 'sv_SE', # Swedish
'no': 'nb_NO', # Norwegian Bokmål
'hi': 'in_HI', # Hindi
'id': 'in_ID', # Indonesian
'cs': 'en', # Czech not supported by timeago, fallback to English
'ja': 'ja', # Japanese
'uk': 'uk', # Ukrainian
@@ -156,6 +158,7 @@ LANGUAGE_DATA = {
'tr': {'flag': 'fi fi-tr fis', 'name': 'Türkçe'},
'ar': {'flag': 'fi fi-sa fis', 'name': 'العربية'},
'hi': {'flag': 'fi fi-in fis', 'name': 'हिन्दी'},
'id': {'flag': 'fi fi-id fis', 'name': 'Bahasa Indonesia'},
'uk': {'flag': 'fi fi-ua fis', 'name': 'Українська'},
}
+11 -2
View File
@@ -25,8 +25,17 @@ DEFAULT_TIMEOUT = int(os.getenv('LLM_TIMEOUT', 300))
# longer deadline (Hermes-style, 30 min). Overridable via LLM_LOCAL_TIMEOUT; see
# evaluator.resolve_llm_timeout() for how the endpoint is classified.
DEFAULT_LOCAL_TIMEOUT = int(os.getenv('LLM_LOCAL_TIMEOUT', 1800))
# Models and reasoning architectures that reject explicit sampling parameters (temperature/top_p)
_NO_TEMPERATURE_MODEL_KEYWORDS = ('flash-lite', 'thinking-exp', 'o1', 'o3', 'o4')
# Models and reasoning architectures that reject explicit sampling parameters (temperature/top_p).
# Substring match against the lowercased model name, so 'gpt-5' also covers gpt-5-mini, gpt-5.1 etc.
#
# This is only an optimisation - the BadRequestError handler below strips and retries for
# anything not listed. Being listed just avoids paying a rejected round trip on every call.
#
# NOT all GPT models: gpt-4o, gpt-4-turbo and gpt-3.5-turbo take temperature normally. It is
# reasoning that forbids it. gpt-5 reports "only temperature=1 is supported unless
# reasoning_effort resolves to 'none'" - we never set reasoning_effort=none, so it is always
# in that state for us.
_NO_TEMPERATURE_MODEL_KEYWORDS = ('flash-lite', 'thinking-exp', 'o1', 'o3', 'o4', 'gpt-5')
DEFAULT_RETRIES = 3
+23 -2
View File
@@ -165,6 +165,12 @@ def _summary_max_tokens(diff: str, max_cap: int = LLM_DEFAULT_MAX_SUMMARY_TOKENS
return max(400, min(len(diff) // 4, max_cap))
# Models that emit chain-of-thought before the answer, on any provider. Reasoning tokens are
# billed and counted against max_tokens, so these need headroom or the answer never lands.
# Kept in step with client._NO_TEMPERATURE_MODEL_KEYWORDS - reasoning is what drives both.
REASONING_MODEL_KEYWORDS = ('o1', 'o3', 'o4', 'gpt-5', 'thinking-exp')
def apply_local_token_multiplier(base_max_tokens: int, llm_cfg: dict) -> int:
"""
Scale max_tokens for endpoints that commonly serve reasoning models
@@ -182,10 +188,25 @@ def apply_local_token_multiplier(base_max_tokens: int, llm_cfg: dict) -> int:
cost change. Ollama / OpenAI-compatible users can dial the multiplier down to 1x
in Settings → AI → Provider if they want to keep costs tight on a paid endpoint.
Activated when `llm_cfg['provider_kind']` is `'ollama'` or `'openai_compatible'`.
Activated when `llm_cfg['provider_kind']` is `'ollama'` or `'openai_compatible'`, OR
when the model itself is a known reasoning model on any provider.
That second case was missing and it broke gpt-5 completely. The "cloud providers have
non-reasoning defaults" assumption above held for gpt-4o and friends, but gpt-5 reasons
by default: with the tight 400 cap it spent all 400 tokens on reasoning
(`completion_tokens_details.reasoning_tokens == 400`), returned content='' with
finish_reason='length', and still billed for the call. Every summary was empty and every
summary cost money. What decides whether headroom is needed is whether the MODEL reasons,
not whether the ENDPOINT is local.
Multiplier defaults to 5x and is user-configurable in Settings → AI → Provider.
"""
if (llm_cfg or {}).get('provider_kind') not in ('ollama', 'openai_compatible'):
llm_cfg = llm_cfg or {}
_model = (llm_cfg.get('model') or '').lower()
_is_reasoning_model = any(k in _model for k in REASONING_MODEL_KEYWORDS)
if (llm_cfg.get('provider_kind') not in ('ollama', 'openai_compatible')
and not _is_reasoning_model):
return base_max_tokens
try:
multiplier = int(llm_cfg.get('local_token_multiplier') or 5)
+149
View File
@@ -0,0 +1,149 @@
"""Background generation of AI change summaries.
The diff-page / watch-list "Summary" button used to hold a request thread for the entire LLM
round-trip (seconds, or the whole provider timeout). Instead the route now starts the work here
and answers immediately with 202. The browser learns the summary is ready either from the
``llm_summary_ready`` Socket.IO event (when realtime is enabled) or by polling the route (when it
is not) - see static/js/llm-summary.js.
Note what is *not* stored in this module: the summary itself. The job writes it to the watch's
on-disk cache (``Watch.save_llm_diff_summary``), which every reader already consults, so there is
no result registry to keep in sync and no memory that grows with history. All this holds is
"something is already running for this key" (so polling can never start a second generation and
pay for the same summary twice), "when that run actually started" (so the routes can hand the
browser a real deadline instead of it guessing one) and "the last attempt failed" (so a failure
reaches the next poll instead of dying with the thread and leaving the browser on a permanent
spinner).
A process restart therefore loses only the in-flight bookkeeping: the next poll sees neither a
cache entry nor a pending job, and restarts the work.
"""
import os
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from loguru import logger
# Bounded on purpose: this is the ceiling on concurrent LLM calls started from the UI, so a
# handful of operators clicking "Summary" at once cannot fan out into provider rate-limiting
# or a surprise token bill. Extra clicks queue.
DEFAULT_MAX_WORKERS = 2
class SummaryJobFailed(Exception):
"""A handled failure that should be reported to the next poll with a specific HTTP status."""
def __init__(self, message, http_status=500):
super().__init__(message)
self.message = message
self.http_status = http_status
class SummaryJobRegistry:
"""In-flight tracking + a small thread pool for on-demand summary generation."""
def __init__(self, max_workers=None):
self._lock = threading.Lock()
# key -> epoch when the worker picked the job up, or None while it is still queued
# behind another generation (the pool is deliberately small, see DEFAULT_MAX_WORKERS).
self._pending = {}
self._errors = {} # key -> (http_status, message)
self._pool = None
self._max_workers = max_workers or int(os.getenv('LLM_SUMMARY_WORKERS', DEFAULT_MAX_WORKERS))
def _get_pool(self):
"""Created on first use - importing this module must never spawn threads.
Threads started at import time would be alive before ``gc.freeze()`` runs in flask_app,
and would be inherited by any subprocess the restock processor spawns.
"""
if self._pool is None:
self._pool = ThreadPoolExecutor(
max_workers=self._max_workers,
thread_name_prefix='llm-summary',
)
return self._pool
def is_pending(self, key):
with self._lock:
return key in self._pending
def run_started_at(self, key):
"""Epoch at which the worker began running `key`, None if it is queued or not pending.
The routes turn this into the deadline they send to the browser: a job still sitting in
the queue has not started burning its LLM timeout yet, so its deadline has to keep moving
rather than expire while it waits its turn.
"""
with self._lock:
return self._pending.get(key)
def take_error(self, key):
"""Pop the stored failure for `key`, or None.
Popping (rather than peeking) means an error is delivered to exactly one poll: the browser
shows it and stops, and a later deliberate click is free to retry from scratch.
"""
with self._lock:
return self._errors.pop(key, None)
def submit(self, key, fn, on_settled=None):
"""Run `fn()` on the pool unless `key` is already running.
`fn` is responsible for persisting its own result before returning; see the ordering note
in `_run`. `on_settled(key)` is called once the job has finished either way - used to
signal the browser over Socket.IO so it does not have to poll. Returns True if this call
started the work, False if it was already in flight.
"""
with self._lock:
if key in self._pending:
logger.debug(f"LLM summary job already in flight, not starting a second: {key}")
return False
# Marked pending *before* the pool sees it, so a poll arriving between submit() and
# the worker actually starting still gets 'pending' rather than 'idle'. No start
# time yet - that is stamped in _run when a worker is free to take it.
self._pending[key] = None
self._errors.pop(key, None)
try:
self._get_pool().submit(self._run, key, fn, on_settled)
except Exception:
with self._lock:
self._pending.pop(key, None)
raise
return True
def _run(self, key, fn, on_settled=None):
with self._lock:
if key in self._pending:
self._pending[key] = time.time()
try:
fn()
except SummaryJobFailed as e:
with self._lock:
self._errors[key] = (e.http_status, e.message)
except Exception as e:
logger.error(f"LLM summary job failed for {key}: {e}")
with self._lock:
self._errors[key] = (500, str(e))
finally:
# Cleared last, and only after fn() has written its result to the summary cache.
# The reverse order would open a window where a poll sees no pending job and no cached
# summary, decides nothing is running, and pays for the same generation again.
with self._lock:
self._pending.pop(key, None)
# Announced only after the flag is cleared and the result (or error) is readable, so
# the single fetch a notified client makes cannot come back "still pending" and leave
# it waiting for an event that has already been sent.
if on_settled:
try:
on_settled(key)
except Exception as e:
logger.warning(f"LLM summary on_settled callback failed for {key}: {e}")
# Process-wide instance used by the UI routes.
summary_jobs = SummaryJobRegistry()
+3 -1
View File
@@ -1,6 +1,7 @@
from os import getenv
from copy import deepcopy
from changedetectionio.blueprint.menu_modes import MENU_SIDEBAR_ACTIONMODES_DEFAULT
from changedetectionio.blueprint.rss import RSS_FORMAT_TYPES, RSS_CONTENT_FORMAT_DEFAULT
from changedetectionio.model.Tags import TagsDict
@@ -76,11 +77,12 @@ class model(dict):
'webdriver_delay': None , # Extra delay in seconds before extracting text
'ui': {
'use_page_title_in_list': True,
'use_share_watch': False, # "Share watch" link is a power-user feature, off by default
'open_diff_in_new_tab': True,
'socket_io_enabled': True,
'favicons_enabled': True,
'timeago_format': 'long', # 'long' = "1 minute ago", 'short' = "1m ago"
'sidebar_mode': 'collapsed', # 'collapsed' = slim icon rail, expands on hover; 'pinned' = always expanded
'sidebar_mode': MENU_SIDEBAR_ACTIONMODES_DEFAULT, # one of blueprint.menu_modes.MENU_SIDEBAR_ACTIONMODES
},
}
}
+58 -11
View File
@@ -30,6 +30,7 @@ from changedetectionio.validate_url import is_safe_valid_url
from changedetectionio.strtobool import strtobool
from changedetectionio.jinja2_custom import render as jinja_render
from changedetectionio import gc_debounce
from . import watch_base
from .persistence import EntityPersistenceMixin
import os
@@ -107,9 +108,21 @@ def _brotli_save(contents, filepath, mode=None, fallback_uncompressed=False):
logger.debug(f"Finished brotli compression - From {original_size} to {total_compressed_size} bytes.")
# Cleanup: Delete compressor, force Python GC, then force C-level memory release
# Cleanup: drop the compressor, then force C-level memory back to the OS.
#
# There is deliberately no gc.collect() here. brotli.Compressor is not gc-tracked,
# so the collector can never reclaim it - `del` frees it immediately by refcount.
# Measured over 60 x 2.2MB compressions, RSS growth was:
#
# neither +0.9MB
# gc.collect() only +0.2MB
# malloc_trim() only +0.0MB <- does all of the work
# both (previous) +0.0MB <- the collect contributed nothing
#
# malloc_trim below is the load-bearing line: brotli's retention is glibc holding
# freed arenas, which only a trim returns. The collect cost ~31ms of stop-the-world
# per snapshot save for no reclamation.
del compressor
gc.collect()
# Force release of C-level memory back to OS (since brotli is a C library)
try:
@@ -272,10 +285,12 @@ class model(EntityPersistenceMixin, watch_base):
def has_unviewed(self):
return int(self.newest_history_key) > int(self['last_viewed']) and self.__history_n >= 2
@property
def link(self):
def _resolve_link(self, url, flash_on_template_error=True):
"""Validate, Jinja2-render and de-'source:' a URL so it is safe to put in an href.
url = self.get('url', '')
Returns 'DISABLED' when the URL is missing/unsafe, or '' when its Jinja2 template
could not be rendered.
"""
if not is_safe_valid_url(url):
return 'DISABLED'
@@ -286,11 +301,15 @@ class model(EntityPersistenceMixin, watch_base):
ready_url = jinja_render(template_str=url)
except Exception as e:
logger.critical(f"Invalid URL template for: '{url}' - {str(e)}")
from flask import flash, url_for
from markupsafe import Markup
message = Markup('<a href="{}#general">The URL {} is invalid and cannot be used, click to edit</a>'.format(
url_for('ui.ui_edit.edit_page', uuid=self.get('uuid')), self.get('url', '')))
flash(message, 'error')
# has_request_context() guard: this is also reached from worker threads and the
# notification path, where flash() would raise "Working outside of request context".
from flask import has_request_context
if flash_on_template_error and has_request_context():
from flask import flash, url_for
from markupsafe import Markup
message = Markup('<a href="{}#general">The URL {} is invalid and cannot be used, click to edit</a>'.format(
url_for('ui.ui_edit.edit_page', uuid=self.get('uuid')), url))
flash(message, 'error')
return ''
if ready_url.startswith('source:'):
@@ -301,6 +320,34 @@ class model(EntityPersistenceMixin, watch_base):
return 'DISABLED'
return ready_url
@property
def link(self):
"""The URL that actually gets fetched/checked."""
return self._resolve_link(self.get('url', ''))
@property
def open_link_override(self):
"""The resolved optional 'Link to Open', or '' when unset/unusable."""
override = str(self.get('link_to_open') or '').strip()
if not override:
return ''
# Don't flash on a broken template here - `link` already surfaces that for the watched URL,
# and a bad "Link to Open" should never stop the page rendering.
ready = self._resolve_link(override, flash_on_template_error=False)
if not ready or ready == 'DISABLED':
return ''
return ready
@property
def open_link(self):
"""The URL a human should be sent to when they click through to "the site".
Some watches point at an API endpoint or RSS feed that is useless in a browser -
'Link to Open' lets the user store the real page for those. Falls back to `link`.
"""
return self.open_link_override or self.link
@property
def domain_only_from_link(self):
from urllib.parse import urlparse
@@ -655,7 +702,7 @@ class model(EntityPersistenceMixin, watch_base):
# reimport
bump = self.history
gc.collect()
gc_debounce.collect('watch.history_bump')
# Save some text file to the appropriate path and bump the history
# result_obj from fetch_site_status.run()
+1
View File
@@ -207,6 +207,7 @@ class watch_base(dict):
'last_error': False,
'last_notification_error': None,
'last_viewed': 0, # history key value of the last viewed via the [diff] link
'link_to_open': '', # Optional human-facing URL to open instead of 'url' (eg. watch an API/RSS endpoint, open the real page)
'llm_backend_profile': True, # @note - now its just a bool but in the near future we can select a LLM profile or 'off'/false
'llm_change_summary': '', # Prompt for AI Change Summary — replaces {{ diff }} in notifications
'llm_change_summary_mode': 'replace', # 'replace' the inherited prompt, or 'append' to it
@@ -241,6 +241,7 @@ class NotificationContextData(dict):
'watch_tag': None,
'watch_title': None,
'watch_url': 'https://WATCH-PLACE-HOLDER/',
'watch_open_url': 'https://WATCH-PLACE-HOLDER/', # watch['link_to_open'] when set, otherwise the same as watch_url
'watch_uuid': 'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', # Converted to 'watch_uuid' in create_notification_parameters
})
@@ -350,6 +351,7 @@ def set_basic_notification_vars(current_snapshot, prev_snapshot, watch, triggere
'triggered_text': triggered_text,
'uuid': watch.get('uuid') if watch else None,
'watch_url': watch.get('url') if watch else None,
'watch_open_url': (watch.open_link_override or watch.get('url')) if watch else None,
'watch_uuid': watch.get('uuid') if watch else None,
'watch_mime_type': watch.get('content-type')
}
@@ -511,6 +513,7 @@ Thanks - Your omniscient changedetection.io installation.
if 'notification_urls' in n_object:
n_object.update({
'watch_url': watch['url'],
'watch_open_url': watch.open_link_override or watch['url'],
'uuid': watch_uuid,
'screenshot': None
})
@@ -560,6 +563,7 @@ Thanks - Your omniscient changedetection.io installation.
if 'notification_urls' in n_object:
n_object.update({
'watch_url': watch['url'],
'watch_open_url': watch.open_link_override or watch['url'],
'uuid': watch_uuid
})
self.notification_q.put(n_object)
+9 -3
View File
@@ -22,11 +22,16 @@ class difference_detection_processor():
preferred_proxy = None
screenshot_format = SCREENSHOT_FORMAT_JPEG
last_raw_content_checksum = None
worker_id = None
def __init__(self, datastore, watch_uuid):
def __init__(self, datastore, watch_uuid, worker_id=None):
self.datastore = datastore
self.watch_uuid = watch_uuid
# Which async worker is driving this check, passed down to the fetcher in call_browser()
# so it can keep per-worker browser state apart, None when we're not called from a worker
self.worker_id = worker_id
# Create a stable snapshot of the watch for processing
# Why deepcopy?
# 1. Prevents "dict changed during iteration" errors if watch is modified during processing
@@ -201,7 +206,8 @@ class difference_detection_processor():
# When browser_connection_url is None, it method should default to working out whats the best defaults (os env vars etc)
self.fetcher = fetcher_obj(proxy_override=proxy_url,
custom_browser_connection_url=custom_browser_connection_url,
screenshot_format=self.screenshot_format
screenshot_format=self.screenshot_format,
worker_id=self.worker_id
)
# Stamp the resolved backend name so downstream consumers (processors, plugins)
@@ -261,7 +267,7 @@ class difference_detection_processor():
await self.fetcher.run(
current_include_filters=self.watch.get('include_filters'),
empty_pages_are_a_change=empty_pages_are_a_change,
fetch_favicon=self.watch.favicon_is_expired(),
fetch_favicon=self.watch.favicon_is_expired() and self.datastore.data['settings']['application'].get('ui', {}).get('favicons_enabled', True),
ignore_status_codes=ignore_status_codes,
is_binary=is_binary,
request_body=request_body,
@@ -430,7 +430,8 @@ def render(watch, datastore, request, url_for, render_template, flash, redirect)
change_percentage=change_percentage,
comparison_data=comparison_data, # Full history for charts/visualization
comparison_method=method_display,
current_diff_url=watch['url'],
current_diff_label=watch.label,
current_diff_url=watch.open_link,
from_version=from_version,
percentage_different=change_percentage,
threshold=pixel_difference_threshold_sensitivity,
@@ -106,5 +106,6 @@ def render(watch, datastore, request, url_for, render_template, flash, redirect)
uuid=watch.get('uuid'),
versions=versions,
timestamp=timestamp,
current_diff_url=watch['url']
current_diff_label=watch.label,
current_diff_url=watch.open_link
)
@@ -191,7 +191,8 @@ def render(watch, datastore, request, url_for, render_template, flash, redirect,
'restock_diff/difference.html',
uuid=uuid,
watch=watch,
current_diff_url=watch['url'],
current_diff_label=watch.label,
current_diff_url=watch.open_link,
extra_title=f" - {watch.label} - {gettext('Price history')}",
last_error=watch['last_error'],
screenshot=watch.get_screenshot(),
@@ -86,8 +86,10 @@
price_low: {{ _('Currently low')|tojson }},
price_typical: {{ _('Currently typical')|tojson }},
price_high: {{ _('Currently high')|tojson }},
cheaper_than: {{ _('cheaper than %s% of tracked prices')|tojson }},
pricier_than: {{ _('more expensive than %s% of tracked prices')|tojson }},
{# TRANSLATORS: {pct} is a percentage including the % sign, e.g. "80%" #}
cheaper_than: {{ _('cheaper than {pct} of tracked prices')|tojson }},
{# TRANSLATORS: {pct} is a percentage including the % sign, e.g. "80%" #}
pricier_than: {{ _('more expensive than {pct} of tracked prices')|tojson }},
typical_note: {{ _('around the usual price')|tojson }},
avg_label: {{ _('avg')|tojson }}
};
@@ -33,6 +33,7 @@
<a href="https://RegExr.com/">{{ _('Be sure to test your RegEx here.') }}</a>
</p>
<p>
{# TRANSLATORS: FIXME: fragment msgids (#4074) #}
{{ _('Each RegEx group bracket') }} <code>()</code> {{ _('will be in its own column, the first column value is always the date.') }}
</p>
</span>
@@ -216,7 +216,8 @@ def render(watch, datastore, request, url_for, render_template, flash, redirect,
#initial_scroll_line_number=100,
bottom_horizontal_offscreen_contents=offscreen_content,
content=content,
current_diff_url=watch['url'],
current_diff_label=watch.label,
current_diff_url=watch.open_link,
diff_cell_grid=diff_cell_grid,
diff_prefs=diff_prefs,
extra_classes=' '.join(filter(None, ['difference-page', 'llm-configured' if llm_configured else ''])),
@@ -35,6 +35,11 @@ class SignalHandler:
watch_small_status_comment_signal = signal('watch_small_status_comment')
watch_small_status_comment_signal.connect(self.handle_watch_small_status_update, weak=False)
# The AI summary generator runs on a background thread; this lets the browser skip
# polling for the result.
llm_summary_ready_signal = signal('llm_summary_ready')
llm_summary_ready_signal.connect(self.handle_llm_summary_ready, weak=False)
# Connect to the notification_event signal
notification_event_signal = signal('notification_event')
notification_event_signal.connect(self.handle_notification_event, weak=False)
@@ -114,6 +119,23 @@ class SignalHandler:
})
logger.debug(f"Watch UUID {watch_uuid} got its favicon updated")
def handle_llm_summary_ready(self, *args, **kwargs):
"""An on-demand AI change summary finished generating (successfully or not).
Only identifiers are broadcast: the client turns this into one authenticated request to
the /llm-summary poll route, so the summary text itself never goes to every connected
browser. Clients that do not care about this watch simply ignore the event.
"""
watch_uuid = kwargs.get('watch_uuid')
if watch_uuid:
self.socketio_instance.emit("llm_summary_ready", {
"uuid": watch_uuid,
"from_version": kwargs.get('from_version'),
"to_version": kwargs.get('to_version'),
"event_timestamp": time.time()
})
logger.debug(f"Watch UUID {watch_uuid} AI summary job settled")
def handle_deleted_signal(self, *args, **kwargs):
watch_uuid = kwargs.get('watch_uuid')
if watch_uuid:
+6
View File
@@ -9,6 +9,12 @@
# exit when any command fails
set -e
# Failing fast is deliberate here (the first failure is usually the real problem, and it keeps
# the run short) - but this script runs 8 independent pytest groups, so make it obvious that the
# groups after the failure were SKIPPED rather than passed. Otherwise one failing test reads as
# "the whole basic suite is broken".
trap 'rc=$?; echo "::error::run_basic_tests.sh aborted at line $LINENO (exit $rc) - the test groups after this point were SKIPPED, not run"' ERR
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
rm tests/logs/* -f
Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

@@ -1,25 +0,0 @@
{
"name": "ChangeDetection.io",
"short_name": "ChangeDetect",
"description": "Self-hosted website change detection and monitoring",
"icons": [
{
"src": "android-chrome-192x192.png",
"sizes": "192x192",
"type": "image/png",
"purpose": "any maskable"
},
{
"src": "android-chrome-256x256.png",
"sizes": "256x256",
"type": "image/png",
"purpose": "any maskable"
}
],
"start_url": "/",
"theme_color": "#5bbad5",
"background_color": "#ffffff",
"display": "standalone",
"categories": ["utilities", "productivity"],
"orientation": "any"
}
+4 -2
View File
@@ -78,7 +78,9 @@ $(document).ready(function () {
// bootstrap it, this will trigger everything else
$('#browsersteps-img').bind('load', function () {
$('body').addClass('full-width');
console.log("Loaded background...");
console.log(`Loaded background ${this.naturalWidth}px` );
// For the UI width of the whole edit area
document.documentElement.style.setProperty('--browser-steps-max-width', `${this.naturalWidth+200}px` );
document.getElementById("browsersteps-selector-canvas");
c = document.getElementById("browsersteps-selector-canvas");
@@ -284,7 +286,7 @@ $(document).ready(function () {
$('#browser-steps-ui .loader .spinner').show();
// Request a new session
$.ajax({
type: "GET",
type: "POST",
url: browser_steps_start_url,
statusCode: {
400: function () {
+19 -8
View File
@@ -1,10 +1,21 @@
$(document).ready(function () {
$.ajaxSetup({
beforeSend: function (xhr, settings) {
if (!/^(GET|HEAD|OPTIONS|TRACE)$/i.test(settings.type) && !this.crossDomain) {
xhr.setRequestHeader("X-CSRFToken", csrftoken)
}
// Attach the CSRF token to every state-changing jQuery request.
//
// Applied at script-execution time, NOT inside $(document).ready(). That matters: pages carry
// inline <script> blocks that register their own ready handlers while the document is still
// parsing, whereas this file is loaded with `defer` and therefore runs afterwards. jQuery
// invokes ready callbacks in registration order, so a ready handler registered by an inline
// block always runs BEFORE one registered here - and any POST it fires would go out with no
// token and come back "400 The CSRF token is missing".
//
// That is exactly what happened to the AI summary on /diff once its start call became a POST
// (#4467): diff.html's inline $(function(){...}) calls llmSummary.fetch() immediately.
//
// $.ajaxSetup needs no DOM, so doing it now closes the window entirely - deferred scripts all
// run before DOMContentLoaded, so the setup is in place before any ready handler can fire.
$.ajaxSetup({
beforeSend: function (xhr, settings) {
if (!/^(GET|HEAD|OPTIONS|TRACE)$/i.test(settings.type) && !this.crossDomain) {
xhr.setRequestHeader("X-CSRFToken", csrftoken)
}
})
}
});
+260 -79
View File
@@ -1,74 +1,205 @@
// Previous / Next step both version selects one position back or forward and
// rebuild the href from the current diff options; the arrow keys do the same
// thing. Recomputed at press time rather than cached, because changing either
// select submits the form and re-renders them.
function diffStepHref(direction) {
var $from = $('#diff-from-version option:selected')[direction]();
var $to = $('#diff-to-version option:selected')[direction]();
if (!$from.length || !$to.length) {
return null;
}
var params = new URLSearchParams(window.location.search);
params.set('from_version', $from.val());
params.set('to_version', $to.val());
return '?' + params.toString();
}
function setupDiffNavigation() {
var $fromSelect = $('#diff-from-version');
var $toSelect = $('#diff-to-version');
var $fromSelected = $fromSelect.find('option:selected');
var $toSelected = $toSelect.find('option:selected');
var BUTTONS = {prev: '#btn-previous', next: '#btn-next'};
if ($fromSelected.length && $toSelected.length) {
// Find the previous pair (move both back one position)
var $prevFrom = $fromSelected.prev();
var $prevTo = $toSelected.prev();
if ($('#diff-from-version option:selected').length && $('#diff-to-version option:selected').length) {
$.each(BUTTONS, function (direction, selector) {
var href = diffStepHref(direction);
// Nothing to step to that way: drop the button rather than leave a
// dead one on the bar.
if (href) {
$(selector).attr('href', href);
} else {
$(selector).remove();
}
});
}
// Find the next pair (move both forward one position)
var $nextFrom = $fromSelected.next();
var $nextTo = $toSelected.next();
$(window).on('keydown', function (event) {
// Not while someone is typing or working a select.
if (/^(INPUT|TEXTAREA|SELECT)$/.test(event.target.tagName)) {
return;
}
var direction = {ArrowLeft: 'prev', ArrowRight: 'next'}[event.key];
var href = direction && diffStepHref(direction) && $(BUTTONS[direction]).attr('href');
if (href) {
event.preventDefault();
window.location.href = href;
}
});
}
// Build URL with current diff preferences
var currentParams = new URLSearchParams(window.location.search);
// The seven diff options collapse behind the 'Filters' button so the sticky bar
// stays one toolbar row. Wired up here rather than purely in CSS: until this
// runs the fieldset is inline and the toggle hidden, so with scripting off the
// options are still reachable and still submit with the form.
function setupDiffFilters() {
var $header = $('#diff-header');
var $toggle = $('#diff-filters-toggle');
var $panel = $('#diff-style');
if (!$header.length || !$toggle.length || !$panel.length) {
return;
}
var header = $header[0], toggle = $toggle[0], panel = $panel[0];
// Previous button: only show if both can move back
if ($prevFrom.length && $prevTo.length) {
currentParams.set('from_version', $prevFrom.val());
currentParams.set('to_version', $prevTo.val());
$('#btn-previous').attr('href', '?' + currentParams.toString());
} else {
$('#btn-previous').remove();
$header.addClass('diff-filters-js');
function isOpen() {
return $header.hasClass('diff-filters-open');
}
var EDGE_GAP = 8; // keep the panel clear of the viewport edges
var BUTTON_GAP = 4; // between the toggle and the panel
// position: fixed, because #diff-header's overflow: auto would clip an absolute
// panel - which means parking it by hand, and that scrolling will never bring a
// row hanging off the bottom back into reach. So it has to fit at placement time
// or not at all: cap it to the room there is and let it scroll (see diff.scss).
function place() {
var button = toggle.getBoundingClientRect();
// A fixed element is positioned against the layout viewport, but iOS
// shrinks the *visual* one behind its toolbars, so budget with whichever
// is smaller.
var viewportHeight = document.documentElement.clientHeight;
if (window.visualViewport) {
viewportHeight = Math.min(viewportHeight, window.visualViewport.height);
}
// Next button: only show if both can move forward
if ($nextFrom.length && $nextTo.length) {
currentParams.set('from_version', $nextFrom.val());
currentParams.set('to_version', $nextTo.val());
$('#btn-next').attr('href', '?' + currentParams.toString());
} else {
$('#btn-next').remove();
// Measure unconstrained: a cap left over from the previous placement would
// otherwise read back as the panel's natural height.
panel.style.maxHeight = '';
// Pick the roomier side rather than assume one: below used to be roomier by
// construction, but diff.scss's short-viewport breakpoint drops the bar's
// 50svh cap and at 280x300 above wins 147.6px to 104.2px.
var roomBelow = viewportHeight - button.bottom - BUTTON_GAP - EDGE_GAP;
var roomAbove = button.top - BUTTON_GAP - EDGE_GAP;
var above = roomAbove > roomBelow;
var room = above ? roomAbove : roomBelow;
if (panel.offsetHeight > room) {
// max-height caps the content box, and the panel must stay content-box
// (border-box would fold the padding into min-width and narrow it by
// 24px) - so subtract its own padding and borders.
var style = window.getComputedStyle(panel);
var trim = parseFloat(style.paddingTop) + parseFloat(style.paddingBottom) +
parseFloat(style.borderTopWidth) + parseFloat(style.borderBottomWidth);
panel.style.maxHeight = Math.max(room - trim, 0) + 'px';
}
// offsetHeight after the cap: parking above measures from the panel's bottom.
var top = above ? button.top - BUTTON_GAP - panel.offsetHeight
: button.bottom + BUTTON_GAP;
panel.style.top = top + 'px';
var available = document.documentElement.clientWidth - panel.offsetWidth - EDGE_GAP;
var left = Math.max(EDGE_GAP, Math.min(button.left, available));
// The panel paints the page's own backdrop (see diff.scss), aligned by
// geometry because iOS Safari ignores background-attachment: fixed. Negative
// because the copy's top-left corner belongs at the viewport's.
$panel.css({left: left + 'px', backgroundPosition: (-left) + 'px ' + (-top) + 'px'});
}
function open() {
$header.addClass('diff-filters-open');
$toggle.attr('aria-expanded', 'true');
place();
}
function close(restoreFocus) {
if (!isOpen()) {
return;
}
$header.removeClass('diff-filters-open');
$toggle.attr('aria-expanded', 'false');
if (restoreFocus) {
$toggle.trigger('focus');
}
}
// Keyboard navigation
window.addEventListener('keydown', function (event) {
// Don't trigger if user is typing in an input field
if (event.target.tagName === 'INPUT' || event.target.tagName === 'TEXTAREA' || event.target.tagName === 'SELECT') {
$toggle.on('click', function (event) {
event.preventDefault();
if (isOpen()) {
close(false);
} else {
open();
}
});
$(document).on('click', function (event) {
if (isOpen() && !panel.contains(event.target) && !toggle.contains(event.target)) {
close(false);
}
});
$(document).on('keydown', function (event) {
if (event.key === 'Escape') {
close(true);
}
});
// A tab switch hides #settings and the panel with it, so drop the open state
// rather than leave aria-expanded lying.
$(window).on('hashchange', function () {
close(false);
});
$(window).on('resize', function () {
if (isOpen()) {
place();
}
});
// The button can drift out from under the fixed panel two ways: #settings
// scrolling inside a sticky bar (16px at 390x390, 30px at 320x480), or the whole
// bar scrolling away below diff.scss's max-height: 500px breakpoint. Opposite
// causes, one follow/close rule; each closing test is inert in the other mode.
function followButton() {
if (!isOpen()) {
return;
}
var $fromSelected = $fromSelect.find('option:selected');
var $toSelected = $toSelect.find('option:selected');
if ($fromSelected.length && $toSelected.length) {
if (event.key === 'ArrowLeft') {
var $prevFrom = $fromSelected.prev();
var $prevTo = $toSelected.prev();
if ($prevFrom.length && $prevTo.length) {
var prevHref = $('#btn-previous').attr('href');
if (prevHref) {
event.preventDefault();
window.location.href = prevHref;
}
}
} else if (event.key === 'ArrowRight') {
var $nextFrom = $fromSelected.next();
var $nextTo = $toSelected.next();
if ($nextFrom.length && $nextTo.length) {
var nextHref = $('#btn-next').attr('href');
if (nextHref) {
event.preventDefault();
window.location.href = nextHref;
}
}
}
var bar = header.getBoundingClientRect();
var button = toggle.getBoundingClientRect();
var viewportHeight = document.documentElement.clientHeight;
if (button.bottom <= bar.top || button.top >= bar.bottom ||
button.bottom <= 0 || button.top >= viewportHeight) {
close(false);
} else {
place();
}
}, false);
}
// Native, not jQuery: .on() cannot pass passive or capture. Capture is required
// because scroll does not bubble and the element that scrolls is #settings, a
// descendant; the window registration covers the static mode's document scroll.
header.addEventListener('scroll', followButton, {passive: true, capture: true});
window.addEventListener('scroll', followButton, {passive: true});
// On iOS the visual viewport shrinks on its own - a toolbar or the keyboard -
// without resizing the layout viewport, so no window resize fires and the cap
// place() budgeted stays wrong. Reproduced in Chromium via CDP pinch-zoom.
if (window.visualViewport) {
window.visualViewport.addEventListener('resize', function () {
if (isOpen()) {
place();
}
});
}
}
$(document).ready(function () {
@@ -87,10 +218,70 @@ $(document).ready(function () {
setupDiffNavigation();
}
setupDiffFilters();
// Load it when the #screenshot tab is in use, so we dont give a slow experience when waiting for the text diff to load
window.addEventListener('hashchange', function (e) {
$(window).on('hashchange', function () {
toggle(location.hash);
}, false);
realignPane();
});
// Where the last alignment we performed left the page, so a later one can tell
// "still where we put them" from "the reader has moved since", and the
// scroll-margin-top it was computed against.
var alignedY = null;
var alignedMargin = null;
function targetPane() {
var pane = location.hash.length > 1 &&
document.getElementById(location.hash.slice(1));
return pane && pane.classList.contains('tab-pane-inner') ? pane : null;
}
// The fragment jump runs before toggle() hides #settings, which collapses the
// ~8000px document; the engine then clamps its held offset rather than re-running
// the jump, and on iOS that clamp is never 0. So re-run it once the layout has
// settled. scrollIntoView, not scrollTo(0, 0): .tab-pane-inner already declares
// the sticky stack's offset as scroll-margin-top. setTimeout(0) rather than rAF,
// because that margin resolves against --diff-header-height, which
// diff-render.js's ResizeObserver updates after frame callbacks have run.
function realignPane() {
var pane = targetPane();
if (!pane) {
return;
}
setTimeout(function () {
alignedMargin = parseFloat(getComputedStyle(pane).scrollMarginTop);
pane.scrollIntoView({block: 'start', inline: 'nearest'});
alignedY = window.scrollY;
}, 0);
}
// The load path is stale for a nearer reason: tabs.js rewrites an empty hash and
// the browser jumps before diff-render.js has measured the bar, so
// scroll-margin-top is still on its --diff-header-height: 0 fallback. Traced at
// 844x390: landed at scrollY 231 against a 16px margin that then became 202px.
// Below the short-viewport breakpoint the bar is in flow, so that puts every
// control off the top of the page. Re-run when the bar resizes - comparing the
// margin, so it only fires on a real change - and only while the reader is still
// where we left them, since rotating the phone resizes the bar too.
var bar = document.getElementById('diff-header');
if (bar && typeof ResizeObserver !== 'undefined') {
new ResizeObserver(function () {
var pane = targetPane();
if (!pane) {
return;
}
if (alignedY !== null && Math.abs(window.scrollY - alignedY) > 2) {
return;
}
var margin = parseFloat(getComputedStyle(pane).scrollMarginTop);
if (alignedMargin !== null && Math.abs(margin - alignedMargin) < 0.5) {
return;
}
realignPane();
}).observe(bar);
}
toggle(location.hash);
@@ -108,23 +299,14 @@ $(document).ready(function () {
}
}
const article = $('#difference')[0];
// We could also add the 'touchend' event for touch devices, but since most
// iOS/Android browsers already show a dialog when you select text (often with a
// Share option) we'll skip that. mouseup goes on the page rather than the
// article, because they might 'mouse up' outside it.
$('#difference').on('mousedown', clean);
$('.difference-page').on('mouseup', dragTextHandler);
// We could also add the 'touchend' event for touch devices, but since
// most iOS/Android browsers already show a dialog when you select
// text (often with a Share option) we'll skip that
if (article) {
article.addEventListener('mousedown', clean, false);
}
// Because they might 'mouse up' outside the article but on the page
const d_page = $(".difference-page")[0]
if (d_page ) {
d_page.addEventListener('mouseup', dragTextHandler, false);
}
$('#highlightSnippetActions a').bind('click', function (e) {
$('#highlightSnippetActions a').on('click', function (e) {
if (!window.getSelection().toString().trim().length) {
alert('Oops no text selected!');
return;
@@ -155,12 +337,11 @@ $(document).ready(function () {
$('#bottom-horizontal-offscreen').hide();
}
// Listen for Escape key press
window.addEventListener('keydown', function (e) {
$(window).on('keydown', function (e) {
if (e.key === 'Escape') {
clean();
}
}, false);
});
function dragTextHandler(event) {
console.log('mouseupped');
+58 -9
View File
@@ -11,6 +11,59 @@ $(document).ready(function () {
var visualizerResolutionCells = $cells.length;
var cellHeight;
var header = document.getElementById('diff-header');
// The app top menu, sticky above #diff-header on the diff page.
var appHeader = document.querySelector('.app-main > .header');
// Controls can wrap or disappear when switching tabs, and the top menu
// rewraps on narrow viewports. Keep each sticky layer, the minimap and the
// anchor links below the actual measured heights instead of fixed offsets.
// Order matters: #diff-header's max-height is calc(50dvh - the app header),
// so store that first and measure the diff header against the new cap. The
// other way round reads it under the previous cap and one pass is not enough
// to settle.
function updateHeaderHeight() {
if (appHeader) {
document.body.style.setProperty('--app-header-height', appHeader.offsetHeight + 'px');
}
if (header) {
document.body.style.setProperty('--diff-header-height', header.offsetHeight + 'px');
}
}
if (header || appHeader) {
// Measure once regardless of observer support, so the offsets are never
// left at their 0 fallback.
updateHeaderHeight();
if (typeof ResizeObserver !== 'undefined') {
var headerObserver = new ResizeObserver(updateHeaderHeight);
if (header) headerObserver.observe(header);
if (appHeader) headerObserver.observe(appHeader);
} else {
// Without it, catch the two things that actually resize the bars:
// the window rewrapping them, and the tab switch that shows or hides
// #settings. diff-overview.js toggles that from its own hashchange
// handler, so defer past it rather than depend on listener order.
$(window).on('resize.diffheader', updateHeaderHeight.debounce(100));
$(window).on('hashchange.diffheader', function () {
setTimeout(updateHeaderHeight, 0);
});
// Opening a link straight at #screenshot / #extract hides #settings
// from diff-overview.js's ready handler, with no hashchange to
// follow. Re-measure once the whole ready pass has run.
setTimeout(updateHeaderHeight, 0);
}
}
// Centre of the region left visible below the sticky stack, not of the whole
// viewport - every sticky layer above the diff has to be counted or jumps
// land half its height too high.
function viewportCenterOffset() {
var appHeaderHeight = appHeader ? appHeader.offsetHeight : 0;
var headerHeight = header ? header.offsetHeight : 0;
var visualizerHeight = $visualizer.is(':visible') ? $visualizer.outerHeight() : 0;
return (appHeaderHeight + headerHeight + visualizerHeight + $(window).height()) / 2;
}
if ($difference.length && visualizerResolutionCells > 0) {
var docHeight = $difference[0].scrollHeight;
cellHeight = docHeight / visualizerResolutionCells;
@@ -21,11 +74,10 @@ $(document).ready(function () {
$(this).on('click', function() {
var cellIndex = $(this).data('cellIndex');
var targetPositionInDifference = cellIndex * cellHeight;
var viewportHeight = $(window).height();
// Scroll so target is at viewport center (where eyes expect it)
window.scrollTo({
top: $difference.offset().top + targetPositionInDifference - (viewportHeight / 2),
top: $difference.offset().top + targetPositionInDifference - viewportCenterOffset(),
behavior: "smooth"
});
});
@@ -37,8 +89,7 @@ $(document).ready(function () {
// Find the next change after current scroll position
var currentScrollPos = $(window).scrollTop();
var viewportHeight = $(window).height();
var currentCenter = currentScrollPos + (viewportHeight / 2);
var currentCenter = currentScrollPos + viewportCenterOffset();
// Add small buffer (50px) to jump past changes already near center
var searchFromPosition = currentCenter + 50;
@@ -59,7 +110,7 @@ $(document).ready(function () {
// Scroll to position the element at viewport center
var elementTop = $(nextElement).offset().top;
var targetScrollPos = elementTop - (viewportHeight / 2);
var targetScrollPos = elementTop - viewportCenterOffset();
window.scrollTo({
top: targetScrollPos,
@@ -73,7 +124,7 @@ $(document).ready(function () {
var scrollTop = $(window).scrollTop();
var viewportHeight = $(window).height();
var viewportCenter = scrollTop + (viewportHeight / 2);
var viewportCenter = scrollTop + viewportCenterOffset();
var differenceTop = $difference.offset().top;
var differenceHeight = $difference[0].scrollHeight;
var positionInDifference = viewportCenter - differenceTop;
@@ -129,10 +180,9 @@ $(document).ready(function () {
var estimatedTop = (charPosition / totalChars) * totalHeight;
// Scroll to position with line at viewport center
var viewportHeight = $(window).height();
setTimeout(function() {
window.scrollTo({
top: $difference.offset().top + estimatedTop - (viewportHeight / 2),
top: $difference.offset().top + estimatedTop - viewportCenterOffset(),
behavior: "smooth"
});
}, 100); // Small delay to ensure page is fully loaded
@@ -149,4 +199,3 @@ $(document).ready(function () {
}
});
@@ -1,4 +1,16 @@
$(document).ready(function () {
// Secret fields (the LLM API key) render with readonly set, because Chrome ignores
// autocomplete="off" on type=password and will happily drop the saved site password in.
// That is destructive rather than merely annoying: the field renders blank so that
// saving it untouched PRESERVES the stored key, so anything prefilled silently replaces
// a working key on the next Save, with no copy kept anywhere to restore from.
//
// readonly rather than disabled - a disabled input is never submitted, so the key could
// never be set at all. Unlocked on focus as well as click so keyboard tabbing works.
$('[data-unlock-on-interact]').on('focus click', function () {
$(this).removeAttr('readonly');
});
$("#api-key").hover(
function () {
$("#api-key-copy").html('copy').fadeIn();
+211
View File
@@ -0,0 +1,211 @@
/**
* AI change summary: start with POST, then wait for the result.
*
* Generation is an LLM round-trip, so the server does not hold the request open for it - the POST
* returns 202 "pending" and the summary lands in the watch's cache some seconds later. There are
* two ways to find out it arrived, and which one is in use is logged to the console:
*
* realtime Socket.IO is connected, so we sit idle until the server emits `llm_summary_ready`
* for this watch and then make exactly one request. No polling.
* polling Socket.IO is off (settings > UI, batch mode) or not connected, so we re-check the
* route on an interval.
*
* Realtime carries identifiers only, never the summary text - the event just means "go look", and
* the actual read still goes through the authenticated route. A watchdog re-checks once in a long
* while even in realtime mode, so a dropped event cannot leave a permanent spinner.
*
* Why the start is a POST: it spends tokens and marks the watch as viewed, so it must not be
* reachable from a cross-site GET. csrf.js ($.ajaxSetup) attaches X-CSRFToken to every non-GET
* jQuery request, so nothing here has to handle the token. Parameters go in the query string for
* both verbs (not the POST body) so the server reads them from request.args either way.
*
* When to stop waiting is the server's call, not ours. A local Ollama/vLLM box can spend many
* minutes on prompt prefill and the server grants it up to LLM_LOCAL_TIMEOUT (30 min by default),
* where a cloud provider gets LLM_TIMEOUT (5 min); on top of that the job may have been started
* by another tab minutes ago, so even the right duration measured from our own first request is
* the wrong answer. Every 'pending' reply therefore carries the deadline (`expires_in`, plus
* `timeout_at` as the absolute instant) and we adopt it. Giving up earlier than the server does
* means reporting a failure for a job that is still running and will still write its summary -
* which is exactly what a fixed 180s limit used to do against a local model.
*/
window.llmSummary = (function () {
var POLL_MS = 1500; // polling mode: first re-check
var POLL_MAX_MS = 15000; // ...backing off to this, so a slow local model is not hammered
var POLL_BACKOFF = 1.5;
var WATCHDOG_MS = 45000; // realtime mode: re-check once if no event turns up
var MAX_WAIT_MS = 180000; // fallback only - replaced by the deadline the server sends
var MAX_RESTARTS = 1; // an 'idle' reply means the job vanished (server restart); retry once
var LOG = '[llm-summary]';
function withQuery(url, params) {
var qs = params ? $.param(params) : '';
if (!qs) return url;
return url + (url.indexOf('?') >= 0 ? '&' : '?') + qs;
}
function errorFromXhr(xhr) {
var body = xhr.responseJSON;
if (body && body.error) return body.error;
return 'AI summary request failed (HTTP ' + xhr.status + ').';
}
function realtimeSocket() {
// realtime.js exposes the connection as window.cdioSocket when socket_io_enabled.
var socket = window.cdioSocket;
return (socket && socket.connected) ? socket : null;
}
/**
* @param {object} opts
* url {string} the /llm-summary route for one watch
* uuid {string} watch uuid, used to match realtime events (optional - without it
* this falls back to polling)
* params {object} query parameters (version pair, diff prefs) or null
* maxWaitMs {number} how long to wait before the server has told us otherwise
* (optional); each 'pending' reply can push the deadline out
* onDone {function} (summary, data)
* onError {function} (message)
* isCancelled {function} () => bool, stops all work when it returns true
*/
function fetchSummary(opts) {
var target = withQuery(opts.url, opts.params);
var restarts = 0;
var isCancelled = opts.isCancelled || function () { return false; };
var socket = opts.uuid ? realtimeSocket() : null;
var watchdog = null;
var finished = false;
var onReady = null;
var deadline = Date.now() + (opts.maxWaitMs || MAX_WAIT_MS);
var pollDelay = POLL_MS;
console.log(LOG, socket
? 'realtime available - waiting for llm_summary_ready event (no polling)'
: 'realtime unavailable - polling from ' + POLL_MS + 'ms, backing off to '
+ POLL_MAX_MS + 'ms');
function cleanup() {
finished = true;
if (watchdog) { clearTimeout(watchdog); watchdog = null; }
if (socket && onReady) { socket.off('llm_summary_ready', onReady); onReady = null; }
}
function done(data) {
cleanup();
opts.onDone(data.summary, data);
}
function fail(message) {
cleanup();
opts.onError(message);
}
function expired() {
return Date.now() > deadline;
}
function adoptServerDeadline(data) {
// Counted off `expires_in` rather than `timeout_at`, because that arrives as an
// absolute server-side epoch and a browser clock minutes out of step with the server
// would expire the wait the moment it was set. Only ever moves the deadline out: a
// queued job reports a deadline that keeps sliding as it waits its turn, and a
// shorter answer from a later poll must not cut short a wait already in progress.
if (!data || typeof data.expires_in !== 'number') return;
var wanted = Date.now() + (data.expires_in * 1000);
if (wanted > deadline) {
deadline = wanted;
console.log(LOG, 'server will keep working for another ' + data.expires_in
+ 's - waiting with it');
}
}
function stillPending() {
// Nothing to do but wait: in realtime mode the event will tell us, in polling mode
// the timer will.
if (expired()) {
// The last thing the server told us was 'pending', so the job has *not* failed -
// it is still generating and will write to the cache when it lands. Say that,
// instead of reporting a timeout the summary never had.
fail('The AI summary is still being generated and is taking longer than usual - '
+ 'reload this page shortly to see it.');
return;
}
if (socket) {
armWatchdog();
} else {
setTimeout(check, pollDelay);
// Back off: a local model can run for minutes, and a 1.5s poll for all of it is
// hundreds of pointless requests.
pollDelay = Math.min(Math.round(pollDelay * POLL_BACKOFF), POLL_MAX_MS);
}
}
function armWatchdog() {
if (watchdog) clearTimeout(watchdog);
watchdog = setTimeout(function () {
if (finished || isCancelled()) return;
if (!realtimeSocket()) {
console.log(LOG, 'socket dropped while waiting - switching to polling');
socket = null;
check();
return;
}
console.log(LOG, 'no llm_summary_ready event after ' + WATCHDOG_MS + 'ms - re-checking once');
check();
}, WATCHDOG_MS);
}
function handle(data, fromStart) {
if (finished || isCancelled()) return;
if (data.summary) { done(data); return; }
if (data.error) { fail(data.error); return; }
adoptServerDeadline(data);
if (data.status === 'idle') {
// No cached summary and no job running: the process restarted mid-generation.
if (fromStart || restarts >= MAX_RESTARTS) {
fail('AI summary could not be started.');
return;
}
restarts++;
pollDelay = POLL_MS;
console.log(LOG, 'job had vanished - restarting generation');
start();
return;
}
stillPending();
}
function check() {
if (finished || isCancelled()) return;
$.ajax({url: target, type: 'GET', dataType: 'json'})
.done(function (data) { handle(data, false); })
.fail(function (xhr) { if (!finished && !isCancelled()) fail(errorFromXhr(xhr)); });
}
function start() {
$.ajax({url: target, type: 'POST', dataType: 'json'})
.done(function (data) { handle(data, true); })
.fail(function (xhr) { if (!finished && !isCancelled()) fail(errorFromXhr(xhr)); });
}
if (socket) {
// Matched on uuid alone: the event cannot know which version pair this particular
// view asked for, so a match only means "worth a look". The request that follows is
// authoritative - if it comes back pending, the event was for a different pair and we
// keep waiting.
onReady = function (data) {
if (finished || isCancelled() || !data || data.uuid !== opts.uuid) return;
console.log(LOG, 'llm_summary_ready for', data.uuid, '- fetching summary');
check();
};
socket.on('llm_summary_ready', onReady);
}
start();
}
return {fetch: fetchSummary};
})();
+1 -1
View File
@@ -73,7 +73,7 @@ $(function () {
// Request start, needs CSRF?
$.ajax({
type: "GET",
type: "POST",
url: recheck_proxy_start_url,
}).done(function (data) {
$.each(data, function (proxy_key, state) {
@@ -23,8 +23,11 @@
no_data: 'No price data available to graph yet.', load_error: 'Could not load price history.',
changes: 'Changes', avg_price: 'Average price',
price_low: 'Currently low', price_typical: 'Currently typical', price_high: 'Currently high',
cheaper_than: 'cheaper than %s% of tracked prices',
pricier_than: 'more expensive than %s% of tracked prices',
// {pct} is substituted with an already-formatted percentage, e.g. "80%". The percent sign is
// NOT part of the msgid on purpose: a bare "%" followed by a letter (" of") parses as a
// python-format conversion ("% o"), which makes pybabel reject any translation of it.
cheaper_than: 'cheaper than {pct} of tracked prices',
pricier_than: 'more expensive than {pct} of tracked prices',
typical_note: 'around the usual price', avg_label: 'avg' };
const OVERLAY_MIN_POINTS = 5; // need enough history for low/typical/high to be meaningful
@@ -260,9 +263,9 @@
$pill.append($('<span class="rg-status-label"></span>').text(i18n['price_' + summary.status] || ''));
let sub;
if (summary.status === 'low') {
sub = (i18n.cheaper_than || '').replace('%s', summary.cheaper_than_pct);
sub = (i18n.cheaper_than || '').replace('{pct}', summary.cheaper_than_pct + '%');
} else if (summary.status === 'high') {
sub = (i18n.pricier_than || '').replace('%s', summary.pricier_than_pct);
sub = (i18n.pricier_than || '').replace('{pct}', summary.pricier_than_pct + '%');
} else {
sub = i18n.typical_note || '';
}
@@ -0,0 +1,30 @@
// The watch title in the top menu is a horizontal scroll container (see
// parts/_top_menu.scss), so a long one can be dragged or swiped to read its end.
// The CSS ships a fixed right-hand fade for the no-JS case; once this is
// running, the fade has to follow the scroll instead, or the ending the reader
// just dragged over is the part that is dimmed.
//
// Global rather than part of diff-overview.js: the restock difference page and
// the image-SSIM preview page render the same line and neither loads that file.
$(function () {
var $title = $('.current-diff-url');
if (!$title.length) return;
var el = $title[0];
function updateFades() {
// The 1px slack absorbs the sub-pixel gap a fractional layout leaves
// between scrollWidth and clientWidth, which would otherwise hold
// fade-right on for a line that is not actually cut off.
$title
.toggleClass('fade-left', el.scrollLeft > 1)
.toggleClass('fade-right', el.scrollLeft + el.clientWidth < el.scrollWidth - 1);
}
// The class and the first state are set in the same task, so the no-JS mask
// is never seen to blink off before its replacement arrives.
$title.addClass('js-fades').on('scroll', updateFades);
// Resizing changes what fits without scrolling the line, so the state can
// become wrong without a scroll event: rotating a phone is the common case.
$(window).on('resize', updateFades);
updateFades();
});
+12 -39
View File
@@ -7,7 +7,6 @@
// The Search button is rendered in the left rail and the mobile drawer.
const openSearchButtons = document.querySelectorAll('.js-open-search-modal');
const closeSearchButton = document.getElementById('close-search-modal');
const searchForm = document.getElementById('search-form');
const searchInput = document.getElementById('search-modal-input');
if (!searchModal || openSearchButtons.length === 0) {
@@ -63,6 +62,15 @@
// Close modal when clicking the backdrop
searchModal.addEventListener('click', function(e) {
// Only real pointer clicks can land on the backdrop. Keyboard-synthesised clicks
// report detail 0 and coordinates of 0,0, which the geometry test below reads as
// "outside the dialog" - and implicit form submission (Enter in the input) fires
// exactly such a click at the Search button. That closed the modal and blanked
// the input mid-dispatch, so the submit that followed hit an empty `required`
// field and was rejected: Enter appeared to just dismiss the form.
if (e.detail === 0) {
return;
}
const rect = searchModal.getBoundingClientRect();
const isInDialog = (
rect.top <= e.clientY &&
@@ -93,43 +101,8 @@
}
});
// Handle Enter key in search input
if (searchInput) {
searchInput.addEventListener('keydown', function(e) {
if (e.key === 'Enter') {
e.preventDefault();
if (searchForm) {
// Trigger form submission programmatically
searchForm.dispatchEvent(new Event('submit'));
}
}
});
}
// Handle form submission
if (searchForm) {
searchForm.addEventListener('submit', function(e) {
e.preventDefault();
// Get form data
const formData = new FormData(searchForm);
const searchQuery = formData.get('q');
const tags = formData.get('tags');
// Build URL
const params = new URLSearchParams();
if (searchQuery) {
params.append('q', searchQuery);
}
if (tags) {
params.append('tags', tags);
}
// Navigate to search results (always redirect to watchlist home)
// Use base_path if available (for sub-path deployments like /enlighten-richerx)
const basePath = typeof base_path !== 'undefined' ? base_path : '';
window.location.href = basePath + '/?' + params.toString();
});
}
// Submission is left to the browser: the form carries a server-rendered action
// (correct under a reverse-proxy sub-path) and Enter in the input triggers implicit
// submission via the footer's submit button, which also runs `required` validation.
});
})();
+9 -7
View File
@@ -1,13 +1,15 @@
// Left-rail expand/collapse state.
// Adds `action-side-bar-expanded` to <body> whenever the rail is showing its
// labels. In pinned mode (body.actionside-bar-on) the class is already present
// from page load; in collapsed mode (body.actionsidebar-minimal) the rail only
// expands on hover/focus, so we toggle the class to match.
// Adds `actionsidebar-expanded` to <body> whenever the rail is showing its
// labels. In 'pinned-expanded' mode (body.actionside-bar-on) the class is already
// present from page load; in 'expandable' mode (body.actionsidebar-minimal) the rail
// only expands on hover/focus, so we toggle the class to match. 'minimal' mode is
// also the collapsed rail but carries `actionsidebar-no-expand` and never rolls out.
(function() {
'use strict';
document.addEventListener('DOMContentLoaded', function() {
if (!document.body.classList.contains('actionsidebar-minimal')) {
if (!document.body.classList.contains('actionsidebar-minimal') ||
document.body.classList.contains('actionsidebar-no-expand')) {
return;
}
@@ -16,8 +18,8 @@
return;
}
const expand = () => document.body.classList.add('action-side-bar-expanded');
const collapse = () => document.body.classList.remove('action-side-bar-expanded');
const expand = () => document.body.classList.add('actionsidebar-expanded');
const collapse = () => document.body.classList.remove('actionsidebar-expanded');
inner.addEventListener('mouseenter', expand);
inner.addEventListener('mouseleave', collapse);
@@ -115,7 +115,9 @@ window.initVisualSelector = function (opts) {
$('#selector-current-xpath, #clear-selector').hide();
})
.on('load', () => {
console.log("Loaded background...");
console.log(`Loaded background ${$selectorBackgroundElem[0].naturalWidth}px`);
// For the UI width of the whole edit area
document.documentElement.style.setProperty('--visualselector-max-width', `${$selectorBackgroundElem[0].naturalWidth}px` );
c = $selectorCanvasElem[0];
xctx = c.getContext("2d");
ctx = c.getContext("2d");
+19 -22
View File
@@ -460,7 +460,7 @@ $(function () {
var promptUrl = url + '/prompt';
// Fire both requests simultaneously — prompt returns immediately, summary after LLM
// Present the built prompt to the user
$.getJSON(promptUrl)
.done(function (data) {
if (data.prompt && $summaryRow.find('.ai-inline-summary-content:not(.loaded)').length) {
@@ -469,34 +469,31 @@ $(function () {
);
}
});
$.getJSON(url)
.done(function (data) {
// And at the same time start the summary: POST kicks off generation (CSRF token
// added by csrf.js), then llm-summary.js polls until it lands. Polling stops by
// itself if the user toggles the row shut.
window.llmSummary.fetch({
url: url,
uuid: uuid,
isCancelled: function () {
return !$.contains(document.documentElement, $summaryRow[0]);
},
onDone: function (summary) {
var $content = $summaryRow.find('.ai-inline-summary-content');
var historyUrl = $btn.attr('href');
if (data.summary) {
$content.addClass('loaded');
$content.find('.ai-inline-text').html(formatSummary(data.summary));
$content.find('.ai-inline-prompt').remove();
} else if (data.error) {
$summaryRow.find('td').html(
'<span class="ai-inline-error">' + $('<span>').text(data.error).html() + '</span>'
);
}
$content.addClass('loaded');
$content.find('.ai-inline-text').html(formatSummary(summary));
$content.find('.ai-inline-prompt').remove();
$content.find('.ai-inline-body').append(
'<a href="' + historyUrl + '" class="ai-inline-history-link">' +
'<a href="' + $btn.attr('href') + '" class="ai-inline-history-link">' +
$('<span>').text(msgHistory).html() + '</a>'
);
})
.fail(function (xhr) {
var msg = (xhr.responseJSON && xhr.responseJSON.error)
? xhr.responseJSON.error
: 'AI summary request failed (HTTP ' + xhr.status + ').';
},
onError: function (msg) {
$summaryRow.find('td').html(
'<span class="ai-inline-error">' + $('<span>').text(msg).html() + '</span>'
);
});
}
});
});
}
});
File diff suppressed because one or more lines are too long
+570 -3
View File
@@ -1,3 +1,48 @@
@use "settings" as *;
// Paint the page's own background (page colour + the fixed 130deg gradient at
// 0.91, pre-composited with color-mix) so a sticky bar stays opaque to content
// scrolling under it while looking identical to the page behind it.
//
// styles.scss paints that backdrop as a real fixed layer: body::after is a
// `position: fixed` box at the viewport's top-left corner, viewport-wide and
// `height: 100vh` tall. Anything that has to disappear into it must show the
// slice of that box it covers. background-attachment: fixed asks the engine for
// exactly that, and iOS Safari does not honour it - it anchors the layer to the
// document instead, so the bar sweeps the whole ramp (cyan to purple over a
// long page) while the gutters beside it hold still. Chromium does honour it,
// which is why only a device caught this.
//
// So derive the alignment from geometry and ask for no feature at all. Each bar
// keeps a constant viewport offset - .header at the top, #diff-header directly
// below it - and both are flush with the viewport's right edge at every width:
// the action rail insets them from the *left* on a wide screen and widens again
// on hover, but nothing moves their right edge. Anchor a viewport-sized copy of
// the gradient to that edge and push it up by the bar's own offset, and the bar
// paints body::after's slice by construction, on any engine.
//
// $y-offset: the bar's distance from the top of the viewport.
@mixin page-surface-gradient($y-offset: 0px) {
background: var(--color-background-gradient-second); // fallback, no color-mix
// Longhands from here down, and no second `background:` shorthand after them -
// one would silently reset size/position/origin back to their initial values.
background-image: linear-gradient(130deg,
color-mix(in srgb, var(--color-background-gradient-first) 91%, var(--color-background-page)),
color-mix(in srgb, var(--color-background-gradient-second) 91%, var(--color-background-page)) 41.07%,
color-mix(in srgb, var(--color-background-gradient-third) 91%, var(--color-background-page)) 84.05%);
// 100vh to match body::after's own height: 100vh - same unit, so the two
// resolve to the same number on iOS, where it means the large (toolbar
// collapsed) viewport and therefore holds still as the toolbar moves.
// 100vw for the width: the bars are as wide as the viewport only on a narrow
// screen, so 100% would compress the ramp into the rail's leftovers on a
// desktop one.
background-size: 100vw 100vh;
background-repeat: no-repeat;
// border-box, or the bar's own padding would offset the copy by that much.
background-origin: border-box;
background-position: right 0 top calc(-1 * #{$y-offset});
}
#diff-form {
background: rgba(0, 0, 0, .05);
@@ -40,8 +85,451 @@
}
body.difference-page {
section.content {
padding-top: 40px;
// Keep the top menu (watch URL, EDIT, theme/GitHub icons) on screen while the
// diff scrolls, stacked above #diff-header. Scoped to the diff page on
// purpose - upstream's @todo in parts/_top_menu.scss wants this globally, but
// that is a separate change.
//
// Sticky makes .header a stacking context, so its mobile drawer / overlay
// (z-index 10000 / 9999) are now capped by this value. 30 clears #diff-header
// below and still leaves toasts (10000, fixed) painting over the bar.
.header {
position: sticky;
top: 0;
z-index: 30;
@include page-surface-gradient(0px);
}
// The top line now carries the watch's name, so give it back the room the two
// stacked left insets were taking. Its text starts $common-gap twice in: the
// menu row's own padding, which every item on the row needs to clear the edge,
// plus the link's own left margin, which on the row's first item only repeats
// that padding. Drop the repeat and keep the row's padding: 8.8px back, and
// the line still sits on the same left edge as the diff below it rather than
// hard against the viewport. The right side of both is untouched - the link's
// right margin still separates it from the menu icons.
.current-diff-url {
margin-left: 0;
}
// Wherever the hamburger is (max-width: $desktop-wide-breakpoint, set in
// parts/_hamburger_menu.scss), the menu row is at its tightest and every item
// on it competes with the watch name for the same line. The heart is the one
// item on that row that is neither navigation nor status, so it is what gives
// way - on this page only, and only under that width. Desktop diff pages and
// every other page keep it.
//
// CSS rather than a {% if %} in base.html: the heart's own reason to exist is
// that it is always there, and a template branch would take it off the page
// entirely rather than fold it away with the rest of the row.
@media only screen and (max-width: $desktop-wide-breakpoint) {
#heart-us {
display: none;
}
}
// .app-main's 0.55rem gap sits between the two sticky bars, so #diff-header
// starts 8.8px below the top menu and then slides up to meet it over the
// first 8.8px of scroll - a visible twitch right as scrolling begins. The gap
// can't be preserved once stuck (it is outside both bars, so the diff would
// scroll through it), so close it here and let them sit flush at every scroll
// position. Only the .header/section.content pair is affected; .app-main has
// no other children.
.app-main {
gap: 0;
}
.tab-pane-inner {
scroll-margin-top: calc(var(--app-header-height, 0px) + var(--diff-header-height, 0px) + 1rem);
}
// The 2px activity strip is fixed on body, so it lives in the root stacking
// context and would paint over the mobile drawer now that the drawer is capped
// inside .header's context above. Drop it below .header rather than raising
// .header past the action rail (60), which would hide the rail's hover flyout.
// Still above #diff-header (20) and the minimap (10); on this page it now also
// passes under the rail's flyout, which is 2px of imperceptible overlap.
#pure-menu-horizontal-spinner {
z-index: 29;
}
// The thin blue line under the active tab. Nothing at that boundary has a
// border - the active tab and #diff-ui are both var(--color-background) - so
// it can only be the page gradient showing through a device-pixel row where
// neither white paints. Whether such a row exists comes down to one number,
// the bar's bottom edge minus the card's top edge, and measured across 14
// viewports at three device pixel ratios that number is 0.813, 0.203 or
// exactly 0.000 and never a whole pixel. At 0.000 the outcome belongs
// entirely to the engine's sub-pixel rounding - which is why a DPR 3 phone
// can show a line a DPR 1 desktop does not, and why 390x844 on the Screenshot
// tab, the exact case reported, measures 0.000.
//
// So take the margin out of the engine's hands instead of hoping two white
// edges land on the same device row. A pixel of overlap covers the gradient
// on any grid; measured the same way afterwards, the worst case over those 42
// readings becomes exactly 1.000px.
//
// The other candidate was a white skirt under the active tab
// (box-shadow: 0 1px 0) which would have touched nothing else, and it does not
// work: the shadow hangs below the bar's last child and #diff-header's
// overflow: auto - the thing enforcing its height cap - clips it. Measured
// with the card held 3px away so the skirt and the card are separable, it
// painted 0 device rows of the 3 it needs at 844x390 and 926x428, and 0 to 2
// in the sticky modes. Recorded here so it is not proposed again.
//
// What the overlap costs, stated rather than discovered later: where the bar
// is sticky it is invisible, and a device-pixel diff of the seam band at
// 390x844 and DPR 3 says so exactly - zero differing pixels, because the bar
// is opaque and paints over the card. Below the short-viewport breakpoint the
// bar has no background of its own, so the card's white also reaches the
// bottom pixel of the *inactive* tabs and the 5px gaps between them: 4374
// device pixels at 844x390, gradient turning white. The tab row ends up
// sitting flush on the card rather than floating a pixel above it, which is
// how tabs normally meet their panel and is the same continuity the active
// tab was asked for.
//
// Scoped to the diff page deliberately. diff.css is NOT diff-page-only - it
// is also served to the preview page (blueprint/ui/preview.py) and the
// extract page (processors/extract.py), and both have a #diff-ui of their own
// with no bar above it to absorb the pixel. Only difference.py sets the
// difference-page class.
#diff-ui {
margin-top: -1px;
}
}
#diff-header {
position: sticky;
// Sits directly under the now-sticky top menu. The 0 fallback (JS off) just
// slides this under the bar - the same degradation the minimap already takes.
top: var(--app-header-height, 0px);
z-index: 20;
// section.content insets its children by $common-gap so they clear the left
// rail, while the top menu above stays flush against it. A bar that paints
// the page's own backdrop therefore leaves an 8.8px strip of real page
// background down each side, and on a device those strips read as mismatched
// borders. Cancel the inset rather than try to blend across it: with no
// gutter there is nothing left to keep aligned, and the flush right edge is
// what page-surface-gradient anchors its copy of the page gradient to. The
// controls do not move - the bar grows outwards by exactly the margin it
// takes back.
//
// The width has to come from align-self, not width: 100%. .content-main
// centres its children, so an item is fit-content unless it stretches, and a
// percentage width resolves against the parent without the negative margins -
// it would leave the bar short by 17.6px.
align-self: stretch;
margin-left: -$common-gap;
margin-right: -$common-gap;
box-sizing: border-box;
// Center children at their intrinsic width, the same way section.content does
// for the non-sticky siblings it still lays out (align-items: center).
// Two rows: the toolbar and the tabs.
display: flex;
flex-direction: column;
align-items: center;
gap: 0.25rem;
// No bottom padding: the tab row is the bar's last child, and the active tab
// and #diff-ui below it are both var(--color-background). 4px of gradient
// between them reads as a seam across an otherwise continuous white surface -
// the same tabs on the extract page, which are not in a bar, measure 0. The
// top padding stays; nothing sits above the toolbar.
padding: 0.25rem 1rem 0;
// Keep the diff readable even with wrapped controls, zoom, or a short viewport.
// The budget is for the whole sticky stack, so the top menu above comes out of
// it - otherwise the two bars together take 64% of a 390px-tall landscape phone.
//
// svh, not dvh. dvh tracks the dynamic viewport, which on iOS Safari grows and
// shrinks as the browser's own toolbar collapses and expands while you scroll:
// the cap would then breathe mid-scroll, changing the header's height and
// shifting everything below it, and each transition also fires resize and
// re-runs the sticky measurement behind it. svh is defined as the
// toolbar-expanded minimum and holds still for the life of the page, so the
// budget is a constant. It is the smaller of the two, which is the safe side to
// be wrong on for a cap. The plain vh line stays as the fallback for engines
// without the new units.
max-height: calc(50vh - var(--app-header-height, 0px));
max-height: calc(50svh - var(--app-header-height, 0px));
overflow: auto;
// Same offset as `top` above: the bar's constant distance from the top of the
// viewport. With JS off the var is absent and the copy is misaligned by the
// top menu's height - the same tier of degradation `top` itself already takes.
@include page-surface-gradient(var(--app-header-height, 0px));
> * {
// A centered flex item is sized to fit-content, which has a min-content
// floor (the version <select> is as wide as its longest option). Without
// this cap a narrow viewport would centre an over-wide child and spill it
// out of both sides of the header.
max-width: 100%;
}
// base.html's wrap detector decides body.wrapped-tabs by flipping the tab <ul>
// to flex-wrap and seeing whether any tab lands on a second row - so what it
// measures against is the width of .tabs. _tabs.scss says in upstream's own
// words that .tabs has to take the full width of the centred column "so the
// wrap-detector JS has the real available space to measure against", and the
// two properties that would do it are commented out directly underneath. As
// shipped, .tabs is fit-content inside any centred column, so the detector
// measures a container that moves with the layout state it is judging: the
// active tab's label is bold, and selecting the widest one ("Current
// screenshot") grows .tabs by ~9px. In the band where that crosses the
// available width the verdict flips per tab - measured here at 350, 355 and
// 358px, where Text and Extract Data report one row and Current screenshot
// reports three. iOS puts the same band at 390px because its fonts are wider,
// which is how a phone showed one row in a screenshot and a vertical stack in
// a video of the same page.
//
// Give the detector a fixed target on this page only. Un-commenting the lines
// in _tabs.scss would fix it everywhere and change every page upstream owns;
// this PR does not have the standing to do that.
.tabs {
align-self: stretch;
// Never give up height to keep the controls whole - see #settings below.
flex: none;
// .tabs no longer shrink-wraps, so the grid would sit left-aligned in it.
ul {
justify-content: center;
// Stretching the container fixes one side of the detector's comparison;
// this fixes the other. `.active a { font-weight: bold }` in _tabs.scss
// makes the row's width depend on which label is currently bold: at 355px
// the Text and Extract states measure 332.9 and 330.5 against a 339.0
// container and stay on one row, while Current screenshot - the widest
// label - measures over it and wraps to three. Neutralising just that rule
// collapses all three states to 330.5 and the flipping stops, which is what
// identifies the weight as the variable rather than the container.
//
// So drop the bold here rather than reserve room for it. Reserving the bold
// width on every tab (a hidden bold copy behind each label) also stabilises
// it, but it makes the row ~14px wider in every state and that pushes 360px
// - a common Android width that fits on one row today - into the stacked
// layout. The active tab does not need the third signal: _tabs.scss already
// gives it its own background-color and its own text colour.
li.active a,
li :target a {
font-weight: inherit;
}
}
}
// The bar's height budget (max-height above) is smaller than its contents on a
// landscape phone: iOS gives the two datetime <select>s enough intrinsic width
// that From and To each take their own row, and From + To + Filters + tabs is
// then taller than 50svh minus the top menu. With overflow: auto on the bar
// itself the row that leaves the visible box is the last one - the tabs, which
// are the page's navigation and have no scrollbar to advertise themselves. Measured in Chromium with no iOS emulation at 736x414, 667x375,
// 780x360 and 844x330: the tab row sits outside #diff-header's own rect.
//
// So spend the budget on the row that can afford it. #settings is the only
// child allowed to shrink, and it is what scrolls; the tab row never does and
// is therefore always inside the bar at any viewport height. The shrink needs
// min-height: 0 - a flex item's automatic minimum size is its content, which
// is what would otherwise push the tabs back out.
#settings {
min-width: 0;
display: flex;
justify-content: center;
flex: 0 1 auto;
min-height: 0;
overflow-y: auto;
// Same reason the Filters popover carries it: a nested scroller inside a
// sticky bar that chains its overscroll to the diff behind is worse than no
// scroller at all on a touch screen.
overscroll-behavior: contain;
}
// One toolbar row: From/To chips and their selects, the version arrows, and
// the Filters toggle.
#diff-form {
display: flex;
flex-wrap: wrap;
align-items: center;
justify-content: center;
gap: 0.25rem 0.9rem;
padding: 0.2rem 0.7rem;
margin-bottom: 0;
fieldset {
margin: 0;
padding: 0;
border: 0;
}
// The From/To labels shrink to chips rather than hiding: display:none takes
// the <label> out of the accessibility tree and the selects lose their
// accessible name entirely, and two identically formatted datetime selects
// side by side need the red/green cue sighted users already know. The
// shipped width: 4rem is what makes them full-size plates. "From" is wider
// than "To", which left-ragged the selects once they stack, so the floor is
// wide enough for the longer word - it costs width only, never height.
.from-to-label {
width: auto;
min-width: 2.4rem;
text-align: center;
font-size: 0.75rem;
padding: 0.1rem 0.35rem;
border-radius: 3px;
line-height: 1.5;
}
}
#keyboard-nav {
display: flex;
align-items: center;
gap: 0.3rem;
// The arrows speak for themselves at this size; the links keep the full
// wording as aria-label and title (see diff.html).
strong,
.keyboard-nav-label {
display: none;
}
}
// Seven set-once options are a popover, not a permanent row of the bar. The
// swap happens only once diff-overview.js has wired the toggle up, so with
// scripting off the fieldset stays inline and submits as it does today.
#diff-filters-toggle {
display: none;
font-size: 0.85rem;
padding: 0.3em 0.8em;
}
&.diff-filters-js {
#diff-filters-toggle {
display: inline-block;
}
#diff-style {
display: none;
}
}
&.diff-filters-js.diff-filters-open {
#diff-style {
// fixed, not absolute: #diff-header carries overflow: auto to enforce the
// 50svh cap above, which clips an absolutely positioned descendant at the
// bar's bottom edge. The bar is sticky with no transform/filter/contain,
// so it is not a containing block for fixed and the panel escapes the
// clip. diff-overview.js parks it under the toggle.
display: block;
position: fixed;
z-index: 25;
min-width: 14rem;
// diff-overview.js caps max-height to the room below the button that the
// viewport actually has - on a landscape phone that is less than the
// seven rows need. The rows past the cap have to stay reachable, and the
// scroll must not chain out to the diff behind once they run out.
overflow-y: auto;
overscroll-behavior: contain;
padding: 0.5rem 0.75rem;
border-radius: 8px;
box-shadow: 0 4px 14px rgba(0, 0, 0, .28);
text-align: left;
// The bar repaints the page's own backdrop; so must anything that has to
// be opaque over the scrolling diff. This one is not pinned to an edge of
// the viewport - diff-overview.js decides where it goes - so the same
// file overwrites background-position from the coordinates it just set.
// The declaration here is what a placement-less first paint falls back to.
@include page-surface-gradient;
}
// Out-specifies #diff-form #diff-style > span's inline-block above.
#diff-form #diff-style > span {
display: block;
padding: 0.2em 0;
}
}
.diff-fieldset {
min-width: 0;
display: flex;
justify-content: center;
flex-wrap: wrap;
gap: 0.5rem;
> span {
max-width: 100%;
min-width: 0;
gap: 0.25rem;
// Size to content so From/To stay side-by-side on desktop (as they were
// before the header existed); shrink and wrap only when the row can't fit.
flex: 0 1 auto;
display: flex;
align-items: center;
// A <select> keeps the intrinsic min-content width of its longest option
// (min-width: 0 does not lower it), so label + select cannot fit a phone
// viewport side by side. Let the label drop above it instead of forcing
// the whole header wider than the screen.
flex-wrap: wrap;
}
select {
min-width: 0;
max-width: 100%;
flex: 1;
}
label {
flex-shrink: 0;
}
}
}
// The one narrow-viewport breakpoint for the whole bar - title, fit and
// gutters all hang off it rather than letting magic numbers multiply.
//
// Every selector below carries an ID on purpose. The rules it has to beat are
// written above as `#diff-header .diff-fieldset > span` and friends, and a
// single ID outranks any number of classes, so a class-only override loses
// silently and the chain goes on shrink-wrapping. Please don't tidy the IDs
// away.
@media (max-width: 700px) {
// Once the From/To chip takes inline width the <select> clips its own
// displayed value, and two snapshots from the same day differ only by the
// meridiem - so they read identically while closed. Shrinking the font alone
// cannot fix that: #settings, #diff-form, .diff-fieldset and the span are all
// sized to max-content, which *is* the select's text, so the container
// shrinks by exactly as much as the text does and the headroom stays pinned
// at zero at every font size. Pin the chain to the row first; only then does
// the smaller font buy anything.
#diff-header > #settings {
width: 100%;
}
#diff-header #diff-form {
width: 100%;
box-sizing: border-box;
// 16px of bar gutter plus 11.2px of form gutter a side is 54px of a 390px
// row spent on nothing.
padding-left: 0.2rem;
padding-right: 0.2rem;
}
#diff-header {
padding-left: 0.5rem;
padding-right: 0.5rem;
}
#diff-header #diff-form > .diff-fieldset {
flex: 1 1 100%;
}
#diff-header .diff-fieldset > span {
flex: 1 1 100%;
}
// With the chain pinned the select holds its width, so this is what actually
// buys the room. The worst realistic English value - a two-digit day, the
// longest month and a two-digit hour - still clips at 0.8rem.
#diff-header .diff-fieldset select {
font-size: 0.75rem;
padding-left: 0.3em;
padding-right: 0.3em;
}
}
@@ -170,7 +658,7 @@ body.difference-page {
border-radius: 3px;
overflow-x: hidden;
position: sticky;
top: 0;
top: calc(var(--app-header-height, 0px) + var(--diff-header-height, 0px));
z-index: 10;
padding-top: 1rem;
padding-bottom: 1rem;
@@ -246,3 +734,82 @@ body.difference-page {
}
}
}
// Placed last on purpose. Several of these rules out-specify nothing - they
// tie with the rule they are overriding, and a tie is decided by source
// order. The minimap's sticky offset is written further down this file as
// `#diff-ui #cell-diff-jump-visualiser`, so a block that sits above it loses
// silently. Keep this one at the bottom.
//
// A phone held sideways is the one place the sticky design cannot pay for
// itself. 50svh of a 390px-tall viewport is 195px, the top menu takes 55 of it,
// and what is left has to hold From, To, Filters and the tab row - three of
// those on their own lines, because iOS gives the two datetime selects enough
// width that From and To cannot share one. #settings shrinking and scrolling
// (see above) keeps the tabs inside the bar, but a bar you have to scroll
// *inside* to reach half its controls is still a bar in the way, and no amount
// of trimming decoration finds 195px.
//
// So stop compensating for the budget here and give up the budget instead.
// Below 500px of viewport height both bars are ordinary static blocks: no
// height cap, no inner scroller, title and every control present at once, and
// the page scrolls as one document. The trade is the sticky guarantee itself -
// the controls and the tab row scroll away with the diff, so switching tabs
// means scrolling back up - which is the right side to be wrong on when the
// alternative is controls you cannot see. Portrait is untouched.
//
// Keyed on viewport height rather than orientation: a short desktop window has
// the identical problem for the identical reason.
@media (max-height: 500px) {
body.difference-page .header {
// Not a new state: parts/_top_menu.scss ships .header with its sticky
// commented out, so this is the position every other page's top menu is
// already in. z-index goes inert along with it, which hands the mobile
// drawer's 10000 back to the root stacking context.
position: static;
// Nothing scrolls underneath a static bar, so it neither needs its
// pre-composited copy of the page gradient nor may keep it: the copy is
// aligned by pushing it up by the bar's own *constant* viewport offset, and
// a bar that scrolls has no constant offset. It would slide out of phase
// with body::after's fixed layer and read as a moving seam. The shorthand,
// not background-image - it resets size, origin and position too.
background: none;
}
#diff-header {
position: static;
// The cap and the scroller are what the sticky design pays for its
// guarantee that the diff keeps half the viewport. There is no longer a
// guarantee to pay for.
max-height: none;
overflow: visible;
background: none;
#settings {
// Same: it only shrank and scrolled to keep the tab row inside the cap.
min-height: auto;
overflow-y: visible;
}
}
// .tab-pane-inner's scroll-margin-top is deliberately NOT overridden here,
// and dropping it to a bare 1rem is a trap worth naming. It reads as an
// allowance for pinned chrome, so with nothing pinned it looks like dead
// weight - but upstream's tabs.js rewrites an empty hash to the first tab's
// (tabs.js:15) and the browser then jumps to that pane on load, before any
// of this is a choice. At 1rem that jump lands with the whole static bar
// already scrolled off the top: measured -175.8px at 844x390, a landscape
// page that opens with no controls on it at all. The existing formula is the
// right number in both modes for two different reasons - in the sticky one it
// clears chrome that is pinned over the pane, in the static one it keeps
// chrome that sits above the pane on screen. Leave it alone.
// The minimap is sticky inside the diff card, offset to sit directly below
// the sticky stack. With that stack gone its offset would pin it a top menu
// plus a whole toolbar down the viewport - past halfway on a 390px-tall one,
// a strip floating in mid-screen beneath nothing. It is still worth pinning;
// pin it to the viewport, which is now free.
#diff-ui #cell-diff-jump-visualiser {
top: 0;
}
}
@@ -12,6 +12,11 @@ $action-sidebar-content-slot: 1100px;
// Desktop left rail — fixed positioning frame, full-height, but only the
// inner item-list block captures hover. Empty area is pointer-events:none so
// the top header stays clickable behind it.
.action-sidebar-inner {
background-color: var(--color-background-page); // fallback under the gradient
background-attachment: fixed, fixed;
}
.action-sidebar {
display: flex;
flex-direction: column;
@@ -25,28 +30,34 @@ $action-sidebar-content-slot: 1100px;
position: sticky;
top: 0;
height: 100vh;
background: rgba(255, 255, 255, 0.05);
z-index: 60;
pointer-events: none; // empty space doesn't intercept clicks/hovers
@media only screen and (max-width: $desktop-wide-breakpoint) {
// Hidden on mobile: the mobile drawer (hamburger) carries these items
display: none;
}
}
body.actionside-bar-on {
.action-sidebar {
/* width: $action-sidebar-width-expanded; // reserve enough to host the expanded inner block*/
// MINIMAL MODE: the rail's footprint in the flex row is frozen at the
// collapsed width (+ the inner block's horizontal padding, which is
// content-box). The inner block is taken out of flow (absolute, below) so
// its hover expansion rolls out ON TOP of the page content instead of
// widening this flex item and reflowing `.app-main`.
body.actionsidebar-minimal & {
flex: 0 0 auto;
width: calc(#{$action-sidebar-width-collapsed} + #{$common-gap * 2});
overflow: visible; // let the expanded inner block escape the frame
}
}
// The actual interactive item block — wraps only the items, not the whole height.
//
// Two body-level modes drive the rail's display state:
// body.actionside-bar-on → always expanded (icons + labels visible all the time)
// body.actionsidebar-minimal → icon-only collapsed rail; expands on hover/focus
// `actionside-bar-on` is the default applied in templates/base.html. Both the
// hover-to-expand width animation and the label fade-in are gated on the
// Body-level classes drive the rail's display state (emitted by
// flask_app.get_sidebar_mode_class() from the `sidebar_mode` setting):
// body.actionside-bar-on → always expanded (icons + labels visible all the time)
// body.actionsidebar-minimal → icon-only collapsed rail; expands on hover/focus
// body.actionsidebar-no-expand → alongside the above: stays collapsed, no hover-out
// Both the hover-to-expand width animation and the label fade-in are gated on the
// minimal class so that the always-expanded mode (or no mode) doesn't trigger
// a needless layout jump when the user mouses across the rail.
.action-sidebar-inner {
@@ -64,12 +75,35 @@ body.actionside-bar-on {
padding-left: $common-gap;
padding-right: $common-gap;
// Hover-to-expand only fires in minimal mode.
body.actionsidebar-minimal &:hover,
body.actionsidebar-minimal &:focus-within {
// MINIMAL MODE: lift the block out of the flex flow so growing it can't
// resize the rail (and therefore can't shift the content column). The rail
// is `position: sticky`, i.e. a positioned element, so top/left/bottom here
// anchor to it; top+bottom keep the column full-height, which the footer
// list's `margin-top: auto` still needs.
//
// Once it floats above the page it needs its own backdrop, or the content
// it rolls over would read straight through the rail's translucent 5% white.
// It can't just be a flat colour: the page background is a fixed, viewport-
// sized gradient (`body:after`), so the rail repaints that same gradient with
// `background-attachment: fixed` — which makes the viewport the positioning
// area — and lays the rail's own 5% white on top. Result: opaque, but
// pixel-identical to the translucent rail it replaces, at rest and rolled out.
body.actionsidebar-minimal & {
position: absolute;
top: 0;
left: 0;
bottom: 0;
z-index: 1;
}
// Hover-to-expand only fires in minimal mode, and not for 'minimal' (no-expand).
body.actionsidebar-minimal:not(.actionsidebar-no-expand) &:hover,
body.actionsidebar-minimal:not(.actionsidebar-no-expand) &:focus-within {
width: $action-sidebar-width-expanded;
// Enter is smoother.
transition: width 0.22s cubic-bezier(0.2, 0.7, 0.2, 1);
// Lift it off the page while it's rolled out over the content.
box-shadow: var(--color-sidebar-shadow);
}
// Always-expanded mode: no hover gate; the rail just sits open.
@@ -85,7 +119,7 @@ ul.action-sidebar-list {
margin: 0;
display: flex;
flex-direction: column;
gap: 0;
gap: 0.8rem;
}
// Footer list (status counters, live spark, version) sits at the bottom of the
@@ -228,7 +262,7 @@ ul.action-sidebar-list {
letter-spacing: 0.06em;
font-weight: 700;
pointer-events: none;
opacity: 0;
opacity: 0.9;
transition: opacity 0.05s ease-out;
// Unread-count variant: a small accent bubble pinned to the right of the row
@@ -247,11 +281,17 @@ ul.action-sidebar-list {
}
}
// Count badges (unread, queue) are always visible in minimal mode so users
// can see at a glance how many items need attention without expanding the rail.
body.actionsidebar-minimal .action-badge--count {
opacity: 1;
}
// When the inner block expands in MINIMAL mode, reveal every label/badge
// together with a smooth enter. Always-expanded mode handles labels via the
// `body.actionside-bar-on` block below — no hover gate needed there.
body.actionsidebar-minimal .action-sidebar-inner:hover,
body.actionsidebar-minimal .action-sidebar-inner:focus-within {
body.actionsidebar-minimal:not(.actionsidebar-no-expand) .action-sidebar-inner:hover,
body.actionsidebar-minimal:not(.actionsidebar-no-expand) .action-sidebar-inner:focus-within {
.action-sidebar-item {
.action-label {
opacity: 1;
@@ -285,10 +325,10 @@ body.actionside-bar-on {
.action-icon {
flex: 0 0 auto;
width: 24px;
height: 24px;
max-width: 20px;
height: 22px;
stroke: currentColor;
stroke-width: 1.9;
stroke-width: 1.4;
fill: none;
stroke-linecap: round;
stroke-linejoin: round;
@@ -431,33 +471,53 @@ body.actionside-bar-on {
padding-left: $common-gap;
}
.actionsidebar-expanded {
#logo-short {
display: none;
}
}
.actionsidebar-minimal {
#checking-now-stats-sidebar {
display: none;
}
#cdio-logo {
#logo-expanded {
display: none;
}
}
&.action-side-bar-expanded {
#checking-now-stats-sidebar{
&.actionsidebar-expanded {
#checking-now-stats-sidebar {
display: block;
}
#cdio-logo {
#logo-expanded {
display: inline-block;
}
}
}
}
.action-side-bar-expanded {
#cdio-logo {
#logo-short {
&:not(.actionsidebar-expanded) {
.action-label {
display: none;
}
.action-badge--count {
position: relative;
left: -8px;
top: -8px;
}
.action-sidebar-item.is-disabled {
.action-badge {
display: none;
}
}
}
}
@@ -158,63 +158,37 @@ body {
display: flex;
flex-direction: column;
gap: 1.1rem;
color: #fff;
@media (max-width: 900px) {
flex: 1 1 auto;
}
.add-watch-option-group {
label {
display: inline-block;
span.label {
padding-bottom: 0.5rem;
display: inline-block !important;
}
label {
margin: 0 !important;; /* override */
}
li {
list-style: none;
font-size: .9rem;
display: grid;
grid-template-columns: auto 1fr;
align-items: center;
gap: .5rem;
input[type="radio"] {
flex: 0 0 auto;
}
> label {
font-weight: normal;
}
}
// Browser picker: fetcher descriptions are long (they carry the driver URL), so this is
// a radio list whose labels wrap inside the narrow pane instead of a <select> that would
// overflow it. The system-default entry is listed even when it can't render a preview.
#quick-watch-fetch-backend {
ul {
margin: 0.35rem 0 0 0;
padding: 0;
list-style: none;
}
li {
display: flex;
align-items: flex-start;
gap: 0.5em;
padding: 0.15rem 0;
input[type="radio"] {
flex: 0 0 auto;
margin-top: 0.2em;
}
label {
// Wrap rather than push the pane wider
display: block;
min-width: 0;
overflow-wrap: anywhere;
font-size: 0.85rem;
line-height: 1.35;
}
}
li.unusable {
opacity: 0.55;
cursor: not-allowed;
label {
cursor: not-allowed;
}
}
.pure-form-message-inline {
display: block;
margin-top: 0.35rem;
font-size: 0.8rem;
opacity: 0.8;
}
li + li { /* gap between */
margin-top: .75rem;
}
#by-element-toggle-group {
@@ -79,7 +79,6 @@
width: 100%;
overflow-y: scroll;
position: relative;
height: 80vh;
> img {
position: absolute;
@@ -3,16 +3,20 @@
// Clean outline button — surface background, hairline border, muted label that
// darkens on hover. The app's neutral/secondary action button (toolbars, list
// actions), a calmer alternative to the solid pure-button. Pairs with .seg.
.cdio-btn {
display: inline-flex;
align-items: center;
vertical-align: middle;
box-sizing: border-box;
gap: 6px;
height: 32px;
padding: 0 12px;
border-radius: var(--common-round-border);
border: 1px solid var(--color-border-table-cell);
background: var(--color-background);
color: var(--color-text-input-description);
color: var(--color-text);
// <button> does not inherit font-family from the page, and this class is used on both
// anchors and submit buttons (toolbar actions are buttons so they can POST) - without this
// the buttons render in the browser's UI font while the links next to them don't.
@@ -27,7 +31,7 @@
&:hover {
border-color: var(--color-border-input);
color: var(--color-text);
/* color: var(--color-text);*/
}
&:focus-visible {
@@ -90,4 +94,31 @@
color: #d68a00;
&:hover { color: #d68a00; border-color: #d68a00; }
}
&.pure-button-primary {
color: #fff;
}
}
// State-mutating controls have to POST, and only a <button> can submit - these strip
// the browser's button chrome so they render exactly like the <a> they replaced.
.bare-btn {
background: none;
border: 0;
padding: 0;
margin: 0;
font: inherit;
// Matches `a { color: var(--color-link) }` - these replace anchors, and the row
// icons stroke with currentColor, so `inherit` would pick up .watch-controls red.
color: var(--color-link);
cursor: pointer;
&:focus-visible {
outline: 2px solid var(--color-link);
outline-offset: 2px;
}
&--link {
text-decoration: underline;
}
}
@@ -94,8 +94,31 @@
transition: right 0.3s cubic-bezier(0.68, -0.55, 0.265, 1.55);
overflow-y: auto;
padding-top: 60px;
.action-sidebar-divider {
display: none;
}
.is-disabled {
display: none;
}
.action-sidebar-item.active {
background-color: inherit;
font-weight: bold;
span {
font-weight: bold;
}
}
#action-sidebar-logo {
padding: 0;
}
.action-badge--count {
position: initial !important;
top: 0;
left: 0;
}
.action-label {
display: block !important;
}
#cdio-logo {
color: var(--color-text);
@@ -113,16 +136,20 @@
&.active {
right: 0;
}
#live-queue-line, #extra-item, #hosting-offer {
display: none;
}
.mobile-menu-items {
list-style: none;
padding: 1rem 0;
margin: 0;
button {
color: var(--color-text);
}
li {
border-bottom: 1px solid var(--color-border-table-cell);
>* {
>* {
display: block;
padding: 1rem 1.5rem;
color: var(--color-text);
@@ -134,6 +161,14 @@
background: var(--color-background-menu-link-hover);
}
}
// Buttons shrink-wrap and centre their label; anchors don't. No global
// border-box reset here, so this must not be folded into the rule above.
>form>button {
box-sizing: border-box;
width: 100%;
text-align: left;
}
&#menu-pause, &#menu-mute {
display: none;
}
@@ -1,7 +1,7 @@
#language-selector-flag {
display: inline-block;
width: 1.2em;
height: 1.2em;
width: 22px;
height: 22px;
vertical-align: middle;
border-radius: 50%;
overflow: hidden;
@@ -18,10 +18,20 @@
flex-direction: column;
gap: 0.5rem;
padding: 0.5rem 0;
// The csrf mini-form around each option is layout-transparent, so the buttons
// stay the flex items.
> form {
display: contents;
}
}
.language-option {
display: flex;
background: none;
font: inherit;
text-align: left;
cursor: pointer;
align-items: center;
gap: 1rem;
padding: 0.25rem;
@@ -10,16 +10,19 @@
top: 50px;
right: -350px;
background-color: var(--color-table-stripe);
border: 1px solid #aaa;
z-index: 2;
border-top-left-radius: var(--common-round-border);
// Slides in by moving its own width to the left (right: -350px -> flush with
// the right edge). The transition MUST name `transform` - it used to say
// `width`, which never changes here, so the slide snapped with no animation.
transform: translateX(0);
transition: transform .5s ease;
will-change: transform;
transition: transform 0.22s cubic-bezier(0.2, 0.7, 0.2, 1), box-shadow 0.22s ease;
&.visible {
transform: translateX(-100%);
box-shadow: var(--color-sidebar-shadow);
}
.content {
@@ -15,6 +15,13 @@
.pure-menu-item {
height: initial;
// Mini POST forms (pause/mute/log out need a csrf_token) are layout-transparent,
// so the button inside sits where the plain <a> used to.
> form {
display: contents;
}
svg {
height: 1.2rem;
}

Some files were not shown because too many files have changed in this diff Show More