* Set Cache-Control: no-store on dynamic responses by default
Pages with per-session content (settings, CSRF-token-bearing forms,
watch data) had no Cache-Control header, so a misconfigured CDN or
reverse proxy sitting in front of the app could cache and replay them
across requests/sessions — most commonly surfacing as "CSRF tokens do
not match" after the edge served a stale cached page. Routes that
already set their own Cache-Control (static assets, screenshots,
favicons) are left untouched.
* Adding tests and moving function
---------
Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>
* fix(http): send a single Date header on werkzeug built-in server
Static resources served via werkzeug send_from_directory/send_file get a
Date header injected into the WSGI response by make_conditional()
(werkzeug/wrappers/response.py:752-757). When the app runs on Werkzeug's
built-in server -- the default path started through
socketio.run(..., allow_unsafe_werkzeug=True) in changedetectionio/__init__.py:694
and used by the docker entrypoint -- BaseHTTPRequestHandler.send_response()
(werkzeug/serving.py:271) emits its own Date header line as well, so the
wire response carries two Date headers. RFC 9110 forbids this and nginx
rejects the response with "upstream sent duplicate header line" (issue
#4299, see also #4101).
Fix: a global after_request hook pops the application-side Date copy so
only the server's single header reaches the wire. Verified safe on
gunicorn too, which also emits its own Date header.
Test: new tests/test_duplicate_date_header.py hits the live_server over
real HTTP with http.client (the Flask test client talks to the WSGI app
directly and never sees the server-added header) and asserts the Date
header appears exactly once, on the exact static resources named in the
issue. Fails on unfixed code with two identical Date lines; passes with
the fix.
Fixes#4299
* Apply suggestion from @dgtlmoon
* Tidy the #4299 Date header fix and its test
flask_app.py: the applied suggestion landed with a 3-space indent and
trailing whitespace - the latter was the only W291 in the file, which
.ruff.toml selects.
test_duplicate_date_header.py:
- drop the 10s socket wait loop, pytest-flask's live_server already
blocks until the port accepts connections
- drop the unused `app` fixture argument (live_server depends on it)
- stop hardcoding jquery-3.6.0.min.js: asserting 200 on a vendored
filename turns a jQuery bump into a failure in a file about HTTP
headers. Any send_from_directory() response exercises the same path,
so styles.css alone is enough.
Still red before the fix (two identical Date lines) and green after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: dgtlmoon <leigh@morresi.net>
Co-authored-by: dgtlmoon <dgtlmoon@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every watch whose xPath filter used contains() started reporting "Warning, no filters were
found" on pages whose HTML plainly contained the target. 18 unrelated watches broke in the same
hour, browser and plain-requests fetchers alike, and the saved snapshot was perfect every time.
elementpath implements the XPath string functions on top of locale.strxfrm:
def contains(self, a, b): return self.strxfrm(b) in self.strxfrm(a)
Under LC_COLLATE=C, strxfrm() is the identity function and that is an ordinary substring test.
Under a real locale it returns a binary collation key, and a substring of a collation key is not
the collation key of the substring - so contains(), starts-with(), ends-with() and
substring-before/after() return false for EVERY input. Reduced to one line, no document needed:
LC_COLLATE=C contains("xx month xx", "month") -> True
LC_COLLATE=en_US.UTF-8 contains("xx month xx", "month") -> False
Name tests, axes and '=' are untouched, which is exactly why it read as "did the page change
layout?" - //div kept working while //div[contains(.,"month")] returned nothing.
No code change caused this. Generating the image's locales (#4429) made ENV LC_ALL=en_US.UTF-8
satisfiable for the first time; flask_app's setlocale(LC_ALL, ...) had been raising locale.Error
and leaving us in C, and once it succeeded it took LC_COLLATE with it. That is why the bug
survived a bisect to 0.60.2 and why reinstalling the exact pip set from a working install did not
shift it - it could only be found by diffing the two containers. Same image base, same Python
3.11.16, lxml 6.1.3, libxml2 2.14.6, elementpath 5.1.1, same HTML:
good-old 0.60.4 setlocale FAILED: unsupported locale setting 67 matches
bad-new 0.60.5 setlocale en_US.UTF-8 0 matches
Fixed in both places, because either alone leaves a hole:
- flask_app sets LC_CTYPE/LC_NUMERIC/LC_MONETARY/LC_TIME individually instead of LC_ALL. This
block exists to make prices render correctly and it still does - 1234567 is still "1,234,567"
- it just no longer touches collation.
- html_tools.xpath_filter() pins the Unicode codepoint collation per evaluation, so a filter
means the same thing whatever an operator puts in LANG/LC_ALL, and does not depend on a
distant module's locale bookkeeping. forms.py's XPath validation pins it too, so validation
cannot accept an expression that then behaves differently at check time.
Per XPath 3.1 the default collation is codepoint and must not consult LC_COLLATE, so the
underlying behaviour is arguably an elementpath bug; the pin above holds regardless.
Verified end to end inside the failing container: LC_COLLATE=C, LC_NUMERIC=en_US.UTF-8,
thousands separators intact, and the reported filter back from 0 to 1190797 chars of output.
Tested: new unit test covers contains/starts-with/ends-with under a UTF-8 collation and was
checked to fail without the fix; 325 unit tests pass.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Scheduler+API Bug - if an invalid timezone was set (through edit of watch or API) it could have crashed the scheduler, Added `timezone` to the official API docs
* adding missing files
* No need to add imported items to the check queue, the scheduler will do this #3762
* Tests - Faster recheck/reschedule loop under pytest environment
* More wait time under test
* Bunch up some tests a little
* fix typo
* woops
* If they want to queue one thats already running, thats up to them.
* WIP
* Fixing queue limit size
* Increase max queue size and many CPU performance fixes
Multi-language / Translations Support (#3696)
- Complete internationalization system implemented
- Support for 7 languages: Czech (cs), German (de), French (fr), Italian (it), Korean (ko), Chinese Simplified (zh), Chinese Traditional (zh_TW)
- Language selector with localized flags and theming
- Flash message translations
- Multiple translation fixes and improvements across all languages
- Language setting preserved across redirects
Pluggable Content Fetchers (#3653)
- New architecture for extensible content fetcher system
- Allows custom fetcher implementations
Image / Screenshot Comparison Processor (#3680)
- New processor for visual change detection (disabled for this release)
- Supporting CSS/JS infrastructure added
UI Improvements
Design & Layout
- Auto-generated tag color schemes
- Simplified login form styling
- Removed hard-coded CSS, moved to SCSS variables
- Tag UI cleanup and improvements
- Automatic tab wrapper functionality
- Menu refactoring for better organization
- Cleanup of offset settings
- Hide sticky tabs on narrow viewports
- Improved responsive layout (#3702)
User Experience
- Modal alerts/confirmations on delete/clear operations (#3693, #3598, #3382)
- Auto-add https:// to URLs in quickwatch form if not present
- Better redirect handling on login (#3699)
- 'Recheck all' now returns to correct group/tag (#3673)
- Language set redirect keeps hash fragment
- More friendly human-readable text throughout UI
Performance & Reliability
Scheduler & Processing
- Soft delays instead of blocking time.sleep() calls (#3710)
- More resilient handling of same UUID being processed (#3700)
- Better Puppeteer timeout handling
- Improved Puppeteer shutdown/cleanup (#3692)
- Requests cleanup now properly async
History & Rendering
- Faster server-side "difference" rendering on History page (#3442)
- Show ignored/triggered rows in history
- API: Retry watch data if watch dict changed (more reliable)
API Improvements
- Watch get endpoint: retry mechanism for changed watch data
- WatchHistoryDiff API endpoint includes extra format args (#3703)
Testing Improvements
- Replace time.sleep with wait_for_notification_endpoint_output (#3716)
- Test for mode switching (#3701)
- Test for #3720 added (#3725)
- Extract-text difference test fixes
- Improved dev workflow
Bug Fixes
- Notification error text output (#3672, #3669, #3280)
- HTML validation fixes (#3704)
- Template discovery path fixes
- Notification debug log now uses system locale for dates/times
- Puppeteer spelling mistake in log output
- Recalculation on anchor change
- Queue bubble update disabled temporarily
Dependency Updates
- beautifulsoup4 updated (#3724)
- psutil 7.1.0 → 7.2.1 (#3723)
- python-engineio ~=4.12.3 → ~=4.13.0 (#3707)
- python-socketio ~=5.14.3 → ~=5.16.0 (#3706)
- flask-socketio ~=5.5.1 → ~=5.6.0 (#3691)
- brotli ~=1.1 → ~=1.2 (#3687)
- lxml updated (#3590)
- pytest ~=7.2 → ~=9.0 (#3676)
- jsonschema ~=4.0 → ~=4.25 (#3618)
- pluggy ~=1.5 → ~=1.6 (#3616)
- cryptography 44.0.1 → 46.0.3 (security) (#3589)
Documentation
- README updated with viewport size setup information
Development Infrastructure
- Dev container only built on dev branch
- Improved dev workflow tooling
