Every watch whose xPath filter used contains() started reporting "Warning, no filters were
found" on pages whose HTML plainly contained the target. 18 unrelated watches broke in the same
hour, browser and plain-requests fetchers alike, and the saved snapshot was perfect every time.
elementpath implements the XPath string functions on top of locale.strxfrm:
def contains(self, a, b): return self.strxfrm(b) in self.strxfrm(a)
Under LC_COLLATE=C, strxfrm() is the identity function and that is an ordinary substring test.
Under a real locale it returns a binary collation key, and a substring of a collation key is not
the collation key of the substring - so contains(), starts-with(), ends-with() and
substring-before/after() return false for EVERY input. Reduced to one line, no document needed:
LC_COLLATE=C contains("xx month xx", "month") -> True
LC_COLLATE=en_US.UTF-8 contains("xx month xx", "month") -> False
Name tests, axes and '=' are untouched, which is exactly why it read as "did the page change
layout?" - //div kept working while //div[contains(.,"month")] returned nothing.
No code change caused this. Generating the image's locales (#4429) made ENV LC_ALL=en_US.UTF-8
satisfiable for the first time; flask_app's setlocale(LC_ALL, ...) had been raising locale.Error
and leaving us in C, and once it succeeded it took LC_COLLATE with it. That is why the bug
survived a bisect to 0.60.2 and why reinstalling the exact pip set from a working install did not
shift it - it could only be found by diffing the two containers. Same image base, same Python
3.11.16, lxml 6.1.3, libxml2 2.14.6, elementpath 5.1.1, same HTML:
good-old 0.60.4 setlocale FAILED: unsupported locale setting 67 matches
bad-new 0.60.5 setlocale en_US.UTF-8 0 matches
Fixed in both places, because either alone leaves a hole:
- flask_app sets LC_CTYPE/LC_NUMERIC/LC_MONETARY/LC_TIME individually instead of LC_ALL. This
block exists to make prices render correctly and it still does - 1234567 is still "1,234,567"
- it just no longer touches collation.
- html_tools.xpath_filter() pins the Unicode codepoint collation per evaluation, so a filter
means the same thing whatever an operator puts in LANG/LC_ALL, and does not depend on a
distant module's locale bookkeeping. forms.py's XPath validation pins it too, so validation
cannot accept an expression that then behaves differently at check time.
Per XPath 3.1 the default collation is codepoint and must not consult LC_COLLATE, so the
underlying behaviour is arguably an elementpath bug; the pin above holds regardless.
Verified end to end inside the failing container: LC_COLLATE=C, LC_NUMERIC=en_US.UTF-8,
thousands separators intact, and the reported filter back from 0 to 1190797 chars of output.
Tested: new unit test covers contains/starts-with/ends-with under a UTF-8 collation and was
checked to fail without the fix; 325 unit tests pass.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Compare against a resolved data_dir in the history path-traversal test
Watch.history resolves entries with os.path.realpath, so
test_normal_snapshot_entry_is_accepted compared a resolved path against an
unresolved data_dir. On macOS the datastore lives under /tmp, which is a
symlink to /private/tmp, so the assertion fails for a path that is in fact
inside the directory. The guard is correct; the test was not.
Resolve both sides, matching what the production code does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Add a test that actually exercises the history containment check
Disabling the containment check in Watch.history left every test in
TestHistoryPathTraversal passing. os.path.basename() reduces both traversal
fixtures ('/etc/passwd', '../../etc/passwd') to 'passwd', so neither reaches
the check — they stop at the os.path.exists() test below it.
A bare '..' survives basename() and resolves to the parent of data_dir, which
exists, so the containment check is what rejects it. With that check disabled
this new test is the only one in the class that fails.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: GG5533 <285285461+GG5533@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(conditions): support zero values in condition filtering and json logic conversion
- Fix filter_complete_rules dropping rules where value is 0/0.0 due to 0 == False in Python
- Fix convert_to_jsonlogic raising EmptyConditionRuleRowNotUsable on value=0 due to truthiness check
- Fix str != 'None' type comparison typo in convert_to_jsonlogic
- Add comprehensive unit tests covering zero value condition filtering, conversion, and execution
* chore: re-trigger CI checks
---------
Co-authored-by: Andrew Peabody <apeabody@users.noreply.github.com>
- Define _NO_TEMPERATURE_MODEL_KEYWORDS constant and omit temperature=0 on initial request
- Omit thinkingConfig in _thinking_extra_body for flash-lite models
- Unconditionally strip rejected sampling params and extra_body thinkingConfig on BadRequestError (handling Gemini 400 INVALID_ARGUMENT without named details)
- Add comprehensive unit tests in test_llm_client_gemini.py
* Scheduler+API Bug - if an invalid timezone was set (through edit of watch or API) it could have crashed the scheduler, Added `timezone` to the official API docs
* adding missing files
* Restock detection - fix inverted condition that skipped the OpenGraph availability fallback
get_itemprop_availability() only dug through OpenGraph properties when
price was missing OR when availability was ALREADY found - so on pages
that expose price via JSON-LD but stock state only via the OpenGraph
commerce tags (<meta property="product:availability">), the availability
(and currency) was silently dropped.
The inner loop already guards each field with 'if not value.get(...)',
so the outer gate now matches that intent: dig OpenGraph when any of
price/availability/currency is still missing.
Co-Authored-By: Claude <noreply@anthropic.com>
* Normalise availability after the OpenGraph fallback, not before
An availability found via OpenGraph was left raw, so the Facebook
commerce spelling 'in stock' never matched the 'instock' test and the
product read as out of stock.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Non blocking improvements
* Test fix
* Background thread re-queue
* Nonblockimg improvements, run tasks in background, add warning about CPU cores
* Misc fixes
Multi-language / Translations Support (#3696)
- Complete internationalization system implemented
- Support for 7 languages: Czech (cs), German (de), French (fr), Italian (it), Korean (ko), Chinese Simplified (zh), Chinese Traditional (zh_TW)
- Language selector with localized flags and theming
- Flash message translations
- Multiple translation fixes and improvements across all languages
- Language setting preserved across redirects
Pluggable Content Fetchers (#3653)
- New architecture for extensible content fetcher system
- Allows custom fetcher implementations
Image / Screenshot Comparison Processor (#3680)
- New processor for visual change detection (disabled for this release)
- Supporting CSS/JS infrastructure added
UI Improvements
Design & Layout
- Auto-generated tag color schemes
- Simplified login form styling
- Removed hard-coded CSS, moved to SCSS variables
- Tag UI cleanup and improvements
- Automatic tab wrapper functionality
- Menu refactoring for better organization
- Cleanup of offset settings
- Hide sticky tabs on narrow viewports
- Improved responsive layout (#3702)
User Experience
- Modal alerts/confirmations on delete/clear operations (#3693, #3598, #3382)
- Auto-add https:// to URLs in quickwatch form if not present
- Better redirect handling on login (#3699)
- 'Recheck all' now returns to correct group/tag (#3673)
- Language set redirect keeps hash fragment
- More friendly human-readable text throughout UI
Performance & Reliability
Scheduler & Processing
- Soft delays instead of blocking time.sleep() calls (#3710)
- More resilient handling of same UUID being processed (#3700)
- Better Puppeteer timeout handling
- Improved Puppeteer shutdown/cleanup (#3692)
- Requests cleanup now properly async
History & Rendering
- Faster server-side "difference" rendering on History page (#3442)
- Show ignored/triggered rows in history
- API: Retry watch data if watch dict changed (more reliable)
API Improvements
- Watch get endpoint: retry mechanism for changed watch data
- WatchHistoryDiff API endpoint includes extra format args (#3703)
Testing Improvements
- Replace time.sleep with wait_for_notification_endpoint_output (#3716)
- Test for mode switching (#3701)
- Test for #3720 added (#3725)
- Extract-text difference test fixes
- Improved dev workflow
Bug Fixes
- Notification error text output (#3672, #3669, #3280)
- HTML validation fixes (#3704)
- Template discovery path fixes
- Notification debug log now uses system locale for dates/times
- Puppeteer spelling mistake in log output
- Recalculation on anchor change
- Queue bubble update disabled temporarily
Dependency Updates
- beautifulsoup4 updated (#3724)
- psutil 7.1.0 → 7.2.1 (#3723)
- python-engineio ~=4.12.3 → ~=4.13.0 (#3707)
- python-socketio ~=5.14.3 → ~=5.16.0 (#3706)
- flask-socketio ~=5.5.1 → ~=5.6.0 (#3691)
- brotli ~=1.1 → ~=1.2 (#3687)
- lxml updated (#3590)
- pytest ~=7.2 → ~=9.0 (#3676)
- jsonschema ~=4.0 → ~=4.25 (#3618)
- pluggy ~=1.5 → ~=1.6 (#3616)
- cryptography 44.0.1 → 46.0.3 (security) (#3589)
Documentation
- README updated with viewport size setup information
Development Infrastructure
- Dev container only built on dev branch
- Improved dev workflow tooling
