* LLM restock: parse string prices with price_parser instead of stripping separators
If the model returns the price as page text, e.g. "1.299,95 €", stripping
everything but digits and dots gave 1.29995. price_parser (already a
dependency) handles the locale formats the prompt lists.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* LLM restock: use Restock.parse_currency() like the other price extractors
Same parser c8afea1b (#4494) moved the structured-data extractors to, so
there's one place price text becomes a number. parse_currency() now lists
every caller, and the call sites point back to it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* LLM restock: expose prompt/cache/parsing helpers so other LLM backends can reuse them
Split the hook body into build_restock_messages(), restock_cache_key/get/put()
and parse_restock_reply(), so a plugin talking to a provider without litellm
only has to supply the HTTP call. accumulate_global_tokens() takes an optional
cost_usd, since the default estimate imports litellm's pricing (~200MB that's
never released). The built-in plugin behaves exactly as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Test: give the no-litellm subprocess check a PYTHONPATH
CI runs pytest from inside the package dir without installing it, so the
child interpreter couldn't import changedetectionio.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Restock plugins: pass html_to_text() output to the restock hook, let third-party backends go first
get_itemprop_availability_override now also gets `text` (the fetched page through
html_to_text()), only computed when an implementation declares it. The hook is
firstresult and the built-in litellm plugin is trylast, so a plugin that talks to an
LLM with plain requests answers first and the built-in one is never billed as well.
Also fixes the built-in plugin never receiving llm_intent: pluggy silently doesn't
pass hookimpl arguments that have a default value, and it declared llm_intent=None.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Watch.history ran os.path.realpath() twice and os.path.exists() once for every
line in history.txt (~10 syscalls per snapshot), on every watch at startup and
on every history read after that.
basename() already reduces each entry to a plain filename, so the only way out
of the watch data dir is a symlink. One os.scandir() of the data dir now
answers "does it exist", and only symlinks pay for a realpath() containment
check. scandir() never lists '.' or '..', so those are still rejected.
~13x faster on 200 watches x 500 snapshots (1.18s -> 0.09s, warm cache),
identical output. Adds tests for symlinks escaping/inside the data dir,
dangling symlinks, and missing snapshots/blank lines.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The extractors stripped everything but [\d.] before converting, so the
decimal comma was thrown away before Restock.parse_currency(), which
already understands it, ever saw the value. Route the values through
parse_currency() instead, and teach it that a lone "1,299" is a
thousands separator so that case does not regress.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Every watch whose xPath filter used contains() started reporting "Warning, no filters were
found" on pages whose HTML plainly contained the target. 18 unrelated watches broke in the same
hour, browser and plain-requests fetchers alike, and the saved snapshot was perfect every time.
elementpath implements the XPath string functions on top of locale.strxfrm:
def contains(self, a, b): return self.strxfrm(b) in self.strxfrm(a)
Under LC_COLLATE=C, strxfrm() is the identity function and that is an ordinary substring test.
Under a real locale it returns a binary collation key, and a substring of a collation key is not
the collation key of the substring - so contains(), starts-with(), ends-with() and
substring-before/after() return false for EVERY input. Reduced to one line, no document needed:
LC_COLLATE=C contains("xx month xx", "month") -> True
LC_COLLATE=en_US.UTF-8 contains("xx month xx", "month") -> False
Name tests, axes and '=' are untouched, which is exactly why it read as "did the page change
layout?" - //div kept working while //div[contains(.,"month")] returned nothing.
No code change caused this. Generating the image's locales (#4429) made ENV LC_ALL=en_US.UTF-8
satisfiable for the first time; flask_app's setlocale(LC_ALL, ...) had been raising locale.Error
and leaving us in C, and once it succeeded it took LC_COLLATE with it. That is why the bug
survived a bisect to 0.60.2 and why reinstalling the exact pip set from a working install did not
shift it - it could only be found by diffing the two containers. Same image base, same Python
3.11.16, lxml 6.1.3, libxml2 2.14.6, elementpath 5.1.1, same HTML:
good-old 0.60.4 setlocale FAILED: unsupported locale setting 67 matches
bad-new 0.60.5 setlocale en_US.UTF-8 0 matches
Fixed in both places, because either alone leaves a hole:
- flask_app sets LC_CTYPE/LC_NUMERIC/LC_MONETARY/LC_TIME individually instead of LC_ALL. This
block exists to make prices render correctly and it still does - 1234567 is still "1,234,567"
- it just no longer touches collation.
- html_tools.xpath_filter() pins the Unicode codepoint collation per evaluation, so a filter
means the same thing whatever an operator puts in LANG/LC_ALL, and does not depend on a
distant module's locale bookkeeping. forms.py's XPath validation pins it too, so validation
cannot accept an expression that then behaves differently at check time.
Per XPath 3.1 the default collation is codepoint and must not consult LC_COLLATE, so the
underlying behaviour is arguably an elementpath bug; the pin above holds regardless.
Verified end to end inside the failing container: LC_COLLATE=C, LC_NUMERIC=en_US.UTF-8,
thousands separators intact, and the reported filter back from 0 to 1190797 chars of output.
Tested: new unit test covers contains/starts-with/ends-with under a UTF-8 collation and was
checked to fail without the fix; 325 unit tests pass.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Compare against a resolved data_dir in the history path-traversal test
Watch.history resolves entries with os.path.realpath, so
test_normal_snapshot_entry_is_accepted compared a resolved path against an
unresolved data_dir. On macOS the datastore lives under /tmp, which is a
symlink to /private/tmp, so the assertion fails for a path that is in fact
inside the directory. The guard is correct; the test was not.
Resolve both sides, matching what the production code does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Add a test that actually exercises the history containment check
Disabling the containment check in Watch.history left every test in
TestHistoryPathTraversal passing. os.path.basename() reduces both traversal
fixtures ('/etc/passwd', '../../etc/passwd') to 'passwd', so neither reaches
the check — they stop at the os.path.exists() test below it.
A bare '..' survives basename() and resolves to the parent of data_dir, which
exists, so the containment check is what rejects it. With that check disabled
this new test is the only one in the class that fails.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: GG5533 <285285461+GG5533@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(conditions): support zero values in condition filtering and json logic conversion
- Fix filter_complete_rules dropping rules where value is 0/0.0 due to 0 == False in Python
- Fix convert_to_jsonlogic raising EmptyConditionRuleRowNotUsable on value=0 due to truthiness check
- Fix str != 'None' type comparison typo in convert_to_jsonlogic
- Add comprehensive unit tests covering zero value condition filtering, conversion, and execution
* chore: re-trigger CI checks
---------
Co-authored-by: Andrew Peabody <apeabody@users.noreply.github.com>
- Define _NO_TEMPERATURE_MODEL_KEYWORDS constant and omit temperature=0 on initial request
- Omit thinkingConfig in _thinking_extra_body for flash-lite models
- Unconditionally strip rejected sampling params and extra_body thinkingConfig on BadRequestError (handling Gemini 400 INVALID_ARGUMENT without named details)
- Add comprehensive unit tests in test_llm_client_gemini.py
* Scheduler+API Bug - if an invalid timezone was set (through edit of watch or API) it could have crashed the scheduler, Added `timezone` to the official API docs
* adding missing files
* Restock detection - fix inverted condition that skipped the OpenGraph availability fallback
get_itemprop_availability() only dug through OpenGraph properties when
price was missing OR when availability was ALREADY found - so on pages
that expose price via JSON-LD but stock state only via the OpenGraph
commerce tags (<meta property="product:availability">), the availability
(and currency) was silently dropped.
The inner loop already guards each field with 'if not value.get(...)',
so the outer gate now matches that intent: dig OpenGraph when any of
price/availability/currency is still missing.
Co-Authored-By: Claude <noreply@anthropic.com>
* Normalise availability after the OpenGraph fallback, not before
An availability found via OpenGraph was left raw, so the Facebook
commerce spelling 'in stock' never matched the 'instock' test and the
product read as out of stock.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Non blocking improvements
* Test fix
* Background thread re-queue
* Nonblockimg improvements, run tasks in background, add warning about CPU cores
* Misc fixes
