95 Commits

Include renames

Author	SHA1	Message	Date
dgtlmoonandClaude Opus 5	ba2280ba81	External CDP Browser engine - replaces the extra_browser_* hackery ... An "extra browser" was a name + a ws(s):// endpoint in settings.requests.extra_browsers, selected by a watch as the magic string 'extra_browser_<name>'. That string resolved to html_webdriver plus a custom connection URL, which meant the protocol the endpoint was spoken to came from env vars rather than from the entry: CDP over a WebSocket with PLAYWRIGHT_DRIVER_URL set, CDP via pyppeteer with FAST_PUPPETEER_CHROME_FETCHER, and the W3C WebDriver protocol over HTTP on a Selenium-only install - where a wss:// URL cannot work at all. The form only ever accepted ws:// / wss://, so the feature was silently broken on exactly the installs that could not honour it. So it becomes an engine, html_external_cdp, which pins the protocol: a subclass of the Playwright fetcher that takes its endpoint from the watch's browser config (FetcherConfig.connection_url) instead of the environment. It is base-only (ready_to_use=False) because an endpoint is required, so each endpoint is one browser config ("variation") on the Browsers page - which is what the old settings list was. update_36 migrates each extra_browsers row to such a variation, keyed by the SAME 'extra_browser_<name>' string watches already hold, so no watch, group override, API value or global default needs rewriting; the legacy selector simply becomes a real browser-config id. A row whose endpoint the model rejects is logged and skipped rather than taking the update chain, and with it startup, down. Knock-on cleanups, all of which delete a special case rather than add one: - The proxy opt-out for custom endpoints is now Fetcher.ignores_proxy_setting, asked of the engine, instead of a string-prefix test in call_browser(). - A live browser-steps / visual-selector session asks the engine where to connect (Fetcher.browser_steps_connection_url, overridden by html_external_cdp) and refuses an engine whose supports_browser_steps is False, instead of reading the env var itself and silently stepping a browser the watch does not check with. That refusal is real: on a Selenium install html_webdriver cannot drive a live session. - is_valid_browser_selector() answers "may a watch store this in fetch_backend?" in one place; the API (create/update/import), the quick-add form validator and the bulk "set browser" operation each had their own copy, which is how they came to disagree about whether a browser-config id was acceptable. - api-spec.yaml's fetch_backend pattern enumerated extra_browser_* while rejecting browser-config ids and every engine newer than html_webdriver. Valid values are per-install, so the schema now bounds the string and the handlers do the real check. - html_external_cdp registers unconditionally (unlike html_playwright_builtin): migrated configs name it, so it must resolve even without the playwright library, or those watches would quietly fetch with the plain HTTP client. The library is imported lazily inside run(), and an unavailable engine now warns instead of falling back silently. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-17 19:39:16 +02:00
dgtlmoon	00a00b664f	Merge branch 'master' into browser-ui	2026-09-17 17:26:16 +02:00
Jeff Hedlund	2f60135cd0	feature: Implemented optional override link (#4290) ... * Implemented optional link to override feature * Added translation strings (untranslated)	2026-09-09 16:06:24 +02:00
dgtlmoon	7f3645f4a8	API - DELETE for Watch history #4397 (#4403)	2026-09-09 15:46:00 +02:00
dgtlmoon	68a445df1b	Env var - PAGE_WATCH_LIMIT enhancements (#4359)	2026-09-02 13:37:55 +02:00
dgtlmoon	dce6f23cf9	Scheduler+API Bug - if an invalid timezone was set (through edit of watch or API) it could have crashed the scheduler, Added timezone to the official API docs (#4350) ... * Scheduler+API Bug - if an invalid timezone was set (through edit of watch or API) it could have crashed the scheduler, Added `timezone` to the official API docs * adding missing files	2026-09-01 17:00:03 +02:00
dgtlmoon	288926b44e	GHSA-4j9m-cxq4-9c36 - Improved validation Stored CSS Injection via tag_colour Field (#4317)	2026-08-20 17:59:02 +02:00
dgtlmoon	ee35b3da1a	UI - Another attempt at configurable browsers	2026-07-16 21:30:52 +02:00
dgtlmoon	71d332d5a0	UI full refactor / upgrade (#4183)	2026-07-16 15:08:29 +02:00
dgtlmoon	624dee60d5	API Security - Watch GET history snapshot - Should return text/plain mimetype so it cant be accidently executed in the browser (#4158)	2026-05-19 11:36:42 +02:00
dgtlmoon	82795fe883	API - Better support for watch API private/internal vars	2026-05-15 12:41:30 +02:00
dgtlmoon	aadf8df7ae	API - Add restock config to API /v1/watch/ json output #4099 (#4103)	2026-04-29 12:10:47 +02:00
dgtlmoon	44ac324a41	API - watch.link was accidently a tuple, enforcing string (#4104)	2026-04-29 12:10:16 +02:00
dgtlmoon	b32617d700	API - Invert changes_only flag for include_equal parameter, add test, fixes changesOnly option for history diff API call (#3976)	2026-03-12 16:15:37 +01:00
dgtlmoon	02c03fc32b	API - Create (POST) tag/group through API do not save processor_config_restock_diff values #3966 (#3968)	2026-03-10 11:19:59 +01:00
dgtlmoon	5669509255	API - Processors configuration is now part of the API (#3902)	2026-02-25 11:30:39 +01:00
dgtlmoon	a385c89abf	CVE-2026-27645 - Reflected XSS in RSS Single Watch request	2026-02-23 21:55:59 +01:00
dgtlmoon	fafea1b5c6	Updates/migration - Re-run tag update, re-save to cleanup changedetection.json, code refactor (#3898)	2026-02-18 14:05:23 +01:00
dgtlmoon	383c3b427f	API - Adding automated test for API with NGINX sub-path, Skip validation errors about server path (allows use on sub-paths/reverse proxy etc) (#3886)	2026-02-16 20:32:35 +01:00
dgtlmoon	b6d3d63372	Avoid reprocessing if the page was the same (#3867)	2026-02-14 21:24:28 +01:00
dgtlmoon	bcd32852ca	API - Remove flask_expects_json validation, this is covered entirely by OpenAPI, update OpenAPI spec. (#3871)	2026-02-13 16:30:59 +01:00
dgtlmoon	5726c5a0ac	API - Import use background task to import large lists (#3858)	2026-02-11 08:15:58 +01:00
dgtlmoon	c66a29b011	API - Import - Ability to set any watch value as HTTP URL Query value, for example ?processor=restock_diff&time_between_check={'hours':24} Re #3845 (#3857)	2026-02-11 07:26:48 +01:00
dgtlmoon	a1a2e5c5bf	API - Include missing tags in fetching watch information. #3854 (#3856)	2026-02-11 06:45:19 +01:00
dgtlmoon	892b645147	Refactor for Tags storage (#3848)	2026-02-07 13:13:02 +01:00
dgtlmoon	c577bd700c	Refactor watch saving backend, closes #3846 (#3847)	2026-02-07 03:41:35 +01:00
dgtlmoon	2349344d9e	Improved watch global settings handling (#3839)	2026-02-05 16:40:00 +01:00
dgtlmoon	83d7ce0fcf	Processor plugin improvements - Now supports creating your own processor (for example, monitor DNS changes) (#3739)	2026-02-04 14:23:08 +01:00
dgtlmoon	30dc4ac23b	Refactor of queue systenm and improve tests, improves multiple workers (#3826)	2026-02-02 22:28:27 +01:00
dgtlmoon	2658f81f02	Ability to limit total number of watches with env var PAGE_WATCH_LIMIT (#3828)	2026-02-02 11:21:44 +01:00
dgtlmoon	0b9cfcdf09	API - Notification URLs werent always being validated (#3812)	2026-01-28 17:30:58 +01:00
dgtlmoon	7e886e0c56	Memory - Favicon reader had a memory leak, Restart fetch workers between jobs, misc tweaks (#3787)	2026-01-20 12:49:53 +01:00
dgtlmoon	151e603af7	API - Validation improvements (#3782)	2026-01-19 18:16:25 +01:00
dgtlmoon	9b96689072	API & UI - Recheck all - Dont requeue existing queued or processing watches. (#3770)	2026-01-17 18:20:22 +01:00
dgtlmoon	5e5674f48d	Non blocking improvements (#3767) ... * Non blocking improvements * Test fix * Background thread re-queue * Nonblockimg improvements, run tasks in background, add warning about CPU cores * Misc fixes	2026-01-17 17:25:18 +01:00
dgtlmoon	9438d38dc6	Queues and Scheduler - No need to add imported items to the check queue, the scheduler will do this #3762 (#3763), CPU usage improvements. ... * No need to add imported items to the check queue, the scheduler will do this #3762 * Tests - Faster recheck/reschedule loop under pytest environment * More wait time under test * Bunch up some tests a little * fix typo * woops * If they want to queue one thats already running, thats up to them. * WIP * Fixing queue limit size * Increase max queue size and many CPU performance fixes	2026-01-17 13:43:24 +01:00
dgtlmoon	3d14df6a11	Development branch merge into release/master ... Multi-language / Translations Support (#3696) - Complete internationalization system implemented - Support for 7 languages: Czech (cs), German (de), French (fr), Italian (it), Korean (ko), Chinese Simplified (zh), Chinese Traditional (zh_TW) - Language selector with localized flags and theming - Flash message translations - Multiple translation fixes and improvements across all languages - Language setting preserved across redirects Pluggable Content Fetchers (#3653) - New architecture for extensible content fetcher system - Allows custom fetcher implementations Image / Screenshot Comparison Processor (#3680) - New processor for visual change detection (disabled for this release) - Supporting CSS/JS infrastructure added UI Improvements Design & Layout - Auto-generated tag color schemes - Simplified login form styling - Removed hard-coded CSS, moved to SCSS variables - Tag UI cleanup and improvements - Automatic tab wrapper functionality - Menu refactoring for better organization - Cleanup of offset settings - Hide sticky tabs on narrow viewports - Improved responsive layout (#3702) User Experience - Modal alerts/confirmations on delete/clear operations (#3693, #3598, #3382) - Auto-add https:// to URLs in quickwatch form if not present - Better redirect handling on login (#3699) - 'Recheck all' now returns to correct group/tag (#3673) - Language set redirect keeps hash fragment - More friendly human-readable text throughout UI Performance & Reliability Scheduler & Processing - Soft delays instead of blocking time.sleep() calls (#3710) - More resilient handling of same UUID being processed (#3700) - Better Puppeteer timeout handling - Improved Puppeteer shutdown/cleanup (#3692) - Requests cleanup now properly async History & Rendering - Faster server-side "difference" rendering on History page (#3442) - Show ignored/triggered rows in history - API: Retry watch data if watch dict changed (more reliable) API Improvements - Watch get endpoint: retry mechanism for changed watch data - WatchHistoryDiff API endpoint includes extra format args (#3703) Testing Improvements - Replace time.sleep with wait_for_notification_endpoint_output (#3716) - Test for mode switching (#3701) - Test for #3720 added (#3725) - Extract-text difference test fixes - Improved dev workflow Bug Fixes - Notification error text output (#3672, #3669, #3280) - HTML validation fixes (#3704) - Template discovery path fixes - Notification debug log now uses system locale for dates/times - Puppeteer spelling mistake in log output - Recalculation on anchor change - Queue bubble update disabled temporarily Dependency Updates - beautifulsoup4 updated (#3724) - psutil 7.1.0 → 7.2.1 (#3723) - python-engineio ~=4.12.3 → ~=4.13.0 (#3707) - python-socketio ~=5.14.3 → ~=5.16.0 (#3706) - flask-socketio ~=5.5.1 → ~=5.6.0 (#3691) - brotli ~=1.1 → ~=1.2 (#3687) - lxml updated (#3590) - pytest ~=7.2 → ~=9.0 (#3676) - jsonschema ~=4.0 → ~=4.25 (#3618) - pluggy ~=1.5 → ~=1.6 (#3616) - cryptography 44.0.1 → 46.0.3 (security) (#3589) Documentation - README updated with viewport size setup information Development Infrastructure - Dev container only built on dev branch - Improved dev workflow tooling	2026-01-12 17:50:53 +01:00
dgtlmoon	e6553065fd	API - Watch get, retry watch data if watch dict changed (more reliable)	2026-01-05 10:31:17 +01:00
dgtlmoon	b9305faf21	Forcing UTF-8 when reading JSON DB (Fixes data not loaded for some platforms #3622 #3611 #3628), Always create new versions of the backup DB if one exists for that step when running updates, Adding extra sanity checks on DB load	2025-11-12 12:58:59 +01:00
dgtlmoon	caffd804fe	Revert "Windows - JSON DB fixes - Forcing utf-8 for json DB read/writes should solve windows saving/loading problems. (#3615 #3611)" ... This reverts commit e2b407c6f3.	2025-11-10 13:31:51 +01:00
dgtlmoon	e2b407c6f3	Windows - JSON DB fixes - Forcing utf-8 for json DB read/writes should solve windows saving/loading problems. (#3615 #3611)	2025-11-10 11:59:17 +01:00
dgtlmoon	a8d5ea067d	Watch history - Don't rescan whole history.txt when looking up a timestamp <->filepath (#3602)	2025-11-05 18:50:27 +01:00
dgtlmoon	d18029ffe4	API - Support optional processor on Watch create to set the restock_diff or text_json_diff mode on watch create.	2025-10-30 17:55:39 +01:00
dgtlmoon	ab0b85d088	Unify safe URL checking to the one function, strengthen tests and filters (#3564)	2025-10-28 13:24:37 +01:00
dgtlmoon	2116b2cb93	CVE-2025-62780 - Stored XSS in Watch update via API	2025-10-28 10:09:30 +01:00
dgtlmoon	8aa012ba8e	API - Import - Automatically assume text/plain content type on Import (makes it easier for changedetection to add new URLs) #3547 #3542	2025-10-25 18:47:09 +02:00
dgtlmoon	ec692ed727	pip build - Improving fix for #3509, Adding automated test for #3509	2025-10-17 12:48:50 +02:00
dgtlmoon	cde42c8a49	Reducing memory usage (#3476)	2025-10-09 18:31:19 +02:00
dgtlmoon	6e1f5a8503	API - Adding page title link, bumping docs (#3431)	2025-09-17 11:51:46 +02:00
dgtlmoon	b74b76c9f9	"Time between check" field is now validated correctly (requires atleast one of the weeks days hours minutes seconds to be set)	2025-09-16 19:09:45 +02:00