173 Commits

Include renames

Author	SHA1	Message	Date
dgtlmoonandClaude Opus 5	6318fc70f4	Browser fetchers - Report the real status code when Chrome aborts a bodiless error response (#4420) ... * Browser fetchers - Report the real status code when Chrome aborts a bodiless error response Chrome 153+ refuses to commit a navigation when a 4xx/5xx arrives with a zero-length body: page.goto() raises net::ERR_HTTP_RESPONSE_CODE_FAILURE instead of returning the response. The response is received fine, we just never get it as a return value, so the raw net:: string landed in last_error instead of "Error - 404". Verified against two browser images, same HTTP server: Chrome 153 empty-body 404 -> raises ERR_HTTP_RESPONSE_CODE_FAILURE Chrome 153 404 with body -> status=404 Chromium 119 empty-body 404 -> status=404 Chromium 119 404 with body -> status=404 The fix keeps the main-frame response from the 'response' event and hands that back when goto raises, so .status / .all_headers() and the existing non-200 branch (which also captures the screenshot) work unchanged. The latest matching response wins, so a redirect chain still reports its final hop. Any other error re-raises as before, and if no response was captured we re-raise too - the status is never invented, which keeps older browsers on exactly their old path. Two independent navigation sites needed it: - browser_steps.py action_goto_url - covers the playwright fetcher, the live Browser Steps UI, the Goto URL / Goto site steps, and the CloakBrowser plugin which imports it. This is the one that broke CI: content_fetchers/__init__.py forces playwright when a watch has browser steps, so test_non_200_errors_report_browsersteps ran the playwright path in the pyppeteer jobs too. - puppeteer.py - its own goto retry loop, used when FAST_PUPPETEER_CHROME_FETCHER is set and the watch has no browser steps. No test covers that path; verified by driving the fetcher directly. Note pyppeteer exposes isNavigationRequest / frame / mainFrame as properties where playwright uses is_navigation_request() as a method. Mixing them up raises 'bool' object is not callable, which gets swallowed as a renderer page error rather than failing loudly. Checked against the pinned pyppeteer-ng==2.0.0rc16. Selenium is unaffected - it hardcodes status_code = 200 because WebDriver cannot see the HTTP status, so it never reaches the non-200 branch. Tested with the full CI browser set (test_content, test_errorhandling, test_fetch_data, test_custom_js_before_content): 10 passed on each of Chrome 153 + playwright, Chrome 153 + pyppeteer, Chromium 119 + playwright, Chromium 119 + pyppeteer, the last two against a canonical Dockerfile.chromium119 build so Chromium is the only variable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * CI - Make a fail-fast test abort say what was skipped rather than looking like a total failure Fail-fast is kept deliberately - the first failure is nearly always the real problem and it keeps the run short - but nothing said so, which made a single failing assertion read as "every browser test is broken". The playwright and pyppeteer jobs each ran four pytest files as four commands in one `run:` block, which GitHub executes under `bash -e`. tests/visualselector/test_fetch_data.py is the third, so when one 404 assertion failed there, test_custom_js_before_content.py never ran, and the later "Headers and requests" and "Restock detection" steps were skipped as a consequence - three test files silently dropped, reported only as dashes in the job list. run_basic_tests.sh has the same shape: 8 independent pytest groups under `set -e`, so a failure in the first parallel group hides the 7 after it. No behaviour change to when we stop - only to what gets reported: - Each browser test file now runs inside its own ::group:: so the log is navigable, and the failing file is named in a ::error:: annotation that states plainly that the remaining files and steps were SKIPPED, not failed. - run_basic_tests.sh gets an ERR trap saying the same thing, with the line number of the group that aborted. Verified the loop stops on the third file, names it and exits 1, that the all-pass path still exits 0, and that the trap reports the failing line while preserving the exit code. YAML parses and run_basic_tests.sh passes bash -n. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fix unit test failure - install the navigation-response tracker only on a page that supports events action_goto_url() registered its 'response' listener unconditionally, which broke test_fetch_url_gate.py::TestBrowserStepGotoUrlGate::test_permitted_url_still_navigates: self.page.on("response", _keep_navigation_response) E AttributeError: '_RecordingPage' object has no attribute 'on' The three refusal tests in that class still passed because validate_fetch_url_async() raises before reaching the listener, so only the permitted-URL case (the one that actually navigates) hit it. The listener now lives in track_latest_navigation_response(), which returns None for a page that has no event support instead of raising. That also removes a real inefficiency: registering per navigation meant a page accumulated a listener per goto(), and 'response' fires for every subresource - measured 133 events on getastra.com (54 script, 37 image, 19 fetch, 11 xhr, ...) of which only 2 were navigations. The tracker is installed once per page and shared, verified as one listener remaining after an install plus four navigations. Tested: 494 unit + llm tests pass (was 1 failed / 480 passed), and the full browser set still passes 10/10 on playwright and 10/10 on pyppeteer, so the Chrome 153 "Error - 404" recovery still works through the shared tracker. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-12 13:15:44 +02:00
dgtlmoon	050172129e	Translations - support for translation overlays/local translations of existing strings (#4365) ... * Translations - support for translation overlays/local translations of existing strings * Dont forget translations_overlay for pip	2026-09-03 11:15:46 +02:00
dgtlmoon	7b61c20e68	Python 3.10 EOL October/2026 - Dropping support - conflicts with jsomschema and litellm (#4347)	2026-08-31 21:19:12 +02:00
dependabot[bot]	c5d04a8c96	Build robot - Bump actions/setup-python in the all group across 1 directory (#4291)	2026-08-24 14:55:17 +02:00
Nameless-Monster-NerdandNameless-Monster-Nerd	3a7d18dd3c	Move development tools out of runtime requirements (#4280) ... Co-authored-by: Nameless-Monster-Nerd <207905925+Nameless-Monster-Nerd@users.noreply.github.com>	2026-08-20 11:10:52 +02:00
dependabot[bot]	44633202de	Build - Bump the all group across 1 directory with 2 updates (#4238)	2026-07-13 08:54:13 +02:00
skkzsh	851c054f8b	lint: Bump dennis — adopt --strict mode and drop false-positive workarounds (#4182)	2026-05-25 16:51:45 +02:00
skkzsh	e45f87578b	i18n: Fix broken HTML tags and enforce dennis lint warnings in CI (#4116)	2026-05-05 05:51:54 +02:00
skkzsh	281c1ea7e1	i18n: Clear pre-existing dennis warnings in messages.pot (#4112)	2026-05-02 04:57:13 +02:00
skkzsh	cf31823d53	i18n: Enforce dennis lint warnings in CI (#4105)	2026-04-30 13:04:12 +02:00
skkzsh	7831a499b2	i18n: Add dennis .pot/.po lint (#4097)	2026-04-29 09:11:03 +02:00
skkzsh	2839a4276e	Ruff INT (flake8-gettext) (#4096)	2026-04-28 19:46:58 +10:00
dgtlmoon	79d75f7926	Translations - Playwright macro unused, add extra linting for translations, add TRANSLATORS.md (#4087)	2026-04-26 12:36:50 +02:00
dgtlmoon	e9e8c8d218	LLM / AI Change detection rules and Summaries	2026-04-24 15:21:09 +02:00
dgtlmoon	76dcd7b99a	CI - Translation sync check (#4085)	2026-04-24 12:11:02 +02:00
dgtlmoon	788acabdd8	CI - Re #4080 msgfmt linting (#4081)	2026-04-24 10:45:50 +02:00
dgtlmoon	7c9eb02df4	Ensure all unit tests are run (#4022)	2026-04-03 07:16:52 +02:00
dgtlmoon	0ad4090d68	Extendable theme pluggy implementation for main theme/template <head> section (#4011)	2026-03-27 12:28:13 +01:00
dgtlmoon	c18421fbe9	CI - YML tidyup	2026-03-12 16:46:14 +01:00
dgtlmoon	f29d6a857b	Docker image - Improving org.opencontainers labels for dev containers	2026-03-12 16:41:45 +01:00
dgtlmoon	fcfe089a53	Docker image - Improving org.opencontainers labels #3794	2026-03-12 16:36:07 +01:00
dependabot[bot]	e400e463a4	CI - Bump the all group with 5 updates (#3955)	2026-03-06 09:31:10 +01:00
dgtlmoon	b09ebcbef6	Python 3.14 CI test and support (#3941)	2026-03-02 11:28:52 +01:00
dependabot[bot]	6d6a0fd7ef	CI workflow - Bump the all group with 2 updates (#3931)	2026-02-27 10:06:08 +01:00
dgtlmoon	2f7315e29c	Tests - Tweaks to upgrade path tests	2026-02-23 22:20:13 +01:00
dgtlmoon	bf3f8eae45	Tests - Run upgrade path test with ALLOW_IANA_RESTRICTED_ADDRESSES=true	2026-02-23 21:59:00 +01:00
dgtlmoon	383c3b427f	API - Adding automated test for API with NGINX sub-path, Skip validation errors about server path (allows use on sub-paths/reverse proxy etc) (#3886)	2026-02-16 20:32:35 +01:00
dgtlmoon	ef41dd304c	Refactoring upgrade path (#3861)	2026-02-11 16:13:08 +01:00
dgtlmoon	83d7ce0fcf	Processor plugin improvements - Now supports creating your own processor (for example, monitor DNS changes) (#3739)	2026-02-04 14:23:08 +01:00
dgtlmoon	30dc4ac23b	Refactor of queue systenm and improve tests, improves multiple workers (#3826)	2026-02-02 22:28:27 +01:00
dgtlmoon	fceb3cf39f	Big refactor to save watches as their own datafile with some agnostic data store backend, saves writing a huge JSON file every time (#3775)	2026-01-28 10:18:21 +01:00
dgtlmoon	4dec1e017b	CLI extra options, "batch mode" see --help allows re-checking and adding watches from the CLI (#3802)	2026-01-24 14:18:06 +01:00
dgtlmoon	10f233a939	Improving container version labeling, using master branch as docker :dev tag. Re #3794	2026-01-22 05:38:56 +01:00
dgtlmoon	bee1130c6e	Important fix for possible wrong detection of changes under high-concurrency setups (many many fetch workers)	2026-01-17 14:45:23 +01:00
dgtlmoon	3d14df6a11	Development branch merge into release/master ... Multi-language / Translations Support (#3696) - Complete internationalization system implemented - Support for 7 languages: Czech (cs), German (de), French (fr), Italian (it), Korean (ko), Chinese Simplified (zh), Chinese Traditional (zh_TW) - Language selector with localized flags and theming - Flash message translations - Multiple translation fixes and improvements across all languages - Language setting preserved across redirects Pluggable Content Fetchers (#3653) - New architecture for extensible content fetcher system - Allows custom fetcher implementations Image / Screenshot Comparison Processor (#3680) - New processor for visual change detection (disabled for this release) - Supporting CSS/JS infrastructure added UI Improvements Design & Layout - Auto-generated tag color schemes - Simplified login form styling - Removed hard-coded CSS, moved to SCSS variables - Tag UI cleanup and improvements - Automatic tab wrapper functionality - Menu refactoring for better organization - Cleanup of offset settings - Hide sticky tabs on narrow viewports - Improved responsive layout (#3702) User Experience - Modal alerts/confirmations on delete/clear operations (#3693, #3598, #3382) - Auto-add https:// to URLs in quickwatch form if not present - Better redirect handling on login (#3699) - 'Recheck all' now returns to correct group/tag (#3673) - Language set redirect keeps hash fragment - More friendly human-readable text throughout UI Performance & Reliability Scheduler & Processing - Soft delays instead of blocking time.sleep() calls (#3710) - More resilient handling of same UUID being processed (#3700) - Better Puppeteer timeout handling - Improved Puppeteer shutdown/cleanup (#3692) - Requests cleanup now properly async History & Rendering - Faster server-side "difference" rendering on History page (#3442) - Show ignored/triggered rows in history - API: Retry watch data if watch dict changed (more reliable) API Improvements - Watch get endpoint: retry mechanism for changed watch data - WatchHistoryDiff API endpoint includes extra format args (#3703) Testing Improvements - Replace time.sleep with wait_for_notification_endpoint_output (#3716) - Test for mode switching (#3701) - Test for #3720 added (#3725) - Extract-text difference test fixes - Improved dev workflow Bug Fixes - Notification error text output (#3672, #3669, #3280) - HTML validation fixes (#3704) - Template discovery path fixes - Notification debug log now uses system locale for dates/times - Puppeteer spelling mistake in log output - Recalculation on anchor change - Queue bubble update disabled temporarily Dependency Updates - beautifulsoup4 updated (#3724) - psutil 7.1.0 → 7.2.1 (#3723) - python-engineio ~=4.12.3 → ~=4.13.0 (#3707) - python-socketio ~=5.14.3 → ~=5.16.0 (#3706) - flask-socketio ~=5.5.1 → ~=5.6.0 (#3691) - brotli ~=1.1 → ~=1.2 (#3687) - lxml updated (#3590) - pytest ~=7.2 → ~=9.0 (#3676) - jsonschema ~=4.0 → ~=4.25 (#3618) - pluggy ~=1.5 → ~=1.6 (#3616) - cryptography 44.0.1 → 46.0.3 (security) (#3589) Documentation - README updated with viewport size setup information Development Infrastructure - Dev container only built on dev branch - Improved dev workflow tooling	2026-01-12 17:50:53 +01:00
dependabot[bot]	f78c2dcffd	Bump actions/checkout from 5 to 6 in the all group (#3651)	2025-11-24 02:03:13 +01:00
dependabot[bot]	f638bc6332	Bump actions/download-artifact from 5 to 6 in the all group (#3585)	2025-11-03 09:59:26 +01:00
dgtlmoon	b0980f45b8	Dockerfile cache tweaks and build layer github cache re-enable (#3575)	2025-10-30 12:18:12 +01:00
dgtlmoon	1628586553	Optimisations to GitHub test flow	2025-10-28 22:35:09 +01:00
dgtlmoon	ae532c82e8	Run all pytests in parallel (#3569)	2025-10-28 21:32:25 +01:00
dgtlmoon	66aec365c2	Build/test - Parallel test jobs for faster testing (#3568)	2025-10-28 13:24:22 +01:00
dependabot[bot]	a7573b10ec	Build - Actions / Bump the all group with 2 updates (#3550)	2025-10-27 10:27:54 +01:00
dgtlmoon	ec692ed727	pip build - Improving fix for #3509, Adding automated test for #3509	2025-10-17 12:48:50 +02:00
dgtlmoon	047c10e23c	Notification service improved failure alerts for filter missing + browsersteps problems (#3507)	2025-10-16 14:30:50 +02:00
dgtlmoon	233189e4f7	Build - Splitting memory report (#3493)	2025-10-12 19:20:28 +02:00
dgtlmoon	b237fd7201	Replace stream/filetype detection library with puremagic, 20Mb less RAM usage (#3491)	2025-10-12 18:40:37 +02:00
dependabot[bot]	0970c087c8	Bump github/codeql-action from 3 to 4 in the all group (#3468)	2025-10-09 10:47:12 +02:00
dgtlmoon	bb61a35a54	Build - Fixing the multi platform container build test (repairs to cache) (#3455)	2025-10-03 17:22:16 +02:00
dgtlmoon	f4e8d1963f	Build - linux/arm64 and linux/arm64/v8 are the same, remove v8	2025-09-29 14:12:10 +02:00
dependabot[bot]	7576bec66a	Build - Bump actions/setup-python from 5 to 6 in the all group (#3408)	2025-09-08 12:12:04 +02:00