Compare commits

..

40 Commits

Author SHA1 Message Date
dgtlmoon 6fb0621263 More translation updates 2026-01-03 14:46:34 +01:00
dgtlmoon de5d5c1d86 More translation improvements 2026-01-03 14:33:59 +01:00
dgtlmoon d85842d1cd Improving translations 2026-01-03 14:24:28 +01:00
dgtlmoon 0e58ffdc82 More styling improvements 2026-01-03 13:48:07 +01:00
dgtlmoon 7f03a3e132 Fix styling 2026-01-03 13:27:18 +01:00
dgtlmoon 85737694f9 Adding toast libraries 2026-01-03 13:19:18 +01:00
dgtlmoon 40cae6fc42 Adding toast 2026-01-03 13:18:56 +01:00
dgtlmoon 060f9a806d Fixing mobile mobile menu and adding action menu 2026-01-03 12:59:31 +01:00
dgtlmoon 9627c63695 Adding test sidebar 2026-01-03 12:33:23 +01:00
dgtlmoon cedabf4ff6 Language set redirect - keep hash
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2026-01-03 01:59:42 +01:00
dgtlmoon 03116fef8f Adding small test for switching modes (#3701)
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Has been cancelled
2026-01-02 19:51:21 +01:00
dgtlmoon b1257dd196 UI - Handling redirects on login to the correct page (#3699) 2026-01-02 17:46:25 +01:00
dgtlmoon 7e61f5b663 more resilient same UUID being processed (#3700) 2026-01-02 17:46:12 +01:00
dgtlmoon afa8451448 Puppeteer - Improvements to timeout handling 2026-01-02 17:45:41 +01:00
dgtlmoon b5023a6fda Adding flash() translations (#3698) 2026-01-02 16:41:31 +01:00
dgtlmoon 895368144f Localising flags 2026-01-02 15:14:18 +01:00
dgtlmoon 9096407fcb Multi-language / Translations support (#3696) 2026-01-02 15:01:22 +01:00
dgtlmoon df8f86ccbf Fix template discovery path 2026-01-02 14:49:27 +01:00
dgtlmoon 40dc3fef7e Difference - Fixing test for extract-text
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2026-01-02 11:22:43 +01:00
dgtlmoon 5f4998960d Puppeteer - Spelling mistake in log output 2026-01-02 11:22:31 +01:00
dependabot[bot] 7a515c4202 Bump cryptography from 44.0.1 to 46.0.3 (#3589)
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-12-31 01:06:06 +01:00
dgtlmoon 48e21226a1 UI - Add modal alert/confirmations on delete/clear #3598 #3382 (#3693)
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-12-30 19:03:26 +01:00
dgtlmoon cdf34bf614 CSS/JS For image comparison 2025-12-30 18:17:33 +01:00
dgtlmoon a94560190f Adding new Processor - Image / screenshot comparison (disabled for this release) (#3680) 2025-12-30 18:03:52 +01:00
dgtlmoon fefaf40514 UI - Add https:// to URL on quickwatch form if not present
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-12-29 14:52:03 +01:00
dgtlmoon 6f66c39628 Requests - cleanup should be async function 2025-12-29 14:51:56 +01:00
dgtlmoon eb0f83b45b Puppeteer fetcher - Better shutdown/cleanup handling (#3692)
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Has been cancelled
2025-12-29 10:24:01 +01:00
dependabot[bot] f2284f7a9b Update flask-socketio requirement from ~=5.5.1 to ~=5.6.0 (#3691) 2025-12-29 10:23:00 +01:00
dependabot[bot] 4b0ad525f3 Update brotli requirement from ~=1.1 to ~=1.2 (#3687)
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-12-22 10:19:46 +01:00
dgtlmoon a748a43224 "History" page - Use faster server side "difference" rendering, show ignored/triggered rows (#3442)
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-12-15 15:39:07 +01:00
dependabot[bot] acfcaf42d4 Update lxml requirement (#3590) 2025-12-15 15:38:12 +01:00
dependabot[bot] 6158bb48b8 Update pytest requirement from ~=7.2 to ~=9.0 (#3676)
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-12-15 11:46:33 +01:00
dependabot[bot] d4fc1a3b6e Bump the all group with 3 updates (#3678) 2025-12-15 11:45:54 +01:00
dependabot[bot] f39b5e5a46 Update jsonschema requirement from ~=4.0 to ~=4.25 (#3618)
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-12-15 00:04:32 +01:00
dgtlmoon 30ba603956 UI - 'Recheck all' should return back to the correct group/tag (#3673)
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-12-11 17:24:29 +01:00
dependabot[bot] 3147c5a3e2 Update pluggy requirement from ~=1.5 to ~=1.6 (#3616) 2025-12-11 17:16:30 +01:00
dgtlmoon f599efacab Pluggable content fetchers (#3653) 2025-12-11 17:16:14 +01:00
dgtlmoon d7dbc50d70 UI - Notification error text output fix #3669 #3280 (#3672) 2025-12-11 16:57:06 +01:00
dgtlmoon 51bb358ea7 Improving dev workflow 2025-11-28 16:20:11 +01:00
dgtlmoon fe4df1d41f 'dev' container should be only built on 'dev' branch 2025-11-28 16:16:23 +01:00
75 changed files with 814 additions and 6076 deletions
-3
View File
@@ -183,9 +183,6 @@ docker compose pull && docker compose up -d
See the wiki for more information https://github.com/dgtlmoon/changedetection.io/wiki
## Different browser viewport sizes (mobile, desktop etc)
If you are using the recommended `sockpuppetbrowser` (which is in the docker-compose.yml as a setting to be uncommented) you can easily set different viewport sizes for your web page change detection, [see more information here about setting up different viewport sizes](https://github.com/dgtlmoon/sockpuppetbrowser?tab=readme-ov-file#setting-viewport-size).
## Filters
+1 -1
View File
@@ -2,7 +2,7 @@
# Read more https://github.com/dgtlmoon/changedetection.io/wiki
# Semver means never use .01, or 00. Should be .1.
__version__ = '0.52.3'
__version__ = '0.51.4'
from changedetectionio.strtobool import strtobool
from json.decoder import JSONDecodeError
+9 -28
View File
@@ -64,17 +64,8 @@ class Watch(Resource):
@validate_openapi_request('getWatch')
def get(self, uuid):
"""Get information about a single watch, recheck, pause, or mute."""
import time
from copy import deepcopy
watch = None
for _ in range(20):
try:
watch = deepcopy(self.datastore.data['watching'].get(uuid))
break
except RuntimeError:
# Incase dict changed, try again
time.sleep(0.01)
watch = deepcopy(self.datastore.data['watching'].get(uuid))
if not watch:
abort(404, message='No watch exists with the UUID of {}'.format(uuid))
@@ -311,28 +302,18 @@ class WatchHistoryDiff(Resource):
from_version_file_contents = watch.get_history_snapshot(from_timestamp)
to_version_file_contents = watch.get_history_snapshot(to_timestamp)
# Get diff preferences from query parameters (matching UI preferences in DIFF_PREFERENCES_CONFIG)
# Support both 'type' (UI parameter) and 'word_diff' (API parameter) for backward compatibility
diff_type = request.args.get('type', 'diffLines')
if diff_type == 'diffWords':
word_diff = True
# Get diff preferences (using defaults similar to the existing code)
diff_prefs = {
'diff_ignoreWhitespace': False,
'diff_changesOnly': True
}
# Get boolean diff preferences with defaults from DIFF_PREFERENCES_CONFIG
changes_only = strtobool(request.args.get('changesOnly', 'true'))
ignore_whitespace = strtobool(request.args.get('ignoreWhitespace', 'false'))
include_removed = strtobool(request.args.get('removed', 'true'))
include_added = strtobool(request.args.get('added', 'true'))
include_replaced = strtobool(request.args.get('replaced', 'true'))
# Generate the diff with all preferences
# Generate the diff
content = diff.render_diff(
previous_version_file_contents=from_version_file_contents,
newest_version_file_contents=to_version_file_contents,
ignore_junk=ignore_whitespace,
include_equal=changes_only,
include_removed=include_removed,
include_added=include_added,
include_replaced=include_replaced,
ignore_junk=diff_prefs.get('diff_ignoreWhitespace'),
include_equal=not diff_prefs.get('diff_changesOnly'),
word_diff=word_diff,
)
+13 -26
View File
@@ -17,48 +17,36 @@ from loguru import logger
# Async version of update_worker
# Processes jobs from AsyncSignalPriorityQueue instead of threaded queue
async def async_update_worker(worker_id, q, notification_q, app, datastore, executor=None):
async def async_update_worker(worker_id, q, notification_q, app, datastore):
"""
Async worker function that processes watch check jobs from the queue.
Args:
worker_id: Unique identifier for this worker
q: AsyncSignalPriorityQueue containing jobs to process
notification_q: Standard queue for notifications
app: Flask application instance
datastore: Application datastore
executor: ThreadPoolExecutor for queue operations (optional)
"""
# Set a descriptive name for this task
task = asyncio.current_task()
if task:
task.set_name(f"async-worker-{worker_id}")
logger.info(f"Starting async worker {worker_id}")
while not app.config.exit.is_set():
update_handler = None
watch = None
try:
# Use sync interface via run_in_executor since each worker has its own event loop
loop = asyncio.get_event_loop()
queued_item_data = await asyncio.wait_for(
loop.run_in_executor(executor, q.get, True, 1.0), # block=True, timeout=1.0
timeout=1.5
)
# Use native janus async interface - no threads needed!
queued_item_data = await asyncio.wait_for(q.async_get(), timeout=1.0)
except asyncio.TimeoutError:
# No jobs available, continue loop
continue
except Exception as e:
# Handle expected Empty exception from queue timeout
import queue
if isinstance(e, queue.Empty):
# Queue is empty, normal behavior - just continue
continue
# Unexpected exception - log as critical
logger.critical(f"CRITICAL: Worker {worker_id} failed to get queue item: {type(e).__name__}: {e}")
# Log queue health for debugging
@@ -426,13 +414,14 @@ async def async_update_worker(worker_id, q, notification_q, app, datastore, exec
datastore.update_watch(uuid=uuid, update_obj={'last_error': f"Worker error: {str(e)}"})
finally:
try:
await update_handler.fetcher.quit(watch=watch)
except Exception as e:
logger.error(f"Exception while cleaning/quit after calling browser: {e}")
# Always cleanup - this runs whether there was an exception or not
if uuid:
try:
if update_handler and hasattr(update_handler, 'fetcher') and update_handler.fetcher:
await update_handler.fetcher.quit(watch=watch)
except Exception as e:
logger.error(f"Exception while cleaning/quit after calling browser: {e}")
try:
# Mark UUID as no longer being processed by this worker
worker_handler.set_uuid_processing(uuid, worker_id=worker_id, processing=False)
@@ -471,9 +460,7 @@ async def async_update_worker(worker_id, q, notification_q, app, datastore, exec
logger.debug(f"Worker {worker_id} completed watch {uuid} in {time.time()-fetch_start_time:.2f}s")
except Exception as cleanup_error:
logger.error(f"Worker {worker_id} error during cleanup: {cleanup_error}")
del(uuid)
# Brief pause before continuing to avoid tight error loops (only on error)
if 'e' in locals():
await asyncio.sleep(1.0)
@@ -92,12 +92,7 @@ def construct_blueprint(datastore: ChangeDetectionStore):
# Be sure we're written fresh
datastore.sync_to_json()
zip_thread = threading.Thread(
target=create_backup,
args=(datastore.datastore_path, datastore.data.get("watching")),
daemon=True,
name="BackupCreator"
)
zip_thread = threading.Thread(target=create_backup, args=(datastore.datastore_path, datastore.data.get("watching")))
zip_thread.start()
backup_threads.append(zip_thread)
flash(gettext("Backup building in background, check back in a few minutes."))
@@ -21,154 +21,31 @@ from changedetectionio.flask_app import login_optionally_required
from loguru import logger
browsersteps_sessions = {}
browsersteps_watch_to_session = {} # Maps watch_uuid -> browsersteps_session_id
io_interface_context = None
import json
import hashlib
from flask import Response
import asyncio
import threading
import time
# Dedicated event loop for ALL browser steps sessions
_browser_steps_loop = None
_browser_steps_thread = None
_browser_steps_loop_lock = threading.Lock()
def _start_browser_steps_loop():
"""Start a dedicated event loop for browser steps in its own thread"""
global _browser_steps_loop
# Create and set the event loop for this thread
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
_browser_steps_loop = loop
logger.debug("Browser steps event loop started")
try:
# Run the loop forever - handles all browsersteps sessions
loop.run_forever()
except Exception as e:
logger.error(f"Browser steps event loop error: {e}")
finally:
try:
# Cancel all remaining tasks
pending = asyncio.all_tasks(loop)
for task in pending:
task.cancel()
# Wait for tasks to finish cancellation
if pending:
loop.run_until_complete(asyncio.gather(*pending, return_exceptions=True))
except Exception as e:
logger.debug(f"Error during browser steps loop cleanup: {e}")
finally:
loop.close()
logger.debug("Browser steps event loop closed")
def _ensure_browser_steps_loop():
"""Ensure the browser steps event loop is running"""
global _browser_steps_loop, _browser_steps_thread
with _browser_steps_loop_lock:
if _browser_steps_thread is None or not _browser_steps_thread.is_alive():
logger.debug("Starting browser steps event loop thread")
_browser_steps_thread = threading.Thread(
target=_start_browser_steps_loop,
daemon=True,
name="BrowserStepsEventLoop"
)
_browser_steps_thread.start()
# Wait for the loop to be ready
timeout = 5.0
start_time = time.time()
while _browser_steps_loop is None:
if time.time() - start_time > timeout:
raise RuntimeError("Browser steps event loop failed to start")
time.sleep(0.01)
logger.debug("Browser steps event loop thread started and ready")
def run_async_in_browser_loop(coro):
"""Run async coroutine using the dedicated browser steps event loop"""
_ensure_browser_steps_loop()
if _browser_steps_loop and not _browser_steps_loop.is_closed():
logger.debug("Browser steps using dedicated event loop")
future = asyncio.run_coroutine_threadsafe(coro, _browser_steps_loop)
"""Run async coroutine using the existing async worker event loop"""
from changedetectionio import worker_handler
# Use the existing async worker event loop instead of creating a new one
if worker_handler.USE_ASYNC_WORKERS and worker_handler.async_loop and not worker_handler.async_loop.is_closed():
logger.debug("Browser steps using existing async worker event loop")
future = asyncio.run_coroutine_threadsafe(coro, worker_handler.async_loop)
return future.result()
else:
raise RuntimeError("Browser steps event loop is not available")
def cleanup_expired_sessions():
"""Remove expired browsersteps sessions and cleanup their resources"""
global browsersteps_sessions, browsersteps_watch_to_session
expired_session_ids = []
# Find expired sessions
for session_id, session_data in browsersteps_sessions.items():
browserstepper = session_data.get('browserstepper')
if browserstepper and browserstepper.has_expired:
expired_session_ids.append(session_id)
# Cleanup expired sessions
for session_id in expired_session_ids:
logger.debug(f"Cleaning up expired browsersteps session {session_id}")
session_data = browsersteps_sessions[session_id]
# Cleanup playwright resources asynchronously
browserstepper = session_data.get('browserstepper')
if browserstepper:
try:
run_async_in_browser_loop(browserstepper.cleanup())
except Exception as e:
logger.error(f"Error cleaning up session {session_id}: {e}")
# Remove from sessions dict
del browsersteps_sessions[session_id]
# Remove from watch mapping
for watch_uuid, mapped_session_id in list(browsersteps_watch_to_session.items()):
if mapped_session_id == session_id:
del browsersteps_watch_to_session[watch_uuid]
break
if expired_session_ids:
logger.info(f"Cleaned up {len(expired_session_ids)} expired browsersteps session(s)")
def cleanup_session_for_watch(watch_uuid):
"""Cleanup a specific browsersteps session for a watch UUID"""
global browsersteps_sessions, browsersteps_watch_to_session
session_id = browsersteps_watch_to_session.get(watch_uuid)
if not session_id:
logger.debug(f"No browsersteps session found for watch {watch_uuid}")
return
logger.debug(f"Cleaning up browsersteps session {session_id} for watch {watch_uuid}")
session_data = browsersteps_sessions.get(session_id)
if session_data:
browserstepper = session_data.get('browserstepper')
if browserstepper:
try:
run_async_in_browser_loop(browserstepper.cleanup())
except Exception as e:
logger.error(f"Error cleaning up session {session_id} for watch {watch_uuid}: {e}")
# Remove from sessions dict
del browsersteps_sessions[session_id]
# Remove from watch mapping
del browsersteps_watch_to_session[watch_uuid]
logger.debug(f"Cleaned up session for watch {watch_uuid}")
# Opportunistically cleanup any other expired sessions
cleanup_expired_sessions()
# Fallback: create a new event loop (for sync workers or if async loop not available)
logger.debug("Browser steps creating temporary event loop")
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
return loop.run_until_complete(coro)
finally:
loop.close()
def construct_blueprint(datastore: ChangeDetectionStore):
browser_steps_blueprint = Blueprint('browser_steps', __name__, template_folder="templates")
@@ -246,9 +123,6 @@ def construct_blueprint(datastore: ChangeDetectionStore):
if not watch_uuid:
return make_response('No Watch UUID specified', 500)
# Cleanup any existing session for this watch
cleanup_session_for_watch(watch_uuid)
logger.debug("Starting connection with playwright")
logger.debug("browser_steps.py connecting")
@@ -257,10 +131,6 @@ def construct_blueprint(datastore: ChangeDetectionStore):
browsersteps_sessions[browsersteps_session_id] = run_async_in_browser_loop(
start_browsersteps_session(watch_uuid)
)
# Store the mapping of watch_uuid -> browsersteps_session_id
browsersteps_watch_to_session[watch_uuid] = browsersteps_session_id
except Exception as e:
if 'ECONNREFUSED' in str(e):
return make_response('Unable to start the Playwright Browser session, is sockpuppetbrowser running? Network configuration is OK?', 401)
@@ -47,6 +47,9 @@ def construct_single_watch_routes(rss_blueprint, datastore):
if len(dates) < 2:
return f"Watch {uuid} does not have enough history snapshots to show changes (need at least 2)", 400
# Add uuid to watch for proper functioning
watch['uuid'] = uuid
# Get the number of diffs to include (default: 5)
rss_diff_length = datastore.data['settings']['application'].get('rss_diff_length', 5)
@@ -98,7 +101,7 @@ def construct_single_watch_routes(rss_blueprint, datastore):
date_index_from, date_index_to)
# Create and populate feed entry
guid = f"{uuid}/{timestamp_to}"
guid = f"{watch['uuid']}/{timestamp_to}"
fe = fg.add_entry()
title_suffix = f"Change @ {res['original_context']['change_datetime']}"
populate_feed_entry(fe, watch, res.get('body', ''), guid, timestamp_to,
+5 -2
View File
@@ -63,8 +63,11 @@ def construct_tag_routes(rss_blueprint, datastore):
# Only include unviewed watches
if not watch.viewed:
# Include a link to the diff page (use uuid from loop, don't modify watch dict)
diff_link = {'href': url_for('ui.ui_diff.diff_history_page', uuid=uuid, _external=True)}
# Add uuid to watch for proper functioning
watch['uuid'] = uuid
# Include a link to the diff page
diff_link = {'href': url_for('ui.ui_diff.diff_history_page', uuid=watch['uuid'], _external=True)}
# Get watch label
watch_label = get_watch_label(datastore, watch)
@@ -85,7 +85,9 @@
<div class="tab-pane-inner" id="notifications">
<fieldset>
{{ render_common_settings_form(form.application.form, emailprefix, settings_application, extra_notification_token_placeholder_info) }}
<div class="field-group">
{{ render_common_settings_form(form.application.form, emailprefix, settings_application, extra_notification_token_placeholder_info) }}
</div>
</fieldset>
<div class="pure-control-group" id="notification-base-url">
{{ render_field(form.application.form.base_url, class="m-d") }}
@@ -126,7 +128,7 @@
</div>
<div class="pure-control-group">
{{ render_field(form.requests.form.timeout) }}
<span class="pure-form-message-inline">For regular plain requests (not chrome based), maximum number of seconds until timeout, 1-999.</span><br>
<span class="pure-form-message-inline">For regular plain requests (not chrome based), maximum number of seconds until timeout, 1-999.<br>
</div>
<div class="pure-control-group inline-radio">
{{ render_field(form.requests.form.default_ua) }}
@@ -217,7 +219,7 @@ nav
<a id="chrome-extension-link"
title="Try our new Chrome Extension!"
href="https://chromewebstore.google.com/detail/changedetectionio-website/kefcfmgmlhmankjmnbijimhofdjekbop">
<img alt="Chrome store icon" src="{{ url_for('static_content', group='images', filename='google-chrome-icon.png') }}" >
<img alt="Chrome store icon" src="{{ url_for('static_content', group='images', filename='google-chrome-icon.png') }}" alt="Chrome">
Chrome Webstore
</a>
</p>
@@ -258,14 +260,14 @@ nav
Ensure the settings below are correct, they are used to manage the time schedule for checking your web page watches.
</div>
<div class="pure-control-group">
<p><strong>UTC Time &amp; Date from Server:</strong> <span id="utc-time" >{{ utc_time }}</span></p>
<p><strong>Local Time &amp; Date in Browser:</strong> <span class="local-time" data-utc="{{ utc_time }}"></span></p>
<div>
<p><strong>UTC Time &amp Date from Server:</strong> <span id="utc-time" >{{ utc_time }}</span></p>
<p><strong>Local Time &amp Date in Browser:</strong> <span class="local-time" data-utc="{{ utc_time }}"></span></p>
<p>
{{ render_field(form.application.form.scheduler_timezone_default) }}
<datalist id="timezones" style="display: none;">
{%- for timezone in available_timezones -%}<option value="{{ timezone }}">{{ timezone }}</option>{%- endfor -%}
</datalist>
</div>
</p>
</div>
</div>
<div class="tab-pane-inner" id="ui-options">
@@ -334,7 +336,7 @@ nav
</div>
</div>
<p><strong>Tip</strong>: "Residential" and "Mobile" proxy type can be more successfull than "Data Center" for blocked websites.</p>
<p><strong>Tip</strong>: "Residential" and "Mobile" proxy type can be more successfull than "Data Center" for blocked websites.
<div class="pure-control-group" id="extra-proxies-setting">
{{ render_fieldlist_with_inline_errors(form.requests.form.extra_proxies) }}
@@ -50,8 +50,7 @@
<td>{{ "{:,}".format(tag_count[uuid]) if uuid in tag_count else 0 }}</td>
<td class="title-col inline"> <a href="{{url_for('watchlist.index', tag=uuid) }}">{{ tag.title }}</a></td>
<td>
<a class="pure-button pure-button-primary" href="{{ url_for('tags.form_tag_edit', uuid=uuid) }}">{{ _('Edit') }}</a>
<a href="{{ url_for('ui.form_watch_checknow', tag=uuid) }}" class="pure-button pure-button-primary" >{{ _('Recheck') }}</a>
<a class="pure-button pure-button-primary" href="{{ url_for('tags.form_tag_edit', uuid=uuid) }}">{{ _('Edit') }}</a>&nbsp;
<a class="pure-button button-error"
href="{{ url_for('tags.delete', uuid=uuid) }}"
data-requires-confirm
+2 -7
View File
@@ -238,13 +238,6 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, queuedWatchMe
datastore.data['watching'][uuid] = watch_class(datastore_path=datastore.datastore_path, default=datastore.data['watching'][uuid])
flash(gettext("Updated watch - unpaused!") if request.args.get('unpause_on_save') else gettext("Updated watch."))
# Cleanup any browsersteps session for this watch
try:
from changedetectionio.blueprint.browser_steps import cleanup_session_for_watch
cleanup_session_for_watch(uuid)
except Exception as e:
logger.debug(f"Error cleaning up browsersteps session: {e}")
# Re #286 - We wait for syncing new data to disk in another thread every 60 seconds
# But in the case something is added we should save straight away
datastore.needs_write_urgent = True
@@ -332,6 +325,8 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, queuedWatchMe
'url': url_for('rss.rss_single_watch', uuid=watch['uuid'], token=app_rss_token)
},
'settings_application': datastore.data['settings']['application'],
'system_has_playwright_configured': os.getenv('PLAYWRIGHT_DRIVER_URL'),
'system_has_webdriver_configured': os.getenv('WEBDRIVER_URL'),
'ui_edit_stats_extras': collect_ui_edit_stats_extras(watch),
'visual_selector_data_ready': datastore.visualselector_data_is_ready(watch_uuid=uuid),
'timezone_default_config': datastore.data['settings']['application'].get('scheduler_timezone_default'),
@@ -118,7 +118,6 @@ def construct_blueprint(datastore: ChangeDetectionStore):
sent_obj = process_notification(n_object, datastore)
except Exception as e:
logger.error(e)
e_str = str(e)
# Remove this text which is not important and floods the container
e_str = e_str.replace(
@@ -87,7 +87,7 @@
</form>
</div>
<div id="diff-jump" style="display:none;"><!-- disabled for now -->
<div id="diff-jump">
<a id="jump-next-diff" title="{{ _('Jump to next difference') }}">{{ _('Jump') }}</a>
</div>
@@ -206,7 +206,7 @@ Math: {{ 1 + 1 }}") }}
<div class="tab-pane-inner" id="browser-steps">
{% if capabilities.supports_browser_steps %}
{% if true %}
{% if visual_selector_data_ready %}
<img class="beta-logo" src="{{url_for('static_content', group='images', filename='beta-logo.png')}}" alt="New beta functionality">
<fieldset>
<div class="pure-control-group">
@@ -86,7 +86,6 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, queuedWatchMe
datastore=datastore,
errored_count=errored_count,
form=form,
generate_tag_colors=processors.generate_processor_badge_colors,
guid=datastore.data['app_guid'],
has_proxies=datastore.proxy_list,
hosted_sticky=os.getenv("SALTED_PASS", False) == False,
@@ -22,33 +22,6 @@ document.addEventListener('DOMContentLoaded', function() {
/* Auto-generated processor badge colors */
{{ processor_badge_css|safe }}
/* Auto-generated tag colors */
{%- for uuid, tag in tags -%}
{%- if tag and tag.title -%}
{%- set class_name = tag.title|sanitize_tag_class -%}
{%- set colors = generate_tag_colors(tag.title) -%}
.button-tag.tag-{{ class_name }} {
background-color: {{ colors['light']['bg'] }};
color: {{ colors['light']['color'] }};
}
.watch-tag-list.tag-{{ class_name }} {
background-color: {{ colors['light']['bg'] }};
color: {{ colors['light']['color'] }};
}
html[data-darkmode="true"] .button-tag.tag-{{ class_name }} {
background-color: {{ colors['dark']['bg'] }};
color: {{ colors['dark']['color'] }};
}
html[data-darkmode="true"] .watch-tag-list.tag-{{ class_name }} {
background-color: {{ colors['dark']['bg'] }};
color: {{ colors['dark']['color'] }};
}
{%- endif -%}
{%- endfor -%}
</style>
<div class="box" id="form-quick-watch-add">
@@ -109,7 +82,7 @@ html[data-darkmode="true"] .watch-tag-list.tag-{{ class_name }} {
<!-- tag list -->
{%- for uuid, tag in tags -%}
{%- if tag != "" -%}
<a href="{{url_for('watchlist.index', tag=uuid) }}" class="pure-button button-tag tag-{{ tag.title|sanitize_tag_class }} {{'active' if active_tag_uuid == uuid }}">{{ tag.title }}</a>
<a href="{{url_for('watchlist.index', tag=uuid) }}" class="pure-button button-tag {{'active' if active_tag_uuid == uuid }}">{{ tag.title }}</a>
{%- endif -%}
{%- endfor -%}
</div>
@@ -196,7 +169,7 @@ html[data-darkmode="true"] .watch-tag-list.tag-{{ class_name }} {
<div class="flex-wrapper">
{% if 'favicons_enabled' not in ui_settings or ui_settings['favicons_enabled'] %}
<div>{# A page might have hundreds of these images, set IMG options for lazy loading, don't set SRC if we dont have it so it doesnt fetch the placeholder' #}
<img alt="Favicon thumbnail" class="favicon" loading="lazy" decoding="async" fetchpriority="low" {% if favicon %} src="{{url_for('static_content', group='favicon', filename=watch.uuid)}}" {% else %} src='data:image/svg+xml;utf8,%3Csvg xmlns="http://www.w3.org/2000/svg" width="7.087" height="7.087" viewBox="0 0 7.087 7.087"%3E%3Ccircle cx="3.543" cy="3.543" r="3.279" stroke="%23e1e1e1" stroke-width="0.45" fill="none" opacity="0.74"/%3E%3C/svg%3E' {% endif %} >
<img alt="Favicon thumbnail" class="favicon" loading="lazy" decoding="async" fetchpriority="low" {% if favicon %} src="{{url_for('static_content', group='favicon', filename=watch.uuid)}}" {% else %} src='data:image/svg+xml;utf8,%3Csvg xmlns="http://www.w3.org/2000/svg" width="7.087" height="7.087" viewBox="0 0 7.087 7.087"%3E%3Ccircle cx="3.543" cy="3.543" r="3.279" stroke="%23e1e1e1" stroke-width="0.45" fill="none" opacity="0.74"/%3E%3C/svg%3E' {% endif %} />
</div>
{% endif %}
<div>
@@ -218,7 +191,7 @@ html[data-darkmode="true"] .watch-tag-list.tag-{{ class_name }} {
<span class="processor-badge processor-badge-{{ watch['processor'] }}" title="{{ processor_descriptions.get(watch['processor'], watch['processor']) }}">{{ processor_badge_texts[watch['processor']] }}</span>
{%- endif -%}
{%- for watch_tag_uuid, watch_tag in datastore.get_all_tags_for_watch(watch['uuid']).items() -%}
<span class="watch-tag-list tag-{{ watch_tag.title|sanitize_tag_class }}">{{ watch_tag.title }}</span>
<span class="watch-tag-list">{{ watch_tag.title }}</span>
{%- endfor -%}
</div>
<div class="status-icons">
@@ -1,4 +1,3 @@
import gc
import json
import os
from urllib.parse import urlparse
@@ -186,33 +185,20 @@ class fetcher(Fetcher):
super().screenshot_step(step_n=step_n)
screenshot = await capture_full_page_async(page=self.page, screenshot_format=self.screenshot_format)
# Request GC immediately after screenshot to free memory
# Screenshots can be large and browser steps take many of them
await self.page.request_gc()
if self.browser_steps_screenshot_path is not None:
destination = os.path.join(self.browser_steps_screenshot_path, 'step_{}.jpeg'.format(step_n))
logger.debug(f"Saving step screenshot to {destination}")
with open(destination, 'wb') as f:
f.write(screenshot)
# Clear local reference to allow screenshot bytes to be collected
del screenshot
gc.collect()
async def save_step_html(self, step_n):
super().save_step_html(step_n=step_n)
content = await self.page.content()
# Request GC after getting page content
await self.page.request_gc()
destination = os.path.join(self.browser_steps_screenshot_path, 'step_{}.html'.format(step_n))
logger.debug(f"Saving step HTML to {destination}")
with open(destination, 'w', encoding='utf-8') as f:
f.write(content)
# Clear local reference
del content
gc.collect()
async def run(self,
fetch_favicon=True,
@@ -319,12 +305,6 @@ class fetcher(Fetcher):
if self.status_code != 200 and not ignore_status_codes:
screenshot = await capture_full_page_async(self.page, screenshot_format=self.screenshot_format)
# Cleanup before raising to prevent memory leak
await self.page.close()
await context.close()
await browser.close()
# Force garbage collection to release Playwright resources immediately
gc.collect()
raise Non200ErrorCodeReceived(url=url, status_code=self.status_code, screenshot=screenshot)
if not empty_pages_are_a_change and len((await self.page.content()).strip()) == 0:
@@ -333,52 +313,48 @@ class fetcher(Fetcher):
await browser.close()
raise EmptyReply(url=url, status_code=response.status)
# Wrap remaining operations in try/finally to ensure cleanup
# Run Browser Steps here
if self.browser_steps_get_valid_steps():
await self.iterate_browser_steps(start_url=url)
await self.page.wait_for_timeout(extra_wait * 1000)
now = time.time()
# So we can find an element on the page where its selector was entered manually (maybe not xPath etc)
if current_include_filters is not None:
await self.page.evaluate("var include_filters={}".format(json.dumps(current_include_filters)))
else:
await self.page.evaluate("var include_filters=''")
await self.page.request_gc()
# request_gc before and after evaluate to free up memory
# @todo browsersteps etc
MAX_TOTAL_HEIGHT = int(os.getenv("SCREENSHOT_MAX_HEIGHT", SCREENSHOT_MAX_HEIGHT_DEFAULT))
self.xpath_data = await self.page.evaluate(XPATH_ELEMENT_JS, {
"visualselector_xpath_selectors": visualselector_xpath_selectors,
"max_height": MAX_TOTAL_HEIGHT
})
await self.page.request_gc()
self.instock_data = await self.page.evaluate(INSTOCK_DATA_JS)
await self.page.request_gc()
self.content = await self.page.content()
await self.page.request_gc()
logger.debug(f"Scrape xPath element data in browser done in {time.time() - now:.2f}s")
# Bug 3 in Playwright screenshot handling
# Some bug where it gives the wrong screenshot size, but making a request with the clip set first seems to solve it
# JPEG is better here because the screenshots can be very very large
# Screenshots also travel via the ws:// (websocket) meaning that the binary data is base64 encoded
# which will significantly increase the IO size between the server and client, it's recommended to use the lowest
# acceptable screenshot quality here
try:
# Run Browser Steps here
if self.browser_steps_get_valid_steps():
await self.iterate_browser_steps(start_url=url)
await self.page.wait_for_timeout(extra_wait * 1000)
now = time.time()
# So we can find an element on the page where its selector was entered manually (maybe not xPath etc)
if current_include_filters is not None:
await self.page.evaluate("var include_filters={}".format(json.dumps(current_include_filters)))
else:
await self.page.evaluate("var include_filters=''")
await self.page.request_gc()
# request_gc before and after evaluate to free up memory
# @todo browsersteps etc
MAX_TOTAL_HEIGHT = int(os.getenv("SCREENSHOT_MAX_HEIGHT", SCREENSHOT_MAX_HEIGHT_DEFAULT))
self.xpath_data = await self.page.evaluate(XPATH_ELEMENT_JS, {
"visualselector_xpath_selectors": visualselector_xpath_selectors,
"max_height": MAX_TOTAL_HEIGHT
})
await self.page.request_gc()
self.instock_data = await self.page.evaluate(INSTOCK_DATA_JS)
await self.page.request_gc()
self.content = await self.page.content()
await self.page.request_gc()
logger.debug(f"Scrape xPath element data in browser done in {time.time() - now:.2f}s")
# Bug 3 in Playwright screenshot handling
# Some bug where it gives the wrong screenshot size, but making a request with the clip set first seems to solve it
# JPEG is better here because the screenshots can be very very large
# Screenshots also travel via the ws:// (websocket) meaning that the binary data is base64 encoded
# which will significantly increase the IO size between the server and client, it's recommended to use the lowest
# acceptable screenshot quality here
# The actual screenshot - this always base64 and needs decoding! horrible! huge CPU usage
self.screenshot = await capture_full_page_async(page=self.page, screenshot_format=self.screenshot_format)
except ScreenshotUnavailable:
# Re-raise screenshot unavailable exceptions
raise
except Exception as e:
# It's likely the screenshot was too long/big and something crashed
raise ScreenshotUnavailable(url=url, status_code=self.status_code)
@@ -413,10 +389,6 @@ class fetcher(Fetcher):
pass
browser = None
# Force Python GC to release Playwright resources immediately
# Playwright objects can have circular references that delay cleanup
gc.collect()
# Plugin registration for built-in fetcher
class PlaywrightFetcherPlugin:
@@ -15,7 +15,7 @@ class fetcher(Fetcher):
proxy_url = None
# Capability flags
supports_browser_steps = False
supports_browser_steps = True
supports_screenshots = True
supports_xpath_element_data = True
+4 -6
View File
@@ -57,15 +57,14 @@ class SignalPriorityQueue(queue.PriorityQueue):
def put(self, item, block=True, timeout=None):
# Call the parent's put method first
super().put(item, block, timeout)
# After putting the item in the queue, check if it has a UUID and emit signal
if hasattr(item, 'item') and isinstance(item.item, dict) and 'uuid' in item.item:
uuid = item.item['uuid']
# Get the signal and send it if it exists
watch_check_update = signal('watch_check_update')
if watch_check_update:
# NOTE: This would block other workers from .put/.get while this signal sends
# Signal handlers may iterate the queue/datastore while holding locks
# Send the watch_uuid parameter
watch_check_update.send(watch_uuid=uuid)
# Send queue_length signal with current queue size
@@ -313,15 +312,14 @@ class AsyncSignalPriorityQueue(asyncio.PriorityQueue):
async def put(self, item):
# Call the parent's put method first
await super().put(item)
# After putting the item in the queue, check if it has a UUID and emit signal
if hasattr(item, 'item') and isinstance(item.item, dict) and 'uuid' in item.item:
uuid = item.item['uuid']
# Get the signal and send it if it exists
watch_check_update = signal('watch_check_update')
if watch_check_update:
# NOTE: This would block other workers from .put/.get while this signal sends
# Signal handlers may iterate the queue/datastore while holding locks
# Send the watch_uuid parameter
watch_check_update.send(watch_uuid=uuid)
# Send queue_length signal with current queue size
+9 -25
View File
@@ -297,25 +297,6 @@ def _jinja2_filter_fetcher_status_icons(fetcher_name):
return ''
@app.template_filter('sanitize_tag_class')
def _jinja2_filter_sanitize_tag_class(tag_title):
"""Sanitize a tag title to create a valid CSS class name.
Removes all non-alphanumeric characters and converts to lowercase.
Args:
tag_title: The tag title string
Returns:
str: A sanitized string suitable for use as a CSS class name
"""
import re
# Remove all non-alphanumeric characters and convert to lowercase
sanitized = re.sub(r'[^a-zA-Z0-9]', '', tag_title).lower()
# Ensure it starts with a letter (CSS requirement)
if sanitized and not sanitized[0].isalpha():
sanitized = 'tag' + sanitized
return sanitized if sanitized else 'tag'
# Import login_optionally_required from auth_decorator
from changedetectionio.auth_decorator import login_optionally_required
@@ -863,13 +844,13 @@ def changedetection_app(config=None, datastore_o=None):
worker_handler.start_workers(n_workers, update_q, notification_q, app, datastore)
# @todo handle ctrl break
ticker_thread = threading.Thread(target=ticker_thread_check_time_launch_checks, daemon=True, name="TickerThread-ScheduleChecker").start()
threading.Thread(target=notification_runner, daemon=True, name="NotificationRunner").start()
ticker_thread = threading.Thread(target=ticker_thread_check_time_launch_checks).start()
threading.Thread(target=notification_runner).start()
in_pytest = "pytest" in sys.modules or "PYTEST_CURRENT_TEST" in os.environ
# Check for new release version, but not when running in test/build or pytest
if not os.getenv("GITHUB_REF", False) and not strtobool(os.getenv('DISABLE_VERSION_CHECK', 'no')) and not in_pytest:
threading.Thread(target=check_for_new_version, daemon=True, name="VersionChecker").start()
threading.Thread(target=check_for_new_version).start()
# Return the Flask app - the Socket.IO will be attached to it but initialized separately
# This avoids circular dependencies
@@ -914,7 +895,7 @@ def notification_runner():
# At the moment only one thread runs (single runner)
n_object = notification_q.get(block=False)
except queue.Empty:
app.config.exit.wait(1)
time.sleep(1)
else:
@@ -951,7 +932,7 @@ def notification_runner():
app.config['watch_check_update_SIGNAL'].send(app_context=app, watch_uuid=n_object.get('uuid'))
# Process notifications
notification_debug_log+= ["{} - SENDING - {}".format(now.strftime("%c"), json.dumps(sent_obj))]
notification_debug_log+= ["{} - SENDING - {}".format(now.strftime("%Y/%m/%d %H:%M:%S,000"), json.dumps(sent_obj))]
# Trim the log length
notification_debug_log = notification_debug_log[-100:]
@@ -1009,7 +990,7 @@ def ticker_thread_check_time_launch_checks():
# Re #438 - Don't place more watches in the queue to be checked if the queue is already large
while update_q.qsize() >= 2000:
logger.warning(f"Recheck watches queue size limit reached ({MAX_QUEUE_SIZE}), skipping adding more items")
app.config.exit.wait(10.0)
time.sleep(3)
recheck_time_system_seconds = int(datastore.threshold_seconds)
@@ -1107,5 +1088,8 @@ def ticker_thread_check_time_launch_checks():
# Reset for next time
watch.jitter_seconds = 0
# Wait before checking the list again - saves CPU
time.sleep(1)
# Should be low so we can break this out in testing
app.config.exit.wait(1)
+2 -2
View File
@@ -781,8 +781,8 @@ class SingleBrowserStep(Form):
class processor_text_json_diff_form(commonSettingsForm):
url = fields.URLField('Web Page URL', validators=[validateURL()])
tags = StringTagUUID('Group Tag', [validators.Optional()], default='')
url = fields.URLField('URL', validators=[validateURL()])
tags = StringTagUUID('Group tag', [validators.Optional()], default='')
time_between_check = EnhancedFormField(
TimeBetweenCheckForm,
+5 -9
View File
@@ -34,16 +34,13 @@ def get_timeago_locale(flask_locale):
'no': 'nb_NO', # Norwegian Bokmål
'hi': 'in_HI', # Hindi
'cs': 'en', # Czech not supported by timeago, fallback to English
'en_GB': 'en', # British English - timeago uses 'en'
'en_US': 'en', # American English - timeago uses 'en'
}
return locale_map.get(flask_locale, flask_locale)
# Language metadata: flag icon CSS class and native name
# Using flag-icons library: https://flagicons.lipis.dev/
LANGUAGE_DATA = {
'en_GB': {'flag': 'fi fi-gb fis', 'name': 'English (UK)'},
'en_US': {'flag': 'fi fi-us fis', 'name': 'English (US)'},
'en': {'flag': 'fi fi-gb fis', 'name': 'English'},
'de': {'flag': 'fi fi-de fis', 'name': 'Deutsch'},
'fr': {'flag': 'fi fi-fr fis', 'name': 'Français'},
'ko': {'flag': 'fi fi-kr fis', 'name': '한국어'},
@@ -74,7 +71,10 @@ def get_available_languages():
"""
translations_dir = Path(__file__).parent / 'translations'
available = {}
# Always include English as base language
available = {
'en': LANGUAGE_DATA['en']
}
# Scan for translation directories
if translations_dir.exists():
@@ -85,10 +85,6 @@ def get_available_languages():
if po_file.exists():
available[lang_dir.name] = LANGUAGE_DATA[lang_dir.name]
# If no English variants found, fall back to adding en_GB as default
if 'en_GB' not in available and 'en_US' not in available:
available['en_GB'] = LANGUAGE_DATA['en_GB']
return available
+10 -9
View File
@@ -10,13 +10,9 @@ from pathlib import Path
from loguru import logger
from .. import jinja2_custom as safe_jinja
from ..diff import ADDED_PLACEMARKER_OPEN
from ..html_tools import TRANSLATE_WHITESPACE_TABLE
FAVICON_RESAVE_THRESHOLD_SECONDS=86400
BROTLI_COMPRESS_SIZE_THRESHOLD = int(os.getenv('SNAPSHOT_BROTLI_COMPRESSION_THRESHOLD', 1024))
minimum_seconds_recheck_time = int(os.getenv('MINIMUM_SECONDS_RECHECK_TIME', 3))
mtable = {'seconds': 1, 'minutes': 60, 'hours': 3600, 'days': 86400, 'weeks': 86400 * 7}
def _brotli_compress_worker(conn, filepath, mode=None):
"""
@@ -33,7 +29,6 @@ def _brotli_compress_worker(conn, filepath, mode=None):
try:
# Receive data from parent process via pipe (avoids pickle overhead)
contents = conn.recv()
logger.debug(f"Starting brotli compression of {len(contents)} bytes.")
if mode is not None:
compressed_data = brotli.compress(contents, mode=mode)
@@ -45,10 +40,9 @@ def _brotli_compress_worker(conn, filepath, mode=None):
# Send success status back
conn.send(True)
logger.debug(f"Finished brotli compression - From {len(contents)} to {len(compressed_data)} bytes.")
# No need for explicit cleanup - process exit frees all memory
except Exception as e:
logger.critical(f"Brotli compression worker failed: {e}")
logger.error(f"Brotli compression worker failed: {e}")
conn.send(False)
finally:
conn.close()
@@ -72,6 +66,7 @@ def _brotli_subprocess_save(contents, filepath, mode=None, timeout=30, fallback_
Raises:
Exception: if compression fails and fallback_uncompressed is False
"""
import brotli
import multiprocessing
import sys
@@ -149,6 +144,11 @@ def _brotli_subprocess_save(contents, filepath, mode=None, timeout=30, fallback_
else:
raise Exception(f"Brotli compression subprocess failed for {filepath}")
FAVICON_RESAVE_THRESHOLD_SECONDS=86400
minimum_seconds_recheck_time = int(os.getenv('MINIMUM_SECONDS_RECHECK_TIME', 3))
mtable = {'seconds': 1, 'minutes': 60, 'hours': 3600, 'days': 86400, 'weeks': 86400 * 7}
class model(watch_base):
__newest_history_key = None
@@ -492,6 +492,7 @@ class model(watch_base):
self.ensure_data_dir_exists()
threshold = int(os.getenv('SNAPSHOT_BROTLI_COMPRESSION_THRESHOLD', 1024))
skip_brotli = strtobool(os.getenv('DISABLE_BROTLI_TEXT_SNAPSHOT', 'False'))
# Binary data - detect file type and save without compression
@@ -515,7 +516,7 @@ class model(watch_base):
# Text data - use brotli compression if enabled and above threshold
else:
if not skip_brotli and len(contents) > BROTLI_COMPRESS_SIZE_THRESHOLD:
if not skip_brotli and len(contents) > threshold:
# Compressed text
import brotli
snapshot_fname = f"{snapshot_id}.txt.br"
+1 -5
View File
@@ -86,7 +86,6 @@ class RecheckPriorityQueue:
def get(self, block: bool = True, timeout: Optional[float] = None):
"""Thread-safe sync get with priority ordering"""
import queue
try:
# Wait for notification
self.sync_q.get(block=block, timeout=timeout)
@@ -104,11 +103,8 @@ class RecheckPriorityQueue:
logger.debug(f"Successfully retrieved item: {self._get_item_uuid(item)}")
return item
except queue.Empty:
# Queue is empty with timeout - expected behavior, re-raise without logging
raise
except Exception as e:
# Re-raise without logging - caller (worker) will handle and log appropriately
logger.critical(f"CRITICAL: Failed to get item from queue: {str(e)}")
raise
# ASYNC INTERFACE (for workers)
+2 -3
View File
@@ -98,12 +98,11 @@ pytest -vv -s --maxfail=1 tests/test_rss.py
pytest -vv -s --maxfail=1 tests/test_unique_lines.py
# Try high concurrency
FETCH_WORKERS=50 pytest tests/test_history_consistency.py -vv -l -s
FETCH_WORKERS=130 pytest tests/test_history_consistency.py -v -l
# Check file:// will pickup a file when enabled
echo "Hello world" > /tmp/test-file.txt
ALLOW_FILE_URI=yes pytest -vv -s tests/test_security.py
# Run it again so that brotli kicks in
TEST_WITH_BROTLI=1 SNAPSHOT_BROTLI_COMPRESSION_THRESHOLD=100 FETCH_WORKERS=20 pytest tests/test_history_consistency.py -vv -l -s
@@ -3,17 +3,17 @@
* Allows users to select their preferred language
*/
$(document).ready(function() {
const $languageButton = $('.language-selector');
const $languageModal = $('#language-modal');
const $closeButton = $('#close-language-modal');
document.addEventListener('DOMContentLoaded', function() {
const languageButton = document.getElementById('language-selector');
const languageModal = document.getElementById('language-modal');
const closeButton = document.getElementById('close-language-modal');
if (!$languageButton.length || !$languageModal.length) {
if (!languageButton || !languageModal) {
return;
}
// Open modal when language button is clicked
$languageButton.on('click', function(e) {
languageButton.addEventListener('click', function(e) {
e.preventDefault();
// Update all language links to include current hash in the redirect parameter
@@ -21,53 +21,51 @@ $(document).ready(function() {
const currentHash = window.location.hash;
if (currentHash) {
const $languageOptions = $languageModal.find('.language-option');
$languageOptions.each(function() {
const $option = $(this);
const url = new URL($option.attr('href'), window.location.origin);
const languageOptions = languageModal.querySelectorAll('.language-option');
languageOptions.forEach(function(option) {
const url = new URL(option.href, window.location.origin);
// Update the redirect parameter to include the hash
const redirectPath = currentPath + currentHash;
url.searchParams.set('redirect', redirectPath);
$option.attr('href', url.pathname + url.search + url.hash);
option.setAttribute('href', url.pathname + url.search + url.hash);
});
}
$languageModal[0].showModal();
languageModal.showModal();
});
// Close modal when cancel button is clicked
if ($closeButton.length) {
$closeButton.on('click', function() {
$languageModal[0].close();
if (closeButton) {
closeButton.addEventListener('click', function() {
languageModal.close();
});
}
// Close modal when clicking outside (on backdrop)
$languageModal.on('click', function(e) {
const rect = this.getBoundingClientRect();
languageModal.addEventListener('click', function(e) {
const rect = languageModal.getBoundingClientRect();
if (
e.clientY < rect.top ||
e.clientY > rect.bottom ||
e.clientX < rect.left ||
e.clientX > rect.right
) {
$languageModal[0].close();
languageModal.close();
}
});
// Close modal on Escape key
$languageModal.on('cancel', function(e) {
languageModal.addEventListener('cancel', function(e) {
e.preventDefault();
$languageModal[0].close();
languageModal.close();
});
// Highlight current language
const currentLocale = $('html').attr('lang') || 'en';
const $languageOptions = $languageModal.find('.language-option');
$languageOptions.each(function() {
const $option = $(this);
if ($option.attr('data-locale') === currentLocale) {
$option.addClass('active');
const currentLocale = document.documentElement.lang || 'en';
const languageOptions = languageModal.querySelectorAll('.language-option');
languageOptions.forEach(function(option) {
if (option.dataset.locale === currentLocale) {
option.classList.add('active');
}
});
});
+1 -2
View File
@@ -120,8 +120,7 @@ $(document).ready(function () {
console.log(`${data.event_timestamp} - Queue size update: ${data.q_length}`);
// Update queue bubble in action sidebar
//if (queueBubble) {
if (0) {
if (queueBubble) {
const count = parseInt(data.q_length) || 0;
const oldCount = parseInt(queueBubble.getAttribute('data-count')) || 0;
+7 -5
View File
@@ -3,12 +3,14 @@
* Toggles theme between light and dark mode.
*/
$(document).ready(function () {
const button = document.getElementById("toggle-light-mode");
$(".toggle-light-mode").on("click", function () {
const isDark = $("html").attr("data-darkmode") === "true";
$("html").attr("data-darkmode", !isDark);
setCookieValue(!isDark);
});
button.onclick = () => {
const htmlElement = document.getElementsByTagName("html");
const isDarkMode = htmlElement[0].dataset.darkmode === "true";
htmlElement[0].dataset.darkmode = !isDarkMode;
setCookieValue(!isDarkMode);
};
const setCookieValue = (value) => {
document.cookie = `css_dark_mode=${value};max-age=31536000;path=/`
@@ -1,7 +0,0 @@
/**
* SCSS variables (compile-time)
* These can be used in media queries and other places where CSS custom properties don't work
*/
// Breakpoints
$desktop-wide-breakpoint: 980px;
@@ -24,7 +24,7 @@
flex-direction: column;
gap: 0.5rem;
align-items: center;
z-index: 0;
z-index: 10;
@media only screen and (max-width: 900px) {
position: relative;
@@ -32,7 +32,7 @@
width: 100%;
flex-direction: row;
justify-content: space-around;
padding: 0;
padding: 1rem 0.5rem;
overflow-x: auto;
}
}
@@ -1,13 +1,15 @@
.toggle-light-mode {
#toggle-light-mode {
/* width: 3rem;*/
/* default */
.icon-dark {
display: none;
}
}
html[data-darkmode="true"] {
.toggle-light-mode {
#toggle-light-mode {
.icon-light {
display: none;
}
@@ -1,5 +1,4 @@
// Hamburger Menu for Mobile Navigation
@use "../settings" as *;
.hamburger-menu {
display: none;
@@ -10,7 +9,7 @@
z-index: 10001;
position: relative;
@media only screen and (max-width: $desktop-wide-breakpoint) {
@media only screen and (max-width: 768px) {
display: flex;
flex-direction: column;
justify-content: center;
@@ -98,7 +97,7 @@
li {
border-bottom: 1px solid var(--color-border-table-cell);
>* {
a {
display: block;
padding: 1rem 1.5rem;
color: var(--color-text);
@@ -136,9 +135,9 @@
margin-left: auto;
}
// Hide regular menu items on mobile (but not in mobile drawer)
@media only screen and (max-width: $desktop-wide-breakpoint) {
#top-right-menu .menu-collapsible {
// Hide regular menu items on mobile
@media only screen and (max-width: 768px) {
.menu-collapsible {
display: none !important;
}
@@ -152,7 +151,7 @@
}
// Desktop - hide mobile menu elements
@media only screen and (min-width: 1025px) {
@media only screen and (min-width: 769px) {
.hamburger-menu,
.mobile-menu-drawer,
.mobile-menu-overlay {
@@ -1,69 +0,0 @@
#language-selector-flag {
display: inline-block;
width: 1.2em;
height: 1.2em;
vertical-align: middle;
border-radius: 50%;
overflow: hidden;
opacity: 0.6;
&:hover {
opacity: 1.0;
}
}
// Language Selector Modal Styles
.language-list {
display: flex;
flex-direction: column;
gap: 0.5rem;
padding: 0.5rem 0;
}
.language-option {
display: flex;
align-items: center;
gap: 1rem;
padding: 0.25rem;
border-radius: 4px;
transition: background-color 0.2s ease;
text-decoration: none;
color: var(--color-text);
border: 1px solid transparent;
&:hover {
background-color: var(--color-background-menu-link-hover);
border-color: var(--color-border-table-cell);
}
&.active {
background-color: var(--color-link);
color: var(--color-text-button);
font-weight: 600;
}
.flag {
font-size: 1.5rem;
flex-shrink: 0;
}
.language-name {
flex-grow: 1;
font-size: 1rem;
}
}
#language-modal {
.language-list {
.lang-option {
display: inline-block;
width: 1.5em;
height: 1.5em;
vertical-align: middle;
margin-right: 0.5em;
border-radius: 50%;
overflow: hidden;
}
}
}
@@ -20,8 +20,23 @@
overflow: hidden;
transition: transform 0.3s ease, box-shadow 0.3s ease;
&:hover {
// Subtle accent line at top
&::before {
content: '';
position: absolute;
top: 0;
left: 0;
right: 0;
height: 4px;
background: linear-gradient(
90deg,
var(--color-link) 0%,
var(--color-menu-accent) 100%
);
}
&:hover {
transform: translateY(-2px);
box-shadow:
0 15px 50px rgba(0, 0, 0, 0.12),
0 5px 15px rgba(0, 0, 0, 0.06);
@@ -93,6 +108,7 @@
box-shadow: 0 2px 8px rgba(27, 152, 248, 0.2);
&:hover {
transform: translateY(-2px);
box-shadow: 0 4px 12px rgba(27, 152, 248, 0.3);
background: #0066cc;
}
@@ -31,13 +31,3 @@
}
}
}
#cdio-logo {
padding-left: 0.5em;
}
#inline-menu-extras-group {
>* {
display: inline-block;
}
}
@@ -1,57 +0,0 @@
body.wrapped-tabs {
.tabs {
ul {
grid-template-columns: repeat(auto-fill, minmax(var(--tab-width, 180px), 1fr));
grid-auto-flow: row;
grid-auto-columns: unset;
gap: 0;
column-gap: 5px;
}
ul li {
border-radius: 0;
}
}
}
.tabs {
ul {
margin: 0px;
padding: 0px;
display: grid;
grid-auto-flow: column;
grid-auto-columns: max-content;
gap: 5px;
list-style: none;
li {
white-space: nowrap;
color: var(--color-text-tab);
border-top-left-radius: 5px;
border-top-right-radius: 5px;
background-color: var(--color-background-tab);
&:not(.active) {
&:hover {
background-color: var(--color-background-tab-hover);
}
}
&.active,
:target {
background-color: var(--color-background);
a {
color: var(--color-text-tab-active);
font-weight: bold;
}
}
a {
display: block;
padding: 0.7em;
color: var(--color-text-tab);
}
}
}
}
+104 -17
View File
@@ -2,7 +2,6 @@
* -- BASE STYLES --
*/
@use "settings" as *;
@use "parts/variables";
@use "parts/arrows";
@use "parts/browser-steps";
@@ -24,14 +23,12 @@
@use "parts/widgets";
@use "parts/diff_image";
@use "parts/modal";
@use "parts/language";
@use "parts/action_sidebar";
@use "parts/hamburger_menu";
@use "parts/search_modal";
@use "parts/notification_bubble";
@use "parts/toast";
@use "parts/login_form";
@use "parts/tabs";
body {
@@ -160,13 +157,7 @@ body.spinner-active {
}
section.content {
@media only screen and (max-width: $desktop-wide-breakpoint) {
padding-top: 80px;
}
@media only screen and (min-width: $desktop-wide-breakpoint) {
padding-top: 100px;
}
padding-top: 100px;
padding-bottom: 1em;
flex-direction: column;
display: flex;
@@ -184,13 +175,13 @@ code {
border-radius: 5px;
padding: 2px 5px;
margin-right: 4px;
line-height: 1.2rem;
}
/* Processor type badges - colors auto-generated from processor names */
.processor-badge {
@extend .inline-tag;
font-weight: 900;
font-size: 0.85em;
font-weight: 500;
}
.watch-tag-list {
@@ -523,9 +514,6 @@ footer {
}
.sticky-tab {
@media only screen and (max-width: $desktop-wide-breakpoint) {
display: none;
}
position: absolute;
top: 60px;
font-size: 65%;
@@ -670,7 +658,7 @@ footer {
@media only screen and (max-width: 760px),
(min-device-width: 768px) and (max-device-width: $desktop-wide-breakpoint) {
(min-device-width: 768px) and (max-device-width: 1024px) {
.edit-form {
padding: 0.5em;
margin: 0;
@@ -682,10 +670,30 @@ footer {
}
@media only screen and (max-width: 760px), (min-device-width: 768px) and (max-device-width: $desktop-wide-breakpoint) {
@media only screen and (max-width: 760px),
(min-device-width: 768px) and (max-device-width: 800px) {
div.sticky-tab#hosted-sticky {
top: 60px;
left: 0px;
right: auto;
}
section.content {
padding-top: 110px;
}
// Make the tabs easier to hit, they will be all nice and horizontal
div.tabs.collapsable ul li {
display: block;
border-radius: 0px;
margin-right: 0px;
}
input[type='text'] {
width: 100%;
}
}
.pure-table {
@@ -761,6 +769,45 @@ textarea::placeholder {
}
.tabs {
ul {
margin: 0px;
padding: 0px;
display: block;
li {
margin-right: 1px;
display: inline-block;
color: var(--color-text-tab);
border-top-left-radius: 5px;
border-top-right-radius: 5px;
background-color: var(--color-background-tab);
&:not(.active) {
&:hover {
background-color: var(--color-background-tab-hover);
}
}
&.active,
:target {
background-color: var(--color-background);
a {
color: var(--color-text-tab-active);
font-weight: bold;
}
}
a {
display: block;
padding: 0.7em;
color: var(--color-text-tab);
}
}
}
}
$form-edge-padding: 20px;
.pure-form-stacked {
@@ -1097,4 +1144,44 @@ ul#highlightSnippetActions {
}
}
// Language Selector Modal Styles
.language-list {
display: flex;
flex-direction: column;
gap: 0.5rem;
padding: 0.5rem 0;
}
.language-option {
display: flex;
align-items: center;
gap: 1rem;
padding: 0.25rem;
border-radius: 4px;
transition: background-color 0.2s ease;
text-decoration: none;
color: var(--color-text);
border: 1px solid transparent;
&:hover {
background-color: var(--color-background-menu-link-hover);
border-color: var(--color-border-table-cell);
}
&.active {
background-color: var(--color-link);
color: var(--color-text-button);
font-weight: 600;
}
.flag {
font-size: 1.5rem;
flex-shrink: 0;
}
.language-name {
flex-grow: 1;
font-size: 1rem;
}
}
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -186,7 +186,7 @@ class ChangeDetectionStore:
# Finally start the thread that will manage periodic data saves to JSON
# Only start if thread is not already running (reload_state might be called multiple times)
if not self.save_data_thread or not self.save_data_thread.is_alive():
self.save_data_thread = threading.Thread(target=self.save_datastore, daemon=True, name="DatastoreSaver")
self.save_data_thread = threading.Thread(target=self.save_datastore)
self.save_data_thread.start()
def rehydrate_entity(self, uuid, entity, processor_override=None):
-2
View File
@@ -17,8 +17,6 @@ _MAP = {
def strtobool(value):
if not value:
return False
try:
return _MAP[str(value).lower()]
except KeyError:
@@ -145,7 +145,6 @@
<div id="notification-test-log" style="display: none;"><span class="pure-form-message-inline">Processing..</span></div>
</div>
</div>
<div class="pure-control-group grey-form-border">
<div class="pure-control-group">
{{ render_field(form.notification_title, class="m-d notification-title", placeholder=settings_application['notification_title']) }}
@@ -170,7 +169,6 @@
</span></li>
</ul>
<br>
</div>
</div>
<div class="">
{{ render_field(form.notification_format , class="notification-format") }}
+124 -89
View File
@@ -53,10 +53,10 @@
<div class="home-menu pure-menu pure-menu-horizontal" id="nav-menu">
{% if has_password and not current_user.is_authenticated %}
<a id="cdio-logo" class="pure-menu-heading" href="https://changedetection.io" rel="noopener">
<a class="pure-menu-heading" href="https://changedetection.io" rel="noopener">
<strong>Change</strong>Detection.io</a>
{% else %}
<a id="cdio-logo" class="pure-menu-heading" href="{{url_for('watchlist.index')}}">
<a class="pure-menu-heading" href="{{url_for('watchlist.index')}}">
<strong>Change</strong>Detection.io</a>
{% endif %}
{% if current_diff_url and is_safe_valid_url(current_diff_url) %}
@@ -72,19 +72,61 @@
<ul class="pure-menu-list" id="top-right-menu">
<!-- Collapsible menu items (hidden on mobile, shown in drawer) -->
{% include "menu.html" %}
{% if current_user.is_authenticated or not has_password %}
{% if not current_diff_url %}
<li class="pure-menu-item menu-collapsible">
<button class="toggle-button" id="open-search-modal" type="button" title="{{ _('Search, or Use Alt+S Key') }}">
{% include "svgs/search-icon.svg" %}
</button>
<li class="pure-menu-item menu-collapsible {% if request.endpoint.startswith('tags.') %}active{% endif %}">
<a href="{{ url_for('tags.tags_overview_page')}}" class="pure-menu-link">{{ _('GROUPS') }}</a>
</li>
<li class="pure-menu-item menu-collapsible {% if request.endpoint.startswith('settings.') %}active{% endif %}">
<a href="{{ url_for('settings.settings_page')}}" class="pure-menu-link">{{ _('SETTINGS') }}</a>
</li>
<li class="pure-menu-item menu-collapsible {% if request.endpoint.startswith('imports.') %}active{% endif %}">
<a href="{{ url_for('imports.import_page')}}" class="pure-menu-link">{{ _('IMPORT') }}</a>
</li>
<li class="pure-menu-item menu-collapsible {% if request.endpoint.startswith('backups.') %}active{% endif %}">
<a href="{{ url_for('backups.index')}}" class="pure-menu-link">{{ _('BACKUPS') }}</a>
</li>
{% else %}
<li class="pure-menu-item menu-collapsible">
<a href="{{ url_for('ui.ui_edit.edit_page', uuid=uuid, next='diff') }}" class="pure-menu-link">{{ _('EDIT') }}</a>
</li>
{% endif %}
{% else %}
<li class="pure-menu-item menu-collapsible">
<a class="pure-menu-link" href="https://changedetection.io">Website Change Detection and Notification.</a>
</li>
{% endif %}
{% if current_user.is_authenticated %}
<li class="pure-menu-item menu-collapsible">
<a href="{{url_for('logout', redirect=request.path)}}" class="pure-menu-link">{{ _('LOG OUT') }}</a>
</li>
{% endif %}
<!-- Always visible items -->
{% if current_user.is_authenticated or not has_password %}
<li class="pure-menu-item">
<button class="toggle-button" id="open-search-modal" type="button" title="{{ _('Search, or Use Alt+S Key') }}">
{% include "svgs/search-icon.svg" %}
</button>
</li>
{% endif %}
<li class="pure-menu-item">
<button class="toggle-button" id ="toggle-light-mode" type="button" title="{{ _('Toggle Light/Dark Mode') }}">
<span class="visually-hidden">{{ _('Toggle light/dark mode') }}</span>
<span class="icon-light">
{% include "svgs/light-mode-toggle-icon.svg" %}
</span>
<span class="icon-dark">
{% include "svgs/dark-mode-toggle-icon.svg" %}
</span>
</button>
</li>
<li class="pure-menu-item">
<button class="toggle-button" id="language-selector" type="button" title="{{ _('Change Language') }}">
<span class="visually-hidden">{{ _('Change language') }}</span>
<span class="{{ get_flag_for_locale(get_locale()) }}" style="display: inline-block; width: 1.2em; height: 1.2em; vertical-align: middle; border-radius: 50%; overflow: hidden;"></span>
</button>
</li>
<li class="pure-menu-item" id="heart-us">
<svg
fill="#ff0000"
@@ -94,9 +136,16 @@
id="svg-heart"
xmlns="http://www.w3.org/2000/svg"
>
<path id="heartpath" d="M 5.338316,0.50302766 C 0.71136983,0.50647126 -3.9576371,7.2707777 8.5004254,15.503028 23.833425,5.3700277 13.220206,-2.5384409 8.6762066,1.6475589 c -0.060791,0.054322 -0.11943,0.1110064 -0.1757812,0.1699219 -0.057,-0.059 -0.1157813,-0.116875 -0.1757812,-0.171875 C 7.4724566,0.86129334 6.4060729,0.50223298 5.338316,0.50302766 Z" style="fill:var(--color-background);fill-opacity:1;stroke:#ff0000;stroke-opacity:1" />
<path id="heartpath" d="M 5.338316,0.50302766 C 0.71136983,0.50647126 -3.9576371,7.2707777 8.5004254,15.503028 23.833425,5.3700277 13.220206,-2.5384409 8.6762066,1.6475589 c -0.060791,0.054322 -0.11943,0.1110064 -0.1757812,0.1699219 -0.057,-0.059 -0.1157813,-0.116875 -0.1757812,-0.171875 C 7.4724566,0.86129334 6.4060729,0.50223298 5.338316,0.50302766 Z"
style="fill:var(--color-background);fill-opacity:1;stroke:#ff0000;stroke-opacity:1" />
</svg>
</li>
<li class="pure-menu-item">
<a class="github-link" href="https://github.com/dgtlmoon/changedetection.io">
{% include "svgs/github.svg" %}
</a>
</li>
<!-- Hamburger menu button (mobile only) -->
<li class="pure-menu-item">
<button class="hamburger-menu" id="hamburger-toggle" aria-label="Toggle menu">
@@ -114,17 +163,27 @@
<div class="mobile-menu-overlay" id="mobile-menu-overlay"></div>
<div class="mobile-menu-drawer" id="mobile-menu-drawer">
<ul class="mobile-menu-items">
{% include "menu.html" %}
<li class="pure-menu-item menu-collapsible">
{%- if right_sticky -%}<div>{{ right_sticky }}</div>{%- endif -%}
<a href="https://changedetection.io/?ref={{ guid }}">Let us host your instance!</a><br>
</li>
{% if current_user.is_authenticated or not has_password %}
{% if not current_diff_url %}
<li><a href="{{ url_for('tags.tags_overview_page')}}">{{ _('GROUPS') }}</a></li>
<li><a href="{{ url_for('settings.settings_page')}}">{{ _('SETTINGS') }}</a></li>
<li><a href="{{ url_for('imports.import_page')}}">{{ _('IMPORT') }}</a></li>
<li><a href="{{ url_for('backups.index')}}">{{ _('BACKUPS') }}</a></li>
{% else %}
<li><a href="{{ url_for('ui.ui_edit.edit_page', uuid=uuid, next='diff') }}">{{ _('EDIT') }}</a></li>
{% endif %}
{% endif %}
{% if current_user.is_authenticated %}
<li><a href="{{url_for('logout', redirect=request.path)}}">{{ _('LOG OUT') }}</a></li>
{% endif %}
</ul>
</div>
<div id="pure-menu-horizontal-spinner"></div>
</div>
</div>
{% if hosted_sticky %}
<div class="sticky-tab" id="hosted-sticky">
<a href="https://changedetection.io/?ref={{guid}}">Let us host your instance!</a>
@@ -189,7 +248,6 @@
</div>
<div class="content-wrapper">
{#
{% if current_user.is_authenticated or not has_password %}
<aside class="action-sidebar">
<a href="{{ url_for('watchlist.index') }}" class="action-sidebar-item {% if request.endpoint.startswith('watchlist.') or request.endpoint.startswith('ui.') %}active{% endif %}" title="{{ _('Watch List') }}">
@@ -199,6 +257,7 @@
</svg>
<span class="action-label">{{ _('Watches') }}</span>
</a>
<a href="{{ url_for('queue_status') }}" class="action-sidebar-item {% if request.endpoint == 'queue_status' %}active{% endif %}" id="queue-action-item" title="{{ _('Queue Status') }}">
<svg class="action-icon" viewBox="0 0 24 24">
<line x1="8" y1="6" x2="21" y2="6"/>
@@ -211,9 +270,56 @@
<span class="action-label">{{ _('Queue') }}</span>
<span class="notification-bubble blue-bubble" id="queue-bubble" data-count="0"></span>
</a>
<a href="{{ url_for('settings.settings_page') }}" class="action-sidebar-item {% if request.endpoint.startswith('settings.') %}active{% endif %}" title="{{ _('Settings') }}">
<svg class="action-icon" viewBox="0 0 24 24">
<circle cx="12" cy="12" r="3"/>
<path d="M12 1v6m0 6v10M3.5 3.5l4.2 4.2m5.6 5.6l4.2 4.2M1 12h6m6 0h10M3.5 20.5l4.2-4.2m5.6-5.6l4.2-4.2"/>
</svg>
<span class="action-label">{{ _('Settings') }}</span>
</a>
<a href="{{ url_for('backups.index') }}" class="action-sidebar-item {% if request.endpoint.startswith('backups.') %}active{% endif %}" title="{{ _('Backups') }}">
<svg class="action-icon" viewBox="0 0 24 24">
<path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/>
<polyline points="14 2 14 8 20 8"/>
<line x1="16" y1="13" x2="8" y2="13"/>
<line x1="16" y1="17" x2="8" y2="17"/>
<polyline points="10 9 9 9 8 9"/>
</svg>
<span class="action-label">{{ _('Backups') }}</span>
</a>
<a href="#" class="action-sidebar-item" title="{{ _('Sitemap Crawler') }}">
<svg class="action-icon" viewBox="0 0 24 24">
<!-- Spider web with map nodes -->
<circle cx="12" cy="12" r="2"/>
<!-- Radial web lines -->
<line x1="12" y1="12" x2="12" y2="4"/>
<line x1="12" y1="12" x2="19" y2="7"/>
<line x1="12" y1="12" x2="20" y2="12"/>
<line x1="12" y1="12" x2="19" y2="17"/>
<line x1="12" y1="12" x2="12" y2="20"/>
<line x1="12" y1="12" x2="5" y2="17"/>
<line x1="12" y1="12" x2="4" y2="12"/>
<line x1="12" y1="12" x2="5" y2="7"/>
<!-- Outer web ring -->
<circle cx="12" cy="12" r="8" fill="none"/>
<!-- Map nodes on web -->
<circle cx="12" cy="4" r="1.5"/>
<circle cx="19" cy="7" r="1.5"/>
<circle cx="20" cy="12" r="1.5"/>
<circle cx="19" cy="17" r="1.5"/>
<circle cx="12" cy="20" r="1.5"/>
<circle cx="5" cy="17" r="1.5"/>
<circle cx="4" cy="12" r="1.5"/>
<circle cx="5" cy="7" r="1.5"/>
</svg>
<span class="action-label">{{ _('Sitemap') }}</span>
</a>
</aside>
{% endif %}
#}
<div class="content-main">
<header>
{% block header %}{% endblock %}
@@ -261,7 +367,7 @@
<div class="language-list">
{% for locale, lang_data in available_languages.items()|sort %}
<a href="{{ url_for('set_language', locale=locale, redirect=request.path) }}" class="language-option" data-locale="{{ locale }}">
<span class="lang-option {{ lang_data.flag }}"></span> <span class="language-name">{{ lang_data.name }}</span>
<span class="{{ lang_data.flag }}" style="display: inline-block; width: 1.5em; height: 1.5em; vertical-align: middle; margin-right: 0.5em; border-radius: 50%; overflow: hidden;"></span> <span class="language-name">{{ lang_data.name }}</span>
</a>
{% endfor %}
</div>
@@ -296,77 +402,6 @@
</dialog>
{% endif %}
<script>
(function() {
/* AUTOMATIC TAB COLUMN-IZER FOR WHEN TABS WRAP */
// Exit early if no tabs on page
if (!document.querySelector('.tab')) return;
const cache = new Map();
function checkWrapping(ul) {
const tabs = ul.querySelectorAll('.tab');
if (tabs.length < 2) return false;
// Init cache on first run
if (!cache.has(ul)) {
ul.style.setProperty('--tab-width', '');
void ul.offsetHeight;
let max = 0;
tabs.forEach(t => max = Math.max(max, t.offsetWidth));
cache.set(ul, max);
}
// Temporarily use flex wrap to check if wrapping occurs
ul.style.display = 'flex';
ul.style.flexWrap = 'wrap';
void ul.offsetHeight;
const top = tabs[0].offsetTop;
const wrapped = Array.from(tabs).some((t, i) => i > 0 && t.offsetTop !== top);
// Reset display to use CSS grid
ul.style.display = '';
ul.style.flexWrap = '';
// Set CSS variable for wrapped mode
if (wrapped) {
ul.style.setProperty('--tab-width', `${cache.get(ul) + 10}px`);
} else {
ul.style.setProperty('--tab-width', '');
}
return wrapped;
}
function check() {
let any = false;
document.querySelectorAll('ul').forEach(ul => {
if (ul.querySelector('.tab') && checkWrapping(ul)) any = true;
});
document.body.classList.toggle('wrapped-tabs', any);
}
check();
let timer;
window.addEventListener('resize', () => {
clearTimeout(timer);
timer = setTimeout(check, 100);
});
// Re-check wrapping when tabs are switched via anchors
window.addEventListener('hashchange', () => {
clearTimeout(timer);
// Use requestAnimationFrame + setTimeout to ensure DOM has settled
requestAnimationFrame(() => {
timer = setTimeout(check, 0);
});
});
})();
</script>
<script src="{{url_for('static_content', group='js', filename='language-selector.js')}}" defer></script>
<script src="{{url_for('static_content', group='js', filename='search-modal.js')}}" defer></script>
<script src="{{url_for('static_content', group='js', filename='toast.js')}}"></script>
-54
View File
@@ -1,54 +0,0 @@
{# Menu items template - used for both desktop and mobile menus #}
{# CSS media queries handle which version displays - no need for conditional classes #}
{% if current_user.is_authenticated or not has_password %}
{% if not current_diff_url %}
<li class="pure-menu-item menu-collapsible {% if request.endpoint.startswith('tags.') %}active{% endif %}">
<a href="{{ url_for('tags.tags_overview_page') }}" class="pure-menu-link">{{ _('GROUPS') }}</a>
</li>
<li class="pure-menu-item menu-collapsible {% if request.endpoint.startswith('settings.') %}active{% endif %}">
<a href="{{ url_for('settings.settings_page') }}" class="pure-menu-link">{{ _('SETTINGS') }}</a>
</li>
<li class="pure-menu-item menu-collapsible {% if request.endpoint.startswith('imports.') %}active{% endif %}">
<a href="{{ url_for('imports.import_page') }}" class="pure-menu-link">{{ _('IMPORT') }}</a>
</li>
<li class="pure-menu-item menu-collapsible {% if request.endpoint.startswith('backups.') %}active{% endif %}">
<a href="{{ url_for('backups.index') }}" class="pure-menu-link">{{ _('BACKUPS') }}</a>
</li>
{% else %}
<li class="pure-menu-item menu-collapsible">
<a href="{{ url_for('ui.ui_edit.edit_page', uuid=uuid, next='diff') }}"
class="pure-menu-link">{{ _('EDIT') }}</a>
</li>
{% endif %}
{%- if current_user.is_authenticated -%}
<li class="pure-menu-item menu-collapsible">
<a href="{{ url_for('logout', redirect=request.path) }}" class="pure-menu-link">{{ _('LOG OUT') }}</a>
</li>
{%- endif -%}
{% else %}
<li class="pure-menu-item menu-collapsible">
<a class="pure-menu-link" href="https://changedetection.io">Website Change Detection and Notification.</a>
</li>
{% endif %}
<li class="pure-menu-item menu-collapsible" id="inline-menu-extras-group">
<button class="toggle-button toggle-light-mode " type="button" title="{{ _('Toggle Light/Dark Mode') }}">
<span class="visually-hidden">{{ _('Toggle light/dark mode') }}</span>
<span class="icon-light">
{% include "svgs/light-mode-toggle-icon.svg" %}
</span>
<span class="icon-dark">
{% include "svgs/dark-mode-toggle-icon.svg" %}
</span>
</button>
<button class="toggle-button language-selector" type="button" title="{{ _('Change Language') }}">
<span class="visually-hidden">{{ _('Change language') }}</span>
<span class="{{ get_flag_for_locale(get_locale()) }}" id="language-selector-flag"></span>
</button>
<a class="github-link" href="https://github.com/dgtlmoon/changedetection.io">
{% include "svgs/github.svg" %}
</a>
</li>
-3
View File
@@ -270,6 +270,3 @@ def app(request, datastore_path):
request.addfinalizer(teardown)
yield app
+1 -66
View File
@@ -165,83 +165,18 @@ def test_api_simple(client, live_server, measure_memory_usage, datastore_path):
assert b'<div id' in res.data
# Fetch the difference between two versions (default text format)
# Fetch the difference between two versions
res = client.get(
url_for("watchhistorydiff", uuid=watch_uuid, from_timestamp='previous', to_timestamp='latest'),
headers={'x-api-key': api_key},
)
assert b'(changed) Which is across' in res.data
# Test htmlcolor format
res = client.get(
url_for("watchhistorydiff", uuid=watch_uuid, from_timestamp='previous', to_timestamp='latest')+'?format=htmlcolor',
headers={'x-api-key': api_key},
)
assert b'aria-label="Changed text" title="Changed text">Which is across multiple lines' in res.data
# Test html format
res = client.get(
url_for("watchhistorydiff", uuid=watch_uuid, from_timestamp='previous', to_timestamp='latest')+'?format=html',
headers={'x-api-key': api_key},
)
assert res.status_code == 200
assert b'<br>' in res.data
# Test markdown format
res = client.get(
url_for("watchhistorydiff", uuid=watch_uuid, from_timestamp='previous', to_timestamp='latest')+'?format=markdown',
headers={'x-api-key': api_key},
)
assert res.status_code == 200
# Test new diff preference parameters
# Test removed=false (should hide removed content)
res = client.get(
url_for("watchhistorydiff", uuid=watch_uuid, from_timestamp='previous', to_timestamp='latest')+'?removed=false',
headers={'x-api-key': api_key},
)
# Should not contain removed content indicator
assert b'(removed)' not in res.data
# Should still contain added content
assert b'(added)' in res.data or b'which has this one new line' in res.data
# Test added=false (should hide added content)
# Note: The test data has replacements, not pure additions, so we test differently
res = client.get(
url_for("watchhistorydiff", uuid=watch_uuid, from_timestamp='previous', to_timestamp='latest')+'?added=false&replaced=false',
headers={'x-api-key': api_key},
)
# With both added and replaced disabled, should have minimal content
# Should not contain added indicators
assert b'(added)' not in res.data
# Test replaced=false (should hide replaced/changed content)
res = client.get(
url_for("watchhistorydiff", uuid=watch_uuid, from_timestamp='previous', to_timestamp='latest')+'?replaced=false',
headers={'x-api-key': api_key},
)
# Should not contain changed content indicator
assert b'(changed)' not in res.data
# Test type=diffWords for word-level diff
res = client.get(
url_for("watchhistorydiff", uuid=watch_uuid, from_timestamp='previous', to_timestamp='latest')+'?type=diffWords&format=htmlcolor',
headers={'x-api-key': api_key},
)
# Should contain HTML formatted diff
assert res.status_code == 200
assert len(res.data) > 0
# Test combined parameters: show only additions with word diff
res = client.get(
url_for("watchhistorydiff", uuid=watch_uuid, from_timestamp='previous', to_timestamp='latest')+'?removed=false&replaced=false&type=diffWords',
headers={'x-api-key': api_key},
)
assert res.status_code == 200
# Should not contain removed or changed markers
assert b'(removed)' not in res.data
assert b'(changed)' not in res.data
# Fetch the whole watch
res = client.get(
@@ -206,10 +206,11 @@ def test_regex_error_handling(client, live_server, measure_memory_usage, datasto
# Add our URL to the import page
test_url = url_for('test_endpoint', _external=True)
uuid = client.application.config.get('DATASTORE').add_watch(url=test_url)
time.sleep(0.2)
client.get(url_for("ui.form_watch_checknow"), follow_redirects=True)
### test regex error handling
res = client.post(
url_for("ui.ui_edit.edit_page", uuid=uuid),
url_for("ui.ui_edit.edit_page", uuid="first"),
data={"extract_text": '/something bad\d{3/XYZ',
"url": test_url,
"fetch_backend": "html_requests",
@@ -4,47 +4,25 @@ import time
import os
import json
from flask import url_for
from loguru import logger
from .. import strtobool
from .util import wait_for_all_checks, delete_all_watches
import brotli
from urllib.parse import urlparse, parse_qs
def test_consistent_history(client, live_server, measure_memory_usage, datastore_path):
# live_server_setup(live_server) # Setup on conftest per function
workers = int(os.getenv("FETCH_WORKERS", 10))
r = range(1, 10+workers)
uuids = set()
sys_fetch_workers = int(os.getenv("FETCH_WORKERS", 10))
workers = range(1, sys_fetch_workers)
now = time.time()
for one in r:
test_url = url_for('test_endpoint', content_type="text/html", content=str(one), _external=True)
res = client.post(
url_for("imports.import_page"),
data={"urls": test_url},
follow_redirects=True
)
for one in workers:
if strtobool(os.getenv("TEST_WITH_BROTLI")):
# A very long string that WILL trigger Brotli compression of the snapshot
# BROTLI_COMPRESS_SIZE_THRESHOLD should be set to say 200
from ..model.Watch import BROTLI_COMPRESS_SIZE_THRESHOLD
content = str(one) + "x" + str(one) * (BROTLI_COMPRESS_SIZE_THRESHOLD + 10)
else:
# Just enough to test datastore
content = str(one)+'x'
test_url = url_for('test_endpoint', content_type="text/html", content=content, _external=True)
uuids.add(client.application.config.get('DATASTORE').add_watch(url=test_url, extras={'title': str(one)}))
client.get(url_for("ui.form_watch_checknow"), follow_redirects=True)
assert b"1 Imported" in res.data
wait_for_all_checks(client)
duration = time.time() - now
per_worker = duration/sys_fetch_workers
if sys_fetch_workers < 20:
per_worker_threshold=0.6
elif sys_fetch_workers < 50:
per_worker_threshold = 0.8
else:
per_worker_threshold = 1.5
logger.debug(f"All fetched in {duration:.2f}s, {per_worker}s per worker")
# Problematic on github
#assert per_worker < per_worker_threshold, f"If concurrency is working good, no blocking async problems, each worker ({sys_fetch_workers} workers) should have done his job in under {per_worker_threshold}s, got {per_worker:.2f}s per worker, total duration was {duration:.2f}s"
# Essentially just triggers the DB write/update
res = client.post(
@@ -56,7 +34,7 @@ def test_consistent_history(client, live_server, measure_memory_usage, datastore
)
assert b"Settings updated." in res.data
# Wait for the sync DB save to happen
time.sleep(2)
json_db_file = os.path.join(live_server.app.config['DATASTORE'].datastore_path, 'url-watches.json')
@@ -66,18 +44,14 @@ def test_consistent_history(client, live_server, measure_memory_usage, datastore
json_obj = json.load(f)
# assert the right amount of watches was found in the JSON
assert len(json_obj['watching']) == len(workers), "Correct number of watches was found in the JSON"
i = 0
assert len(json_obj['watching']) == len(r), "Correct number of watches was found in the JSON"
i=0
# each one should have a history.txt containing just one line
for w in json_obj['watching'].keys():
i += 1
i+=1
history_txt_index_file = os.path.join(live_server.app.config['DATASTORE'].datastore_path, w, 'history.txt')
assert os.path.isfile(history_txt_index_file), f"History.txt should exist where I expect it at {history_txt_index_file}"
# Should be no errors (could be from brotli etc)
assert not live_server.app.config['DATASTORE'].data['watching'][w].get('last_error')
# Same like in model.Watch
with open(history_txt_index_file, "r") as f:
tmp_history = dict(i.strip().split(',', 2) for i in f.readlines())
@@ -89,21 +63,15 @@ def test_consistent_history(client, live_server, measure_memory_usage, datastore
# Find the snapshot one
for fname in files_in_watch_dir:
if fname != 'history.txt' and 'html' not in fname:
if strtobool(os.getenv("TEST_WITH_BROTLI")):
assert fname.endswith('.br'), "Forced TEST_WITH_BROTLI then it should be a .br filename"
full_snapshot_history_path = os.path.join(live_server.app.config['DATASTORE'].datastore_path, w, fname)
# contents should match what we requested as content returned from the test url
if fname.endswith('.br'):
with open(full_snapshot_history_path, 'rb') as f:
contents = brotli.decompress(f.read()).decode('utf-8')
else:
with open(full_snapshot_history_path, 'r') as snapshot_f:
contents = snapshot_f.read()
with open(os.path.join(live_server.app.config['DATASTORE'].datastore_path, w, fname), 'r') as snapshot_f:
contents = snapshot_f.read()
watch_url = json_obj['watching'][w]['url']
u = urlparse(watch_url)
q = parse_qs(u[4])
assert q['content'][0] == contents.strip(), f"Snapshot file {fname} should contain {q['content'][0]}"
watch_title = json_obj['watching'][w]['title']
assert json_obj['watching'][w]['title'], "Watch should have a title set"
assert contents.startswith(watch_title + "x"), f"Snapshot contents in file {fname} should start with '{watch_title}x', got '{contents}'"
assert len(files_in_watch_dir) == 3, "Should be just three files in the dir, html.br snapshot, history.txt and the extracted text snapshot"
+6 -7
View File
@@ -25,13 +25,12 @@ def test_content_filter_live_preview(client, live_server, measure_memory_usage,
test_url = url_for('test_endpoint', _external=True)
uuid = client.application.config.get('DATASTORE').add_watch(url=test_url)
res = client.get(url_for("ui.form_watch_checknow"), follow_redirects=True)
assert b'Queued 1 watch for rechecking.' in res.data
wait_for_all_checks(client)
res = client.post(
url_for("ui.ui_views.form_quick_watch_add"),
data={"url": test_url, "tags": ''},
follow_redirects=True
)
uuid = next(iter(live_server.app.config['DATASTORE'].data['watching']))
res = client.post(
url_for("ui.ui_edit.edit_page", uuid=uuid),
data={
+10 -14
View File
@@ -5,7 +5,7 @@ import re
from flask import url_for
from loguru import logger
from .util import set_original_response, set_modified_response, set_more_modified_response, live_server_setup, wait_for_all_checks, wait_for_notification_endpoint_output
from .util import set_original_response, set_modified_response, set_more_modified_response, live_server_setup, wait_for_all_checks
from . util import extract_UUID_from_client
import logging
import base64
@@ -83,9 +83,7 @@ def test_check_notification(client, live_server, measure_memory_usage, datastore
uuid = next(iter(live_server.app.config['DATASTORE'].data['watching']))
screenshot_dir = os.path.join(datastore_path, str(uuid))
os.makedirs(screenshot_dir, exist_ok=True)
with open(os.path.join(screenshot_dir, 'last-screenshot.png'), 'wb') as f:
with open(os.path.join(datastore_path, str(uuid), 'last-screenshot.png'), 'wb') as f:
f.write(base64.b64decode(testimage_png))
# Goto the edit page, add our ignore text
@@ -144,7 +142,7 @@ def test_check_notification(client, live_server, measure_memory_usage, datastore
# Trigger a check
client.get(url_for("ui.form_watch_checknow"), follow_redirects=True)
wait_for_all_checks(client)
wait_for_notification_endpoint_output(datastore_path=datastore_path)
time.sleep(6)
# Check no errors were recorded
res = client.get(url_for("watchlist.index"))
@@ -201,7 +199,7 @@ def test_check_notification(client, live_server, measure_memory_usage, datastore
set_more_modified_response(datastore_path=datastore_path)
client.get(url_for("ui.form_watch_checknow"), follow_redirects=True)
wait_for_all_checks(client)
wait_for_notification_endpoint_output(datastore_path=datastore_path)
time.sleep(6)
# Verify what was sent as a notification, this file should exist
with open(os.path.join(datastore_path, "notification.txt"), "r") as f:
notification_submission = f.read()
@@ -242,8 +240,7 @@ def test_check_notification(client, live_server, measure_memory_usage, datastore
)
assert b"Updated watch." in res.data
wait_for_all_checks(client)
wait_for_notification_endpoint_output(datastore_path=datastore_path)
time.sleep(2)
# Verify what was sent as a notification, this file should exist
with open(os.path.join(datastore_path, "notification.txt"), "r") as f:
@@ -328,7 +325,7 @@ def test_notification_custom_endpoint_and_jinja2(client, live_server, measure_me
client.get(url_for("ui.form_watch_checknow"), follow_redirects=True)
wait_for_all_checks(client)
wait_for_notification_endpoint_output(datastore_path=datastore_path)
time.sleep(2) # plus extra delay for notifications to fire
# Check no errors were recorded, because we asked for 204 which is slightly uncommon but is still OK
@@ -446,7 +443,7 @@ def test_global_send_test_notification(client, live_server, measure_memory_usage
assert res.status_code != 500
# Give apprise time to fire
wait_for_notification_endpoint_output(datastore_path=datastore_path)
time.sleep(4)
with open(os.path.join(datastore_path, "notification.txt"), 'r') as f:
x = f.read()
@@ -503,7 +500,7 @@ def test_single_send_test_notification_on_watch(client, live_server, measure_mem
test_notification_url = url_for('test_notification_endpoint', _external=True).replace('http://', 'post://')+"?xxx={{ watch_url }}&+custom-header=123"
# 1995 UTF-8 content should be encoded
test_body = 'change detection is cool 网站监测 内容更新了 - {{diff_full}}\n\nCurrent snapshot: {{current_snapshot}}'
test_body = 'change detection is cool 网站监测 内容更新了 - {{diff_full}}'
######### Test global/system settings
res = client.post(
url_for("ui.ui_notification.ajax_callback_send_notification_test")+f"/{uuid}",
@@ -528,8 +525,7 @@ def test_single_send_test_notification_on_watch(client, live_server, measure_mem
assert 'title="Changed into">Example text:' not in x
assert 'span' not in x
assert 'Example text:' in x
#3720 current_snapshot check, was working but lets test it exactly.
assert 'Current snapshot: Example text: example test' in x
os.unlink(os.path.join(datastore_path, "notification.txt"))
def _test_color_notifications(client, notification_body_token, datastore_path):
@@ -576,7 +572,7 @@ def _test_color_notifications(client, notification_body_token, datastore_path):
assert b'Queued 1 watch for rechecking.' in res.data
wait_for_all_checks(client)
wait_for_notification_endpoint_output(datastore_path=datastore_path)
time.sleep(2)
with open(os.path.join(datastore_path, "notification.txt"), 'r') as f:
x = f.read()
+1 -9
View File
@@ -2,7 +2,7 @@
import time
from flask import url_for
from .util import live_server_setup, wait_for_all_checks, wait_for_watch_history, extract_rss_token_from_UI, get_UUID_for_tag_name, delete_all_watches
from .util import live_server_setup, wait_for_all_checks, extract_rss_token_from_UI, get_UUID_for_tag_name, delete_all_watches
import os
@@ -87,9 +87,6 @@ def test_rss_group(client, live_server, measure_memory_usage, datastore_path):
# Wait for initial checks to complete
wait_for_all_checks(client)
# Ensure initial snapshots are saved
assert wait_for_watch_history(client, min_history_count=1, timeout=10), "Watches did not save initial snapshots"
# Trigger a change
set_modified_response(datastore_path=datastore_path)
@@ -97,9 +94,6 @@ def test_rss_group(client, live_server, measure_memory_usage, datastore_path):
res = client.get(url_for("ui.form_watch_checknow"), follow_redirects=True)
wait_for_all_checks(client)
# Ensure all watches have sufficient history for RSS generation
assert wait_for_watch_history(client, min_history_count=2, timeout=10), "Watches did not accumulate sufficient history"
# Get RSS token
rss_token = extract_rss_token_from_UI(client)
assert rss_token is not None
@@ -222,13 +216,11 @@ def test_rss_group_only_unviewed(client, live_server, measure_memory_usage, data
assert b"Watch added" in res.data
wait_for_all_checks(client)
assert wait_for_watch_history(client, min_history_count=1, timeout=10), "Initial snapshots not saved"
# Trigger changes
set_modified_response(datastore_path=datastore_path)
res = client.get(url_for("ui.form_watch_checknow"), follow_redirects=True)
wait_for_all_checks(client)
assert wait_for_watch_history(client, min_history_count=2, timeout=10), "History not accumulated"
# Get RSS token
rss_token = extract_rss_token_from_UI(client)
@@ -1,10 +1,8 @@
import sys
import os
import pytest
from changedetectionio import html_tools
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import html_tools
# test generation guide.
# 1. Do not include encoding in the xml declaration if the test object is a str type.
+2 -35
View File
@@ -164,45 +164,14 @@ def wait_for_all_checks(client=None):
if q_length == 0 and not any_workers_busy:
if empty_since is None:
empty_since = time.time()
# Brief stabilization period for async workers
elif time.time() - empty_since >= 0.3:
elif time.time() - empty_since >= 0.15: # Shorter wait
break
else:
empty_since = None
attempt += 1
time.sleep(0.3)
def wait_for_watch_history(client, min_history_count=2, timeout=10):
"""
Wait for watches to have sufficient history entries.
Useful after wait_for_all_checks() when you need to ensure history is populated.
Args:
client: Test client with access to datastore
min_history_count: Minimum number of history entries required
timeout: Maximum time to wait in seconds
"""
datastore = client.application.config.get('DATASTORE')
start_time = time.time()
while time.time() - start_time < timeout:
all_have_history = True
for uuid, watch in datastore.data['watching'].items():
history_count = len(watch.history.keys())
if history_count < min_history_count:
all_have_history = False
break
if all_have_history:
return True
time.sleep(0.2)
# Timeout - return False
return False
# Replaced by new_live_server_setup and calling per function scope in conftest.py
def live_server_setup(live_server):
return True
@@ -220,8 +189,6 @@ def new_live_server_setup(live_server):
@live_server.app.route('/test-endpoint')
def test_endpoint():
from loguru import logger
logger.debug(f"/test-endpoint hit {request}")
ctype = request.args.get('content_type')
status_code = request.args.get('status_code')
content = request.args.get('content') or None
@@ -144,6 +144,7 @@ def test_basic_browserstep(client, live_server, measure_memory_usage, datastore_
def test_non_200_errors_report_browsersteps(client, live_server, measure_memory_usage, datastore_path):
four_o_four_url = url_for('test_endpoint', status_code=404, _external=True)
four_o_four_url = four_o_four_url.replace('localhost.localdomain', 'cdio')
four_o_four_url = four_o_four_url.replace('localhost', 'cdio')
@@ -185,65 +186,3 @@ def test_non_200_errors_report_browsersteps(client, live_server, measure_memory_
url_for("ui.form_delete", uuid="all"),
follow_redirects=True
)
def test_browsersteps_edit_UI_startsession(client, live_server, measure_memory_usage, datastore_path):
assert os.getenv('PLAYWRIGHT_DRIVER_URL'), "Needs PLAYWRIGHT_DRIVER_URL set for this test"
# Add a watch first
test_url = url_for('test_interactive_html_endpoint', _external=True)
test_url = test_url.replace('localhost.localdomain', 'cdio')
test_url = test_url.replace('localhost', 'cdio')
uuid = client.application.config.get('DATASTORE').add_watch(url=test_url, extras={'fetch_backend': 'html_webdriver', 'paused': True})
# Test starting a browsersteps session
res = client.get(
url_for("browser_steps.browsersteps_start_session", uuid=uuid),
follow_redirects=True
)
assert res.status_code == 200
assert res.is_json
json_data = res.get_json()
assert 'browsersteps_session_id' in json_data
assert json_data['browsersteps_session_id'] # Not empty
browsersteps_session_id = json_data['browsersteps_session_id']
# Verify the session exists in browsersteps_sessions
from changedetectionio.blueprint.browser_steps import browsersteps_sessions, browsersteps_watch_to_session
assert browsersteps_session_id in browsersteps_sessions
assert uuid in browsersteps_watch_to_session
assert browsersteps_watch_to_session[uuid] == browsersteps_session_id
# Verify browsersteps UI shows up on edit page
res = client.get(url_for("ui.ui_edit.edit_page", uuid=uuid))
assert b'browsersteps-click-start' in res.data, "Browsersteps manual UI shows up"
# Session should still exist after GET (not cleaned up yet)
assert browsersteps_session_id in browsersteps_sessions
assert uuid in browsersteps_watch_to_session
# Test cleanup happens on save (POST)
res = client.post(
url_for("ui.ui_edit.edit_page", uuid=uuid),
data={
"url": test_url,
"tags": "",
'fetch_backend': "html_webdriver",
"time_between_check_use_default": "y",
},
follow_redirects=True
)
assert b"Updated watch" in res.data
# NOW verify the session was cleaned up after save
assert browsersteps_session_id not in browsersteps_sessions
assert uuid not in browsersteps_watch_to_session
# Cleanup
client.get(
url_for("ui.form_delete", uuid="all"),
follow_redirects=True
)
@@ -170,7 +170,7 @@ msgstr "Neplatná hodnota."
#: changedetectionio/forms.py:732
msgid "Watch"
msgstr "Monitorovat"
msgstr "# monitory"
#: changedetectionio/forms.py:733 changedetectionio/forms.py:766
msgid "Processor"
@@ -178,7 +178,7 @@ msgstr "Procesor"
#: changedetectionio/forms.py:734
msgid "Edit > Watch"
msgstr "Upravit > Monitorovat"
msgstr "Nejprve upravte a poté sledujte"
#: changedetectionio/forms.py:747 changedetectionio/forms.py:994
msgid "Fetch Method"
@@ -344,7 +344,7 @@ msgstr "Odeslat upozornění, když filtr již na stránce nelze najít"
#: changedetectionio/blueprint/ui/templates/edit.html:59
#: changedetectionio/forms.py:832
msgid "Notifications"
msgstr "Oznámení"
msgstr "Žádné informace"
#: changedetectionio/forms.py:832
msgid "Muted"
@@ -396,7 +396,7 @@ msgstr ""
#: changedetectionio/forms.py:920 changedetectionio/forms.py:932
msgid "Name"
msgstr "Název"
msgstr "Zrušit ztlumení"
#: changedetectionio/forms.py:921
msgid "Proxy URL"
@@ -420,7 +420,7 @@ msgstr "Požadavky na prostý text"
#: changedetectionio/forms.py:946
msgid "Chrome requests"
msgstr "Chrome požadavky"
msgstr "Žádost"
#: changedetectionio/forms.py:952
msgid "Default proxy"
@@ -476,7 +476,7 @@ msgstr "Kontrola zabezpečení přístupového tokenu API povolena"
#: changedetectionio/forms.py:989
msgid "Notification base URL override"
msgstr "Základní URL pro upozornění"
msgstr "Počet upozornění na upozornění"
#: changedetectionio/forms.py:993
msgid "Treat empty pages as a change?"
@@ -608,11 +608,11 @@ msgstr "No backups found."
#: changedetectionio/blueprint/backups/templates/overview.html:28
msgid "Create backup"
msgstr "Vytvořit zálohu"
msgstr "Create backup"
#: changedetectionio/blueprint/backups/templates/overview.html:30
msgid "Remove backups"
msgstr "Odstranit zálohy"
msgstr "Remove backups"
#: changedetectionio/blueprint/imports/importer.py:45
msgid ""
@@ -823,7 +823,7 @@ msgstr "Generál"
#: changedetectionio/blueprint/settings/templates/settings.html:23
msgid "Fetching"
msgstr "Načítání"
msgstr "Hledání"
#: changedetectionio/blueprint/settings/templates/settings.html:24
msgid "Global Filters"
@@ -851,7 +851,7 @@ msgstr "CAPTCHA a proxy"
#: changedetectionio/blueprint/settings/templates/settings.html:35
msgid "Info"
msgstr "Info"
msgstr "Více informací"
#: changedetectionio/blueprint/settings/templates/settings.html:46
msgid "Default recheck time for all watches, current system minimum is"
@@ -1004,7 +1004,7 @@ msgstr ""
#: changedetectionio/blueprint/tags/templates/groups-overview.html:31
msgid "# Watches"
msgstr "# monitorů"
msgstr "# monitory"
#: changedetectionio/blueprint/tags/templates/groups-overview.html:32
msgid "Tag / Label name"
@@ -1251,7 +1251,7 @@ msgstr "nejprve odkaz."
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:17
msgid "Confirmation text"
msgstr "Potvrzovací text"
msgstr "Žádné informace"
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:27
msgid "Type in the word"
@@ -1456,7 +1456,7 @@ msgstr "Podmínky"
#: changedetectionio/blueprint/ui/templates/edit.html:60
msgid "Stats"
msgstr "Statistiky"
msgstr "NASTAVENÍ"
#: changedetectionio/blueprint/ui/templates/edit.html:73
#: changedetectionio/blueprint/ui/templates/edit.html:313
@@ -1895,11 +1895,11 @@ msgstr "Přidejte nové monitory zjišťování změn webové stránky"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:34
msgid "Watch this URL!"
msgstr "Monitorovat tuto URL!"
msgstr "Sledujte tuto adresu URL!"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:35
msgid "Edit first then Watch"
msgstr "Upravit a monitorovat"
msgstr "Nejprve upravte a poté sledujte"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:45
msgid "Create a shareable link"
@@ -2011,7 +2011,9 @@ msgstr "Změněno"
msgid "No website watches configured, please add a URL in the box above, or"
msgstr ""
"Nejsou nakonfigurována žádná sledování webových stránek, do výše "
"uvedeného pole přidejte adresu URL nebo"
"uvedeného pole přidejte adresu URL neboNejsou nakonfigurována žádná "
"sledování webových stránek, do výše uvedeného pole přidejte adresu URL "
"nebo"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:130
msgid "import a list"
@@ -2048,7 +2050,7 @@ msgstr "Ve frontě"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:250
msgid "History"
msgstr "Historie"
msgstr "Dějiny"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:251
msgid "Preview"
@@ -2399,11 +2401,11 @@ msgstr "Změnit jazyk"
#: changedetectionio/templates/base.html:253
msgid "Watch List"
msgstr "Seznam monitorů"
msgstr "# monitory"
#: changedetectionio/templates/base.html:258
msgid "Watches"
msgstr "Monitory"
msgstr "# monitory"
#: changedetectionio/templates/base.html:261
msgid "Queue Status"
@@ -172,7 +172,7 @@ msgstr "Ungültiger Wert."
#: changedetectionio/forms.py:732
msgid "Watch"
msgstr "Monitor"
msgstr "Watch"
#: changedetectionio/forms.py:733 changedetectionio/forms.py:766
msgid "Processor"
@@ -180,7 +180,7 @@ msgstr "Prozessor"
#: changedetectionio/forms.py:734
msgid "Edit > Watch"
msgstr "Bearbeiten > Monitor"
msgstr "Edit > Watch"
#: changedetectionio/forms.py:747 changedetectionio/forms.py:994
msgid "Fetch Method"
@@ -337,7 +337,7 @@ msgstr "Speichern"
#: changedetectionio/forms.py:829
msgid "Proxy"
msgstr "Proxy"
msgstr "Stellvertreter"
#: changedetectionio/forms.py:831
msgid "Send a notification when the filter can no longer be found on the page"
@@ -350,7 +350,7 @@ msgstr ""
#: changedetectionio/blueprint/ui/templates/edit.html:59
#: changedetectionio/forms.py:832
msgid "Notifications"
msgstr "Benachrichtigungen"
msgstr "Keine Informationen"
#: changedetectionio/forms.py:832
msgid "Muted"
@@ -404,7 +404,7 @@ msgstr ""
#: changedetectionio/forms.py:920 changedetectionio/forms.py:932
msgid "Name"
msgstr "Name"
msgstr "Stummschaltung aufheben"
#: changedetectionio/forms.py:921
msgid "Proxy URL"
@@ -428,7 +428,7 @@ msgstr "Klartextanfragen"
#: changedetectionio/forms.py:946
msgid "Chrome requests"
msgstr "Chrome Requests"
msgstr "Anfrage"
#: changedetectionio/forms.py:952
msgid "Default proxy"
@@ -484,7 +484,7 @@ msgstr "Sicherheitsüberprüfung des API-Zugriffstokens aktiviert"
#: changedetectionio/forms.py:989
msgid "Notification base URL override"
msgstr "Basis-URL für Benachrichtigungen"
msgstr "Anzahl der Benachrichtigungsalarme"
#: changedetectionio/forms.py:993
msgid "Treat empty pages as a change?"
@@ -620,11 +620,11 @@ msgstr "Keine Backups gefunden."
#: changedetectionio/blueprint/backups/templates/overview.html:28
msgid "Create backup"
msgstr "Backup erstellen"
msgstr "BACKUPS"
#: changedetectionio/blueprint/backups/templates/overview.html:30
msgid "Remove backups"
msgstr "Backups entfernen"
msgstr "BACKUPS"
#: changedetectionio/blueprint/imports/importer.py:45
msgid ""
@@ -815,7 +815,7 @@ msgstr ""
#: changedetectionio/blueprint/settings/__init__.py:126
msgid "Settings updated."
msgstr "Einstellungen aktualisiert."
msgstr "EINSTELLUNGEN"
#: changedetectionio/blueprint/settings/__init__.py:129
#: changedetectionio/blueprint/ui/edit.py:283
@@ -839,7 +839,7 @@ msgstr "Allgemein"
#: changedetectionio/blueprint/settings/templates/settings.html:23
msgid "Fetching"
msgstr "Abrufen"
msgstr "Suchen"
#: changedetectionio/blueprint/settings/templates/settings.html:24
msgid "Global Filters"
@@ -867,7 +867,7 @@ msgstr "CAPTCHA & Proxys"
#: changedetectionio/blueprint/settings/templates/settings.html:35
msgid "Info"
msgstr "Info"
msgstr "Weitere Informationen"
#: changedetectionio/blueprint/settings/templates/settings.html:46
msgid "Default recheck time for all watches, current system minimum is"
@@ -897,7 +897,7 @@ msgstr "Keine Plugins aktiv"
#: changedetectionio/blueprint/settings/templates/settings.html:405
msgid "Back"
msgstr "Zurück"
msgstr "BACKUPS"
#: changedetectionio/blueprint/settings/templates/settings.html:406
msgid "Clear Snapshot History"
@@ -942,7 +942,7 @@ msgstr "Filter und Trigger"
#: changedetectionio/blueprint/tags/templates/edit-tag.html:50
msgid "These settings are"
msgstr "Diese Einstellungen sind"
msgstr "EINSTELLUNGEN"
#: changedetectionio/blueprint/tags/templates/edit-tag.html:50
msgid "added"
@@ -1271,7 +1271,7 @@ msgstr "Link zuerst."
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:17
msgid "Confirmation text"
msgstr "Bestätigungstext"
msgstr "Keine Informationen"
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:27
msgid "Type in the word"
@@ -1420,7 +1420,7 @@ msgstr ""
#: changedetectionio/blueprint/ui/templates/diff.html:127
msgid "from settings."
msgstr "aus den Einstellungen."
msgstr "EINSTELLUNGEN"
#: changedetectionio/blueprint/ui/templates/diff.html:133
msgid "Goto single snapshot"
@@ -1478,7 +1478,7 @@ msgstr "Bedingungen"
#: changedetectionio/blueprint/ui/templates/edit.html:60
msgid "Stats"
msgstr "Statistiken"
msgstr "EINSTELLUNGEN"
#: changedetectionio/blueprint/ui/templates/edit.html:73
#: changedetectionio/blueprint/ui/templates/edit.html:313
@@ -1931,11 +1931,11 @@ msgstr ""
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:34
msgid "Watch this URL!"
msgstr "Diese URL überwachen!"
msgstr "Sehen Sie sich diese URL an!"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:35
msgid "Edit first then Watch"
msgstr "Bearbeiten > Überwachen"
msgstr "Zuerst bearbeiten, dann ansehen"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:45
msgid "Create a shareable link"
@@ -2396,7 +2396,7 @@ msgstr "EINSTELLUNGEN"
#: changedetectionio/templates/base.html:84
#: changedetectionio/templates/base.html:170
msgid "IMPORT"
msgstr "IMPORTIEREN"
msgstr "IMPORT"
#: changedetectionio/templates/base.html:87
#: changedetectionio/templates/base.html:171
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -181,7 +181,7 @@ msgstr "Processeur"
#: changedetectionio/forms.py:734
msgid "Edit > Watch"
msgstr "Modifier > Surveiller"
msgstr "Modifiez d'abord, puis regardez"
#: changedetectionio/forms.py:747 changedetectionio/forms.py:994
msgid "Fetch Method"
@@ -342,7 +342,7 @@ msgstr "Sauvegarder"
#: changedetectionio/forms.py:829
msgid "Proxy"
msgstr "Proxy"
msgstr "Procuration"
#: changedetectionio/forms.py:831
msgid "Send a notification when the filter can no longer be found on the page"
@@ -353,7 +353,7 @@ msgstr "Envoyer une notification lorsque le filtre n'est plus trouvé sur la pag
#: changedetectionio/blueprint/ui/templates/edit.html:59
#: changedetectionio/forms.py:832
msgid "Notifications"
msgstr "Notifications"
msgstr "Aucune information"
#: changedetectionio/forms.py:832
msgid "Muted"
@@ -405,7 +405,7 @@ msgstr ""
#: changedetectionio/forms.py:920 changedetectionio/forms.py:932
msgid "Name"
msgstr "Nom"
msgstr "Réactiver le son"
#: changedetectionio/forms.py:921
msgid "Proxy URL"
@@ -429,7 +429,7 @@ msgstr "Requêtes en texte brut"
#: changedetectionio/forms.py:946
msgid "Chrome requests"
msgstr "Requêtes Chrome"
msgstr "Demande"
#: changedetectionio/forms.py:952
msgid "Default proxy"
@@ -485,7 +485,7 @@ msgstr "Contrôle de sécurité du jeton d'accès à l'API activé"
#: changedetectionio/forms.py:989
msgid "Notification base URL override"
msgstr "URL de base pour les notifications"
msgstr "Nombre d'alertes de notification"
#: changedetectionio/forms.py:993
msgid "Treat empty pages as a change?"
@@ -621,11 +621,11 @@ msgstr "Aucune sauvegarde trouvée."
#: changedetectionio/blueprint/backups/templates/overview.html:28
msgid "Create backup"
msgstr "Créer sauvegarde"
msgstr "SAUVEGARDES"
#: changedetectionio/blueprint/backups/templates/overview.html:30
msgid "Remove backups"
msgstr "Supprimer sauvegardes"
msgstr "SAUVEGARDES"
#: changedetectionio/blueprint/imports/importer.py:45
msgid ""
@@ -844,7 +844,7 @@ msgstr "Général"
#: changedetectionio/blueprint/settings/templates/settings.html:23
msgid "Fetching"
msgstr "Récupération"
msgstr "Recherche"
#: changedetectionio/blueprint/settings/templates/settings.html:24
msgid "Global Filters"
@@ -872,7 +872,7 @@ msgstr "CAPTCHA et procurations"
#: changedetectionio/blueprint/settings/templates/settings.html:35
msgid "Info"
msgstr "Info"
msgstr "Plus d'informations"
#: changedetectionio/blueprint/settings/templates/settings.html:46
msgid "Default recheck time for all watches, current system minimum is"
@@ -902,7 +902,7 @@ msgstr "Aucun plugin actif"
#: changedetectionio/blueprint/settings/templates/settings.html:405
msgid "Back"
msgstr "Retour"
msgstr "SAUVEGARDES"
#: changedetectionio/blueprint/settings/templates/settings.html:406
msgid "Clear Snapshot History"
@@ -1276,7 +1276,7 @@ msgstr "lien d'abord."
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:17
msgid "Confirmation text"
msgstr "Texte de confirmation"
msgstr "Aucune information"
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:27
msgid "Type in the word"
@@ -1483,7 +1483,7 @@ msgstr "Conditions"
#: changedetectionio/blueprint/ui/templates/edit.html:60
msgid "Stats"
msgstr "Statistiques"
msgstr "PARAMÈTRES"
#: changedetectionio/blueprint/ui/templates/edit.html:73
#: changedetectionio/blueprint/ui/templates/edit.html:313
@@ -1942,7 +1942,7 @@ msgstr "Surveillez cette URL !"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:35
msgid "Edit first then Watch"
msgstr "Modifier > Surveiller"
msgstr "Modifiez d'abord, puis regardez"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:45
msgid "Create a shareable link"
@@ -2054,7 +2054,8 @@ msgstr "Modifié"
msgid "No website watches configured, please add a URL in the box above, or"
msgstr ""
"Aucune surveillance de site Web configurée, veuillez ajouter une URL dans"
" la case ci-dessus, ou"
" la case ci-dessus, ouAucune surveillance de site Web configurée, "
"veuillez ajouter une URL dans la case ci-dessus, ou"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:130
msgid "import a list"
@@ -2091,7 +2092,7 @@ msgstr "En file d'attente"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:250
msgid "History"
msgstr "Historique"
msgstr "Histoire"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:251
msgid "Preview"
@@ -175,15 +175,16 @@ msgstr "Valore non valido."
#: changedetectionio/forms.py:732
msgid "Watch"
msgstr "Monitora"
msgstr "Osserva"
#: changedetectionio/forms.py:733 changedetectionio/forms.py:766
msgid "Processor"
msgstr "Processore"
#: changedetectionio/forms.py:734
#, fuzzy
msgid "Edit > Watch"
msgstr "Modifica > Monitora"
msgstr "Modifica prima poi Monitora"
#: changedetectionio/forms.py:747 changedetectionio/forms.py:994
msgid "Fetch Method"
@@ -414,8 +415,9 @@ msgid "Invalid template syntax in \"%(header)s\" header: %(error)s"
msgstr ""
#: changedetectionio/forms.py:920 changedetectionio/forms.py:932
#, fuzzy
msgid "Name"
msgstr "Nome"
msgstr "Riattiva audio"
#: changedetectionio/forms.py:921
msgid "Proxy URL"
@@ -438,8 +440,9 @@ msgid "Plaintext requests"
msgstr "Richieste in chiaro"
#: changedetectionio/forms.py:946
#, fuzzy
msgid "Chrome requests"
msgstr "Richieste Chrome"
msgstr "Richiesta"
#: changedetectionio/forms.py:952
msgid "Default proxy"
@@ -494,8 +497,9 @@ msgid "API access token security check enabled"
msgstr "Controllo sicurezza token API attivo"
#: changedetectionio/forms.py:989
#, fuzzy
msgid "Notification base URL override"
msgstr "URL base notifiche"
msgstr "Notifiche"
#: changedetectionio/forms.py:993
msgid "Treat empty pages as a change?"
@@ -1022,7 +1026,7 @@ msgstr ""
#: changedetectionio/blueprint/tags/templates/groups-overview.html:31
msgid "# Watches"
msgstr "# Monitoraggi"
msgstr ""
#: changedetectionio/blueprint/tags/templates/groups-overview.html:32
msgid "Tag / Label name"
@@ -1269,7 +1273,7 @@ msgstr ""
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:17
msgid "Confirmation text"
msgstr "Testo di conferma"
msgstr ""
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:27
msgid "Type in the word"
@@ -1911,7 +1915,7 @@ msgstr "Monitora questo URL!"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:35
msgid "Edit first then Watch"
msgstr "Modifica > Monitora"
msgstr "Modifica prima poi Monitora"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:45
msgid "Create a shareable link"
@@ -2021,7 +2025,7 @@ msgstr "Modifica"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:130
msgid "No website watches configured, please add a URL in the box above, or"
msgstr "Nessun monitoraggio configurato, aggiungi un URL nella casella sopra, oppure"
msgstr ""
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:130
msgid "import a list"
@@ -2424,12 +2428,12 @@ msgstr "Cambia lingua"
#: changedetectionio/templates/base.html:253
#, fuzzy
msgid "Watch List"
msgstr "Lista Monitoraggi"
msgstr "Osserva"
#: changedetectionio/templates/base.html:258
#, fuzzy
msgid "Watches"
msgstr "Monitoraggi"
msgstr "Osserva"
#: changedetectionio/templates/base.html:261
msgid "Queue Status"
@@ -176,16 +176,18 @@ msgid "Invalid value."
msgstr "값이 잘못되었습니다."
#: changedetectionio/forms.py:732
#, fuzzy
msgid "Watch"
msgstr "모니터"
msgstr "# 시계"
#: changedetectionio/forms.py:733 changedetectionio/forms.py:766
msgid "Processor"
msgstr "프로세서"
#: changedetectionio/forms.py:734
#, fuzzy
msgid "Edit > Watch"
msgstr "편집 > 모니터"
msgstr "먼저 편집한 다음 보기"
#: changedetectionio/forms.py:747 changedetectionio/forms.py:994
#, fuzzy
@@ -361,7 +363,7 @@ msgstr "구하다"
#: changedetectionio/forms.py:829
msgid "Proxy"
msgstr "프록시"
msgstr "대리"
#: changedetectionio/forms.py:831
msgid "Send a notification when the filter can no longer be found on the page"
@@ -372,7 +374,7 @@ msgstr "페이지에서 필터를 더 이상 찾을 수 없으면 알림 보내
#: changedetectionio/blueprint/ui/templates/edit.html:59
#: changedetectionio/forms.py:832
msgid "Notifications"
msgstr "알림"
msgstr "정보 없음"
#: changedetectionio/forms.py:832
#, fuzzy
@@ -425,8 +427,9 @@ msgid "Invalid template syntax in \"%(header)s\" header: %(error)s"
msgstr ""
#: changedetectionio/forms.py:920 changedetectionio/forms.py:932
#, fuzzy
msgid "Name"
msgstr "이름"
msgstr "음소거 해제"
#: changedetectionio/forms.py:921
msgid "Proxy URL"
@@ -449,8 +452,9 @@ msgid "Plaintext requests"
msgstr "일반 텍스트 요청"
#: changedetectionio/forms.py:946
#, fuzzy
msgid "Chrome requests"
msgstr "Chrome 요청"
msgstr "요구"
#: changedetectionio/forms.py:952
msgid "Default proxy"
@@ -507,8 +511,9 @@ msgid "API access token security check enabled"
msgstr "API 액세스 토큰 보안 확인이 활성화되었습니다."
#: changedetectionio/forms.py:989
#, fuzzy
msgid "Notification base URL override"
msgstr "알림 기본 URL"
msgstr "알림 경고 수"
#: changedetectionio/forms.py:993
msgid "Treat empty pages as a change?"
@@ -642,11 +647,11 @@ msgstr "백업을 찾을 수 없습니다."
#: changedetectionio/blueprint/backups/templates/overview.html:28
msgid "Create backup"
msgstr "백업 생성"
msgstr "백업"
#: changedetectionio/blueprint/backups/templates/overview.html:30
msgid "Remove backups"
msgstr "백업 삭제"
msgstr "백업"
#: changedetectionio/blueprint/imports/importer.py:45
msgid ""
@@ -854,7 +859,7 @@ msgstr "일반적인"
#: changedetectionio/blueprint/settings/templates/settings.html:23
msgid "Fetching"
msgstr "가져오기"
msgstr "수색"
#: changedetectionio/blueprint/settings/templates/settings.html:24
msgid "Global Filters"
@@ -882,7 +887,7 @@ msgstr "보안 문자 및 프록시"
#: changedetectionio/blueprint/settings/templates/settings.html:35
msgid "Info"
msgstr "정보"
msgstr "추가 정보"
#: changedetectionio/blueprint/settings/templates/settings.html:46
msgid "Default recheck time for all watches, current system minimum is"
@@ -910,7 +915,7 @@ msgstr "활성화된 플러그인이 없습니다."
#: changedetectionio/blueprint/settings/templates/settings.html:405
msgid "Back"
msgstr "뒤로"
msgstr "백업"
#: changedetectionio/blueprint/settings/templates/settings.html:406
msgid "Clear Snapshot History"
@@ -1033,7 +1038,7 @@ msgstr ""
#: changedetectionio/blueprint/tags/templates/groups-overview.html:31
msgid "# Watches"
msgstr "# 모니터"
msgstr "# 시계"
#: changedetectionio/blueprint/tags/templates/groups-overview.html:32
msgid "Tag / Label name"
@@ -1285,7 +1290,7 @@ msgstr "먼저 링크하세요."
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:17
msgid "Confirmation text"
msgstr "확인 텍스트"
msgstr "정보 없음"
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:27
msgid "Type in the word"
@@ -1488,7 +1493,7 @@ msgstr "정황"
#: changedetectionio/blueprint/ui/templates/edit.html:60
msgid "Stats"
msgstr "통계"
msgstr "설정"
#: changedetectionio/blueprint/ui/templates/edit.html:73
#: changedetectionio/blueprint/ui/templates/edit.html:313
@@ -1923,11 +1928,11 @@ msgstr "새로운 웹 페이지 변경 감지 감시 추가"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:34
msgid "Watch this URL!"
msgstr "이 URL 모니터!"
msgstr "이 URL을 시청하세요!"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:35
msgid "Edit first then Watch"
msgstr "편집 후 모니터"
msgstr "먼저 편집한 다음 보기"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:45
msgid "Create a shareable link"
@@ -2074,7 +2079,7 @@ msgstr "대기 중"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:250
msgid "History"
msgstr "기록"
msgstr "역사"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:251
msgid "Preview"
@@ -2400,7 +2405,7 @@ msgstr "설정"
#: changedetectionio/templates/base.html:84
#: changedetectionio/templates/base.html:170
msgid "IMPORT"
msgstr "가져오기"
msgstr "수입"
#: changedetectionio/templates/base.html:87
#: changedetectionio/templates/base.html:171
@@ -2440,12 +2445,12 @@ msgstr "언어 변경"
#: changedetectionio/templates/base.html:253
#, fuzzy
msgid "Watch List"
msgstr "모니터 목록"
msgstr "# 시계"
#: changedetectionio/templates/base.html:258
#, fuzzy
msgid "Watches"
msgstr "모니터"
msgstr "# 시계"
#: changedetectionio/templates/base.html:261
msgid "Queue Status"
@@ -176,8 +176,9 @@ msgid "Invalid value."
msgstr "无效值。"
#: changedetectionio/forms.py:732
#, fuzzy
msgid "Watch"
msgstr "监控"
msgstr "# 手表"
#: changedetectionio/forms.py:733 changedetectionio/forms.py:766
msgid "Processor"
@@ -186,7 +187,7 @@ msgstr "处理器"
#: changedetectionio/forms.py:734
#, fuzzy
msgid "Edit > Watch"
msgstr "编辑 > 监控"
msgstr "编辑后观看"
#: changedetectionio/forms.py:747 changedetectionio/forms.py:994
#, fuzzy
@@ -362,7 +363,7 @@ msgstr "节省"
#: changedetectionio/forms.py:829
msgid "Proxy"
msgstr "代理"
msgstr "代理"
#: changedetectionio/forms.py:831
msgid "Send a notification when the filter can no longer be found on the page"
@@ -373,7 +374,7 @@ msgstr "当页面上找不到过滤器时发送通知"
#: changedetectionio/blueprint/ui/templates/edit.html:59
#: changedetectionio/forms.py:832
msgid "Notifications"
msgstr "通知"
msgstr "暂无信息"
#: changedetectionio/forms.py:832
#, fuzzy
@@ -426,8 +427,9 @@ msgid "Invalid template syntax in \"%(header)s\" header: %(error)s"
msgstr ""
#: changedetectionio/forms.py:920 changedetectionio/forms.py:932
#, fuzzy
msgid "Name"
msgstr "名称"
msgstr "取消静音"
#: changedetectionio/forms.py:921
msgid "Proxy URL"
@@ -450,8 +452,9 @@ msgid "Plaintext requests"
msgstr "明文请求"
#: changedetectionio/forms.py:946
#, fuzzy
msgid "Chrome requests"
msgstr "Chrome请求"
msgstr "求"
#: changedetectionio/forms.py:952
msgid "Default proxy"
@@ -508,8 +511,9 @@ msgid "API access token security check enabled"
msgstr "已启用 API 访问令牌安全检查"
#: changedetectionio/forms.py:989
#, fuzzy
msgid "Notification base URL override"
msgstr "通知基础URL"
msgstr "通知警报计数"
#: changedetectionio/forms.py:993
msgid "Treat empty pages as a change?"
@@ -643,11 +647,11 @@ msgstr "未找到备份。"
#: changedetectionio/blueprint/backups/templates/overview.html:28
msgid "Create backup"
msgstr "创建备份"
msgstr "备份"
#: changedetectionio/blueprint/backups/templates/overview.html:30
msgid "Remove backups"
msgstr "删除备份"
msgstr "备份"
#: changedetectionio/blueprint/imports/importer.py:45
msgid ""
@@ -855,7 +859,7 @@ msgstr "一般的"
#: changedetectionio/blueprint/settings/templates/settings.html:23
msgid "Fetching"
msgstr "获取"
msgstr "搜寻中"
#: changedetectionio/blueprint/settings/templates/settings.html:24
msgid "Global Filters"
@@ -883,7 +887,7 @@ msgstr "验证码和代理"
#: changedetectionio/blueprint/settings/templates/settings.html:35
msgid "Info"
msgstr "信息"
msgstr "更多信息"
#: changedetectionio/blueprint/settings/templates/settings.html:46
msgid "Default recheck time for all watches, current system minimum is"
@@ -911,7 +915,7 @@ msgstr "没有激活的插件"
#: changedetectionio/blueprint/settings/templates/settings.html:405
msgid "Back"
msgstr "返回"
msgstr "备份"
#: changedetectionio/blueprint/settings/templates/settings.html:406
msgid "Clear Snapshot History"
@@ -1034,7 +1038,7 @@ msgstr ""
#: changedetectionio/blueprint/tags/templates/groups-overview.html:31
msgid "# Watches"
msgstr "# 监控项"
msgstr "# 手表"
#: changedetectionio/blueprint/tags/templates/groups-overview.html:32
msgid "Tag / Label name"
@@ -1286,7 +1290,7 @@ msgstr "先链接。"
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:17
msgid "Confirmation text"
msgstr "确认文本"
msgstr "暂无信息"
#: changedetectionio/blueprint/ui/templates/clear_all_history.html:27
msgid "Type in the word"
@@ -1489,7 +1493,7 @@ msgstr "状况"
#: changedetectionio/blueprint/ui/templates/edit.html:60
msgid "Stats"
msgstr "统计"
msgstr "设置"
#: changedetectionio/blueprint/ui/templates/edit.html:73
#: changedetectionio/blueprint/ui/templates/edit.html:313
@@ -1924,11 +1928,11 @@ msgstr "添加新的网页更改检测监视"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:34
msgid "Watch this URL!"
msgstr "监控此URL"
msgstr "关注这个网址"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:35
msgid "Edit first then Watch"
msgstr "编辑后监控"
msgstr "编辑后观看"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:45
msgid "Create a shareable link"
@@ -2401,7 +2405,7 @@ msgstr "设置"
#: changedetectionio/templates/base.html:84
#: changedetectionio/templates/base.html:170
msgid "IMPORT"
msgstr "导入"
msgstr "进口"
#: changedetectionio/templates/base.html:87
#: changedetectionio/templates/base.html:171
@@ -2441,12 +2445,12 @@ msgstr "更改语言"
#: changedetectionio/templates/base.html:253
#, fuzzy
msgid "Watch List"
msgstr "监控列表"
msgstr "# 手表"
#: changedetectionio/templates/base.html:258
#, fuzzy
msgid "Watches"
msgstr "监控项"
msgstr "# 手表"
#: changedetectionio/templates/base.html:261
msgid "Queue Status"
@@ -176,16 +176,18 @@ msgid "Invalid value."
msgstr "無效值。"
#: changedetectionio/forms.py:732
#, fuzzy
msgid "Watch"
msgstr "監控"
msgstr "# 手錶"
#: changedetectionio/forms.py:733 changedetectionio/forms.py:766
msgid "Processor"
msgstr "處理器"
#: changedetectionio/forms.py:734
#, fuzzy
msgid "Edit > Watch"
msgstr "編輯 > 監控"
msgstr "編輯後觀看"
#: changedetectionio/forms.py:747 changedetectionio/forms.py:994
#, fuzzy
@@ -361,7 +363,7 @@ msgstr "節省"
#: changedetectionio/forms.py:829
msgid "Proxy"
msgstr "代理"
msgstr "代理"
#: changedetectionio/forms.py:831
msgid "Send a notification when the filter can no longer be found on the page"
@@ -425,8 +427,9 @@ msgid "Invalid template syntax in \"%(header)s\" header: %(error)s"
msgstr ""
#: changedetectionio/forms.py:920 changedetectionio/forms.py:932
#, fuzzy
msgid "Name"
msgstr "名稱"
msgstr "取消靜音"
#: changedetectionio/forms.py:921
msgid "Proxy URL"
@@ -449,8 +452,9 @@ msgid "Plaintext requests"
msgstr "明文請求"
#: changedetectionio/forms.py:946
#, fuzzy
msgid "Chrome requests"
msgstr "Chrome請求"
msgstr "求"
#: changedetectionio/forms.py:952
msgid "Default proxy"
@@ -507,8 +511,9 @@ msgid "API access token security check enabled"
msgstr "已啟用 API 訪問令牌安全檢查"
#: changedetectionio/forms.py:989
#, fuzzy
msgid "Notification base URL override"
msgstr "通知基礎URL"
msgstr "通知警報計數"
#: changedetectionio/forms.py:993
msgid "Treat empty pages as a change?"
@@ -1033,7 +1038,7 @@ msgstr ""
#: changedetectionio/blueprint/tags/templates/groups-overview.html:31
msgid "# Watches"
msgstr "# 監控項"
msgstr "# 手錶"
#: changedetectionio/blueprint/tags/templates/groups-overview.html:32
msgid "Tag / Label name"
@@ -1923,11 +1928,11 @@ msgstr "添加新的網頁更改檢測監視"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:34
msgid "Watch this URL!"
msgstr "監控此URL"
msgstr "關注這個網址"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:35
msgid "Edit first then Watch"
msgstr "編輯後監控"
msgstr "編輯後觀看"
#: changedetectionio/blueprint/watchlist/templates/watch-overview.html:45
msgid "Create a shareable link"
@@ -2400,7 +2405,7 @@ msgstr "設定"
#: changedetectionio/templates/base.html:84
#: changedetectionio/templates/base.html:170
msgid "IMPORT"
msgstr "導入"
msgstr "進口"
#: changedetectionio/templates/base.html:87
#: changedetectionio/templates/base.html:171
@@ -2440,12 +2445,12 @@ msgstr "更改語言"
#: changedetectionio/templates/base.html:253
#, fuzzy
msgid "Watch List"
msgstr "監控列表"
msgstr "# 手錶"
#: changedetectionio/templates/base.html:258
#, fuzzy
msgid "Watches"
msgstr "監控項"
msgstr "# 手錶"
#: changedetectionio/templates/base.html:261
msgid "Queue Status"
+185 -169
View File
@@ -2,18 +2,19 @@
Worker management module for changedetection.io
Handles asynchronous workers for dynamic worker scaling.
Each worker runs in its own thread with its own event loop for isolation.
Sync worker support has been removed in favor of async-only architecture.
"""
import asyncio
import os
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from loguru import logger
# Global worker state - each worker has its own thread and event loop
worker_threads = [] # List of WorkerThread objects
# Global worker state
running_async_tasks = []
async_loop = None
async_loop_thread = None
# Track currently processing UUIDs for async workers - maps {uuid: worker_id}
currently_processing_uuids = {}
@@ -21,118 +22,89 @@ currently_processing_uuids = {}
# Configuration - async workers only
USE_ASYNC_WORKERS = True
# Custom ThreadPoolExecutor for queue operations with named threads
# Scale executor threads with FETCH_WORKERS to avoid bottleneck at high concurrency
_max_executor_workers = max(50, int(os.getenv("FETCH_WORKERS", "10")))
queue_executor = ThreadPoolExecutor(
max_workers=_max_executor_workers,
thread_name_prefix="QueueGetter-"
)
class WorkerThread:
"""Container for a worker thread with its own event loop"""
def __init__(self, worker_id, update_q, notification_q, app, datastore):
self.worker_id = worker_id
self.update_q = update_q
self.notification_q = notification_q
self.app = app
self.datastore = datastore
self.thread = None
self.loop = None
self.running = False
def run(self):
"""Run the worker in its own event loop"""
try:
# Create a new event loop for this thread
self.loop = asyncio.new_event_loop()
asyncio.set_event_loop(self.loop)
self.running = True
# Run the worker coroutine
self.loop.run_until_complete(
start_single_async_worker(
self.worker_id,
self.update_q,
self.notification_q,
self.app,
self.datastore,
queue_executor
)
)
except asyncio.CancelledError:
# Normal shutdown - worker was cancelled
import os
in_pytest = "pytest" in os.sys.modules or "PYTEST_CURRENT_TEST" in os.environ
if not in_pytest:
logger.info(f"Worker {self.worker_id} shutting down gracefully")
except RuntimeError as e:
# Ignore expected shutdown errors
if "Event loop stopped" not in str(e) and "Event loop is closed" not in str(e):
logger.error(f"Worker {self.worker_id} runtime error: {e}")
except Exception as e:
logger.error(f"Worker {self.worker_id} thread error: {e}")
finally:
# Clean up
if self.loop and not self.loop.is_closed():
self.loop.close()
self.running = False
self.loop = None
def start(self):
"""Start the worker thread"""
self.thread = threading.Thread(
target=self.run,
daemon=True,
name=f"PageFetchAsyncUpdateWorker-{self.worker_id}"
)
self.thread.start()
def stop(self):
"""Stop the worker thread"""
if self.loop and self.running:
try:
# Signal the loop to stop
self.loop.call_soon_threadsafe(self.loop.stop)
except RuntimeError:
pass
if self.thread and self.thread.is_alive():
self.thread.join(timeout=2.0)
def start_async_event_loop():
"""Start a dedicated event loop for async workers in a separate thread"""
global async_loop
logger.info("Starting async event loop for workers")
try:
# Create a new event loop for this thread
async_loop = asyncio.new_event_loop()
# Set it as the event loop for this thread
asyncio.set_event_loop(async_loop)
logger.debug(f"Event loop created and set: {async_loop}")
# Run the event loop forever
async_loop.run_forever()
except Exception as e:
logger.error(f"Async event loop error: {e}")
finally:
# Clean up
if async_loop and not async_loop.is_closed():
async_loop.close()
async_loop = None
logger.info("Async event loop stopped")
def start_async_workers(n_workers, update_q, notification_q, app, datastore):
"""Start async workers, each with its own thread and event loop for isolation"""
global worker_threads, currently_processing_uuids
# Clear any stale state
"""Start the async worker management system"""
global async_loop_thread, async_loop, running_async_tasks, currently_processing_uuids
# Clear any stale UUID tracking state
currently_processing_uuids.clear()
# Start each worker in its own thread with its own event loop
logger.info(f"Starting {n_workers} async workers (isolated threads)")
# Start the event loop in a separate thread
async_loop_thread = threading.Thread(target=start_async_event_loop, daemon=True)
async_loop_thread.start()
# Wait for the loop to be available (with timeout for safety)
max_wait_time = 5.0
wait_start = time.time()
while async_loop is None and (time.time() - wait_start) < max_wait_time:
time.sleep(0.1)
if async_loop is None:
logger.error("Failed to start async event loop within timeout")
return
# Additional brief wait to ensure loop is running
time.sleep(0.2)
# Start async workers
logger.info(f"Starting {n_workers} async workers")
for i in range(n_workers):
try:
worker = WorkerThread(i, update_q, notification_q, app, datastore)
worker.start()
worker_threads.append(worker)
# No sleep needed - threads start independently and asynchronously
except Exception as e:
# Use a factory function to create named worker coroutines
def create_named_worker(worker_id):
async def named_worker():
task = asyncio.current_task()
if task:
task.set_name(f"async-worker-{worker_id}")
return await start_single_async_worker(worker_id, update_q, notification_q, app, datastore)
return named_worker()
task_future = asyncio.run_coroutine_threadsafe(create_named_worker(i), async_loop)
running_async_tasks.append(task_future)
except RuntimeError as e:
logger.error(f"Failed to start async worker {i}: {e}")
continue
async def start_single_async_worker(worker_id, update_q, notification_q, app, datastore, executor=None):
async def start_single_async_worker(worker_id, update_q, notification_q, app, datastore):
"""Start a single async worker with auto-restart capability"""
from changedetectionio.async_update_worker import async_update_worker
# Check if we're in pytest environment - if so, be more gentle with logging
import os
in_pytest = "pytest" in os.sys.modules or "PYTEST_CURRENT_TEST" in os.environ
while not app.config.exit.is_set():
try:
await async_update_worker(worker_id, update_q, notification_q, app, datastore, executor)
if not in_pytest:
logger.info(f"Starting async worker {worker_id}")
await async_update_worker(worker_id, update_q, notification_q, app, datastore)
# If we reach here, worker exited cleanly
if not in_pytest:
logger.info(f"Async worker {worker_id} exited cleanly")
@@ -159,38 +131,39 @@ def start_workers(n_workers, update_q, notification_q, app, datastore):
def add_worker(update_q, notification_q, app, datastore):
"""Add a new async worker (for dynamic scaling)"""
global worker_threads
worker_id = len(worker_threads)
logger.info(f"Adding async worker {worker_id}")
try:
worker = WorkerThread(worker_id, update_q, notification_q, app, datastore)
worker.start()
worker_threads.append(worker)
return True
except Exception as e:
logger.error(f"Failed to add worker {worker_id}: {e}")
global running_async_tasks
if not async_loop:
logger.error("Async loop not running, cannot add worker")
return False
worker_id = len(running_async_tasks)
logger.info(f"Adding async worker {worker_id}")
task_future = asyncio.run_coroutine_threadsafe(
start_single_async_worker(worker_id, update_q, notification_q, app, datastore), async_loop
)
running_async_tasks.append(task_future)
return True
def remove_worker():
"""Remove an async worker (for dynamic scaling)"""
global worker_threads
if not worker_threads:
global running_async_tasks
if not running_async_tasks:
return False
# Stop the last worker
worker = worker_threads.pop()
worker.stop()
logger.info(f"Removed async worker, {len(worker_threads)} workers remaining")
# Cancel the last worker
task_future = running_async_tasks.pop()
task_future.cancel()
logger.info(f"Removed async worker, {len(running_async_tasks)} workers remaining")
return True
def get_worker_count():
"""Get current number of async workers"""
return len(worker_threads)
return len(running_async_tasks)
def get_running_uuids():
@@ -276,21 +249,38 @@ def queue_item_async_safe(update_q, item, silent=False):
def shutdown_workers():
"""Shutdown all async workers fast and aggressively"""
global worker_threads
global async_loop, async_loop_thread, running_async_tasks
# Check if we're in pytest environment - if so, be more gentle with logging
import os
in_pytest = "pytest" in os.sys.modules or "PYTEST_CURRENT_TEST" in os.environ
if not in_pytest:
logger.info("Fast shutdown of async workers initiated...")
# Stop all worker threads
for worker in worker_threads:
worker.stop()
worker_threads.clear()
# Cancel all async tasks immediately
for task_future in running_async_tasks:
if not task_future.done():
task_future.cancel()
# Stop the async event loop immediately
if async_loop and not async_loop.is_closed():
try:
async_loop.call_soon_threadsafe(async_loop.stop)
except RuntimeError:
# Loop might already be stopped
pass
running_async_tasks.clear()
async_loop = None
# Give async thread minimal time to finish, then continue
if async_loop_thread and async_loop_thread.is_alive():
async_loop_thread.join(timeout=1.0) # Only 1 second timeout
if async_loop_thread.is_alive() and not in_pytest:
logger.info("Async thread still running after timeout - continuing with shutdown")
async_loop_thread = None
if not in_pytest:
logger.info("Async workers fast shutdown complete")
@@ -300,57 +290,69 @@ def shutdown_workers():
def adjust_async_worker_count(new_count, update_q=None, notification_q=None, app=None, datastore=None):
"""
Dynamically adjust the number of async workers.
Args:
new_count: Target number of workers
update_q, notification_q, app, datastore: Required for adding new workers
Returns:
dict: Status of the adjustment operation
"""
global worker_threads
global running_async_tasks
current_count = get_worker_count()
if new_count == current_count:
return {
'status': 'no_change',
'message': f'Worker count already at {current_count}',
'current_count': current_count
}
if new_count > current_count:
# Add workers
workers_to_add = new_count - current_count
logger.info(f"Adding {workers_to_add} async workers (from {current_count} to {new_count})")
if not all([update_q, notification_q, app, datastore]):
return {
'status': 'error',
'message': 'Missing required parameters to add workers',
'current_count': current_count
}
for i in range(workers_to_add):
add_worker(update_q, notification_q, app, datastore)
worker_id = len(running_async_tasks)
task_future = asyncio.run_coroutine_threadsafe(
start_single_async_worker(worker_id, update_q, notification_q, app, datastore),
async_loop
)
running_async_tasks.append(task_future)
return {
'status': 'success',
'message': f'Added {workers_to_add} workers',
'previous_count': current_count,
'current_count': len(worker_threads)
'current_count': new_count
}
else:
# Remove workers
workers_to_remove = current_count - new_count
logger.info(f"Removing {workers_to_remove} async workers (from {current_count} to {new_count})")
removed_count = 0
for _ in range(workers_to_remove):
if remove_worker():
if running_async_tasks:
task_future = running_async_tasks.pop()
task_future.cancel()
# Wait for the task to actually stop
try:
task_future.result(timeout=5) # 5 second timeout
except Exception:
pass # Task was cancelled, which is expected
removed_count += 1
return {
'status': 'success',
'message': f'Removed {removed_count} workers',
@@ -365,58 +367,72 @@ def get_worker_status():
'worker_type': 'async',
'worker_count': get_worker_count(),
'running_uuids': get_running_uuids(),
'active_threads': sum(1 for w in worker_threads if w.thread and w.thread.is_alive()),
'async_loop_running': async_loop is not None,
}
def check_worker_health(expected_count, update_q=None, notification_q=None, app=None, datastore=None):
"""
Check if the expected number of async workers are running and restart any missing ones.
Args:
expected_count: Expected number of workers
update_q, notification_q, app, datastore: Required for restarting workers
Returns:
dict: Health check results
"""
global worker_threads
global running_async_tasks
current_count = get_worker_count()
# Check which workers are actually alive
alive_count = sum(1 for w in worker_threads if w.thread and w.thread.is_alive())
if alive_count == expected_count:
if current_count == expected_count:
return {
'status': 'healthy',
'expected_count': expected_count,
'actual_count': alive_count,
'actual_count': current_count,
'message': f'All {expected_count} async workers running'
}
# Find dead workers
# Check for crashed async workers
dead_workers = []
for i, worker in enumerate(worker_threads[:]):
if not worker.thread or not worker.thread.is_alive():
dead_workers.append(i)
logger.warning(f"Async worker {worker.worker_id} thread is dead")
alive_count = 0
for i, task_future in enumerate(running_async_tasks[:]):
if task_future.done():
try:
result = task_future.result()
dead_workers.append(i)
logger.warning(f"Async worker {i} completed unexpectedly")
except Exception as e:
dead_workers.append(i)
logger.error(f"Async worker {i} crashed: {e}")
else:
alive_count += 1
# Remove dead workers from tracking
for i in reversed(dead_workers):
if i < len(worker_threads):
worker_threads.pop(i)
if i < len(running_async_tasks):
running_async_tasks.pop(i)
missing_workers = expected_count - alive_count
restarted_count = 0
if missing_workers > 0 and all([update_q, notification_q, app, datastore]):
logger.info(f"Restarting {missing_workers} crashed async workers")
for i in range(missing_workers):
if add_worker(update_q, notification_q, app, datastore):
worker_id = alive_count + i
try:
task_future = asyncio.run_coroutine_threadsafe(
start_single_async_worker(worker_id, update_q, notification_q, app, datastore),
async_loop
)
running_async_tasks.append(task_future)
restarted_count += 1
except Exception as e:
logger.error(f"Failed to restart worker {worker_id}: {e}")
return {
'status': 'repaired' if restarted_count > 0 else 'degraded',
'expected_count': expected_count,
+4 -86
View File
@@ -28,7 +28,7 @@ info:
For example: `x-api-key: YOUR_API_KEY`
version: 0.1.4
version: 0.1.3
contact:
name: ChangeDetection.io
url: https://github.com/dgtlmoon/changedetection.io
@@ -761,9 +761,9 @@ paths:
get:
operationId: getWatchHistoryDiff
tags: [Watch History]
summary: Get the difference between two snapshots
summary: Get diff between two snapshots
description: |
Generate a difference (comparison) between two historical snapshots of a web page change monitor (watch).
Generate a formatted diff (comparison) between two historical snapshots of a web page change monitor (watch).
This endpoint compares content between two points in time and returns the differences in your chosen format.
Perfect for reviewing what changed between specific versions or comparing recent changes.
@@ -798,10 +798,6 @@ paths:
# Compare two specific timestamps in plain text with word-level diff
curl -X GET "http://localhost:5000/api/v1/watch/095be615-a8ad-4c33-8e9c-c7612fbf6c9f/difference/1640995200/1640998800?format=text&word_diff=true" \
-H "x-api-key: YOUR_API_KEY"
# Show only additions (hide removed/replaced content), ignore whitespace
curl -X GET "http://localhost:5000/api/v1/watch/095be615-a8ad-4c33-8e9c-c7612fbf6c9f/difference/previous/latest?format=htmlcolor&removed=false&replaced=false&ignoreWhitespace=true" \
-H "x-api-key: YOUR_API_KEY"
- lang: 'Python'
source: |
import requests
@@ -826,20 +822,6 @@ paths:
params={'format': 'text', 'word_diff': 'true'}
)
print(response.text)
# Show only additions, ignore whitespace and use word-level diff
response = requests.get(
f'http://localhost:5000/api/v1/watch/{uuid}/difference/previous/latest',
headers=headers,
params={
'format': 'htmlcolor',
'type': 'diffWords',
'removed': 'false',
'replaced': 'false',
'ignoreWhitespace': 'true'
}
)
print(response.text)
parameters:
- name: uuid
in: path
@@ -879,10 +861,9 @@ paths:
- `text` (default): Plain text with (removed) and (added) prefixes
- `html`: Basic HTML format
- `htmlcolor`: Rich HTML with colored backgrounds (red for deletions, green for additions)
- `markdown`: Markdown format with HTML rendering
schema:
type: string
enum: [text, html, htmlcolor, markdown]
enum: [text, html, htmlcolor]
default: text
- name: word_diff
in: query
@@ -907,69 +888,6 @@ paths:
type: string
enum: ["true", "false", "1", "0", "yes", "no", "on", "off"]
default: "false"
- name: type
in: query
description: |
Diff granularity type:
- `diffLines` (default): Line-level comparison, showing which lines changed
- `diffWords`: Word-level comparison, showing which words changed within lines
This parameter is an alternative to `word_diff` for better alignment with the UI.
If both are specified, `type=diffWords` will enable word-level diffing.
schema:
type: string
enum: [diffLines, diffWords]
default: diffLines
- name: changesOnly
in: query
description: |
When enabled, only show lines/content that changed (no surrounding context).
When disabled, include unchanged lines for context around changes.
Accepts: true, false, 1, 0, yes, no, on, off
schema:
type: string
enum: ["true", "false", "1", "0", "yes", "no", "on", "off"]
default: "true"
- name: ignoreWhitespace
in: query
description: |
When enabled, ignore whitespace-only changes (spaces, tabs, newlines).
Useful for focusing on content changes and ignoring formatting differences.
Accepts: true, false, 1, 0, yes, no, on, off
schema:
type: string
enum: ["true", "false", "1", "0", "yes", "no", "on", "off"]
default: "false"
- name: removed
in: query
description: |
Include removed/deleted content in the diff output.
When disabled, content that was deleted will not appear in the diff.
Accepts: true, false, 1, 0, yes, no, on, off
schema:
type: string
enum: ["true", "false", "1", "0", "yes", "no", "on", "off"]
default: "true"
- name: added
in: query
description: |
Include added/new content in the diff output.
When disabled, content that was added will not appear in the diff.
Accepts: true, false, 1, 0, yes, no, on, off
schema:
type: string
enum: ["true", "false", "1", "0", "yes", "no", "on", "off"]
default: "true"
- name: replaced
in: query
description: |
Include replaced/modified content in the diff output.
When disabled, content that was modified (changed from one value to another) will not appear in the diff.
Accepts: true, false, 1, 0, yes, no, on, off
schema:
type: string
enum: ["true", "false", "1", "0", "yes", "no", "on", "off"]
default: "true"
responses:
'200':
description: Formatted diff between the two snapshots
+4 -163
View File
File diff suppressed because one or more lines are too long
+4 -4
View File
@@ -12,8 +12,8 @@ janus # Thread-safe async/sync queue bridge
flask_wtf~=1.2
flask~=3.1
flask-socketio~=5.6.0
python-socketio~=5.16.0
python-engineio~=4.13.0
python-socketio~=5.14.3
python-engineio~=4.12.3
inscriptis~=2.2
pytz
timeago~=1.0
@@ -60,7 +60,7 @@ cryptography==46.0.3
paho-mqtt!=2.0.*
# Used for CSS filtering, JSON extraction from HTML
beautifulsoup4>=4.0.0,<=4.14.3
beautifulsoup4>=4.0.0,<=4.14.2
# XPath filtering, lxml is required by bs4 anyway, but put it here to be safe.
# #2328 - 5.2.0 and 5.2.1 had extra CPU flag CFLAGS set which was not compatible on older hardware
@@ -148,7 +148,7 @@ tzdata
pluggy ~= 1.6
# Needed for testing, cross-platform for process and system monitoring
psutil==7.2.1
psutil==7.1.0
ruff >= 0.11.2
pre_commit >= 4.2.0