mirror of https://github.com/dgtlmoon/changedetection.io.git synced 2026-05-02 07:40:39 +00:00

Files

T

dgtlmoon 3d14df6a11 Development branch merge into release/master

Multi-language / Translations Support (#3696)
  - Complete internationalization system implemented
  - Support for 7 languages: Czech (cs), German (de), French (fr), Italian (it), Korean (ko), Chinese Simplified (zh), Chinese Traditional (zh_TW)
  - Language selector with localized flags and theming
  - Flash message translations
  - Multiple translation fixes and improvements across all languages
  - Language setting preserved across redirects

  Pluggable Content Fetchers (#3653)
  - New architecture for extensible content fetcher system
  - Allows custom fetcher implementations

  Image / Screenshot Comparison Processor (#3680)
  - New processor for visual change detection (disabled for this release)
  - Supporting CSS/JS infrastructure added

  UI Improvements

  Design & Layout
  - Auto-generated tag color schemes
  - Simplified login form styling
  - Removed hard-coded CSS, moved to SCSS variables
  - Tag UI cleanup and improvements
  - Automatic tab wrapper functionality
  - Menu refactoring for better organization
  - Cleanup of offset settings
  - Hide sticky tabs on narrow viewports
  - Improved responsive layout (#3702)

  User Experience
  - Modal alerts/confirmations on delete/clear operations (#3693, #3598, #3382)
  - Auto-add https:// to URLs in quickwatch form if not present
  - Better redirect handling on login (#3699)
  - 'Recheck all' now returns to correct group/tag (#3673)
  - Language set redirect keeps hash fragment
  - More friendly human-readable text throughout UI

  Performance & Reliability

  Scheduler & Processing
  - Soft delays instead of blocking time.sleep() calls (#3710)
  - More resilient handling of same UUID being processed (#3700)
  - Better Puppeteer timeout handling
  - Improved Puppeteer shutdown/cleanup (#3692)
  - Requests cleanup now properly async

  History & Rendering
  - Faster server-side "difference" rendering on History page (#3442)
  - Show ignored/triggered rows in history
  - API: Retry watch data if watch dict changed (more reliable)

  API Improvements

  - Watch get endpoint: retry mechanism for changed watch data
  - WatchHistoryDiff API endpoint includes extra format args (#3703)

  Testing Improvements

  - Replace time.sleep with wait_for_notification_endpoint_output (#3716)
  - Test for mode switching (#3701)
  - Test for #3720 added (#3725)
  - Extract-text difference test fixes
  - Improved dev workflow

  Bug Fixes

  - Notification error text output (#3672, #3669, #3280)
  - HTML validation fixes (#3704)
  - Template discovery path fixes
  - Notification debug log now uses system locale for dates/times
  - Puppeteer spelling mistake in log output
  - Recalculation on anchor change
  - Queue bubble update disabled temporarily

  Dependency Updates

  - beautifulsoup4 updated (#3724)
  - psutil 7.1.0 → 7.2.1 (#3723)
  - python-engineio ~=4.12.3 → ~=4.13.0 (#3707)
  - python-socketio ~=5.14.3 → ~=5.16.0 (#3706)
  - flask-socketio ~=5.5.1 → ~=5.6.0 (#3691)
  - brotli ~=1.1 → ~=1.2 (#3687)
  - lxml updated (#3590)
  - pytest ~=7.2 → ~=9.0 (#3676)
  - jsonschema ~=4.0 → ~=4.25 (#3618)
  - pluggy ~=1.5 → ~=1.6 (#3616)
  - cryptography 44.0.1 → 46.0.3 (security) (#3589)

  Documentation

  - README updated with viewport size setup information

  Development Infrastructure

  - Dev container only built on dev branch
  - Improved dev workflow tooling

2026-01-12 17:50:53 +01:00

__init__.py

…

events.py

Development branch merge into release/master

2026-01-12 17:50:53 +01:00

README.md

Realtime UI - Socketio tweaks and refactor (#3220 )

2025-06-03 10:17:19 +02:00

socket_server.py

Development branch merge into release/master

2026-01-12 17:50:53 +01:00

README.md

Real-time Socket.IO Implementation

This directory contains the Socket.IO implementation for changedetection.io's real-time updates.

Architecture Overview

The real-time system provides live updates to the web interface for:

Watch status changes (checking, completed, errors)
Queue length updates
General statistics updates

Current Implementation

Socket.IO Configuration

Async Mode: threading (default) or gevent (optional via SOCKETIO_MODE env var)
Server: Flask-SocketIO with threading support
Background Tasks: Python threading with daemon threads

Async Worker Integration

Workers: Async workers using asyncio for watch processing
Queue: AsyncSignalPriorityQueue for job distribution
Signals: Blinker signals for real-time updates between workers and Socket.IO

Environment Variables

SOCKETIO_MODE=threading (default, recommended)
SOCKETIO_MODE=gevent (optional, has cross-platform limitations)

Architecture Decision: Why Threading Mode?

Previous Issues with Eventlet

Eventlet was completely removed due to fundamental compatibility issues:

Monkey Patching Conflicts: eventlet.monkey_patch() globally replaced Python's threading/socket modules, causing conflicts with:
- Playwright's synchronous browser automation
- Async worker event loops
- Various Python libraries expecting real threading
Python 3.12+ Compatibility: Eventlet had issues with newer Python versions and asyncio integration
CVE-2023-29483: Security vulnerability in eventlet's dnspython dependency

Current Solution Benefits

✅ Threading Mode Advantages:

Full compatibility with async workers and Playwright
No monkey patching - uses standard Python threading
Better Python 3.12+ support
Cross-platform compatibility (Windows, macOS, Linux)
No external async library dependencies
Fast shutdown capabilities

✅ Optional Gevent Support:

Available via SOCKETIO_MODE=gevent for high-concurrency scenarios
Cross-platform limitations documented in requirements.txt
Not recommended as default due to Windows socket limits and macOS ARM build issues

Socket.IO Mode Configuration

Threading Mode (Default)

# Enabled automatically
async_mode = 'threading'
socketio = SocketIO(app, async_mode='threading')

Gevent Mode (Optional)

# Set environment variable
export SOCKETIO_MODE=gevent

Background Tasks

Queue Polling

Threading Mode: threading.Thread with threading.Event for shutdown
Signal Handling: Blinker signals for watch state changes
Real-time Updates: Direct Socket.IO emit() calls to connected clients

Worker Integration

Async Workers: Run in separate asyncio event loop thread
Communication: AsyncSignalPriorityQueue bridges async workers and Socket.IO
Updates: Real-time updates sent when workers complete tasks

Files in This Directory

socket_server.py: Main Socket.IO initialization and event handling
events.py: Watch operation event handlers
__init__.py: Module initialization

Production Deployment

Recommended WSGI Servers

For production with Socket.IO threading mode:

Gunicorn: gunicorn --worker-class eventlet changedetection:app (if using gevent mode)
uWSGI: With threading support
Docker: Built-in Flask server works well for containerized deployments

Performance Considerations

Threading mode: Better memory usage, standard Python threading
Gevent mode: Higher concurrency but platform limitations
Async workers: Separate from Socket.IO, provides scalability

Environment Variables

Variable	Default	Description
`SOCKETIO_MODE`	`threading`	Socket.IO async mode (`threading` or `gevent`)
`FETCH_WORKERS`	`10`	Number of async workers for watch processing
`CHANGEDETECTION_HOST`	`0.0.0.0`	Server bind address
`CHANGEDETECTION_PORT`	`5000`	Server port

Debugging Tips

Socket.IO Issues: Check browser dev tools for WebSocket connection errors
Threading Issues: Monitor with ps -T to check thread count
Worker Issues: Use /worker-health endpoint to check async worker status
Queue Issues: Use /queue-status endpoint to monitor job queue
Performance: Use /gc-cleanup endpoint to trigger memory cleanup

Migration Notes

If upgrading from eventlet-based versions:

Remove any EVENTLET_* environment variables
No code changes needed - Socket.IO mode is automatically configured
Optional: Set SOCKETIO_MODE=gevent if high concurrency is required and platform supports it