Compare commits

...

9 Commits

Author SHA1 Message Date
dgtlmoon
9a2e50fdcd re #3126 Visual Selector & Browser Steps - Always recheck if the data/screenshot is ready under "Visual Selector" tab after using Browser Steps 2025-04-18 09:56:38 +02:00
dgtlmoon
a1fdeeaa29 Only add screenshot warning if capture was greater than trim size (#3123)
Some checks failed
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built 📦 package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
ChangeDetection.io Container Build Test / test-container-build (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
2025-04-17 00:11:20 +02:00
dgtlmoon
40ea2604a7 0.49.14 2025-04-16 23:23:18 +02:00
dgtlmoon
ceda526093 Small fix for multiprocessing start on Mac OS (#3121 #3115) 2025-04-16 22:52:03 +02:00
Justin Goette
4197254c53 docs: Update reference URL (#3119) 2025-04-16 21:37:50 +02:00
dgtlmoon
a0b7efb436 UI - Fix to edit and groups template 2025-04-16 18:40:30 +02:00
dgtlmoon
5f5e8ede6c Updating API documentation
Some checks failed
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built 📦 package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-04-13 21:51:17 +02:00
dgtlmoon
52ca855a29 Undo forced selenium headless mode, small refactor (#3112)
Some checks failed
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built 📦 package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
ChangeDetection.io Container Build Test / test-container-build (push) Has been cancelled
2025-04-12 19:26:17 +02:00
dgtlmoon
079efd0a85 Playwright + Puppeteer fix for when page is taller than viewport but less than screenshot step_size (#3113) 2025-04-12 18:37:59 +02:00
18 changed files with 184 additions and 166 deletions

View File

@@ -68,7 +68,7 @@ COPY changedetection.py /app/changedetection.py
# Github Action test purpose(test-only.yml). # Github Action test purpose(test-only.yml).
# On production, it is effectively LOGGER_LEVEL=''. # On production, it is effectively LOGGER_LEVEL=''.
ARG LOGGER_LEVEL='' ARG LOGGER_LEVEL=''
ENV LOGGER_LEVEL "$LOGGER_LEVEL" ENV LOGGER_LEVEL="$LOGGER_LEVEL"
WORKDIR /app WORKDIR /app
CMD ["python", "./changedetection.py", "-d", "/datastore"] CMD ["python", "./changedetection.py", "-d", "/datastore"]

View File

@@ -3,4 +3,6 @@
# Only exists for direct CLI usage # Only exists for direct CLI usage
import changedetectionio import changedetectionio
changedetectionio.main()
if __name__ == '__main__':
changedetectionio.main()

View File

@@ -2,7 +2,7 @@
# Read more https://github.com/dgtlmoon/changedetection.io/wiki # Read more https://github.com/dgtlmoon/changedetection.io/wiki
__version__ = '0.49.13' __version__ = '0.49.14'
from changedetectionio.strtobool import strtobool from changedetectionio.strtobool import strtobool
from json.decoder import JSONDecodeError from json.decoder import JSONDecodeError

View File

@@ -104,6 +104,9 @@ def construct_blueprint(datastore: ChangeDetectionStore):
uuid = list(datastore.data['settings']['application']['tags'].keys()).pop() uuid = list(datastore.data['settings']['application']['tags'].keys()).pop()
default = datastore.data['settings']['application']['tags'].get(uuid) default = datastore.data['settings']['application']['tags'].get(uuid)
if not default:
flash("Tag not found", "error")
return redirect(url_for('watchlist.index'))
form = group_restock_settings_form( form = group_restock_settings_form(
formdata=request.form if request.method == 'POST' else None, formdata=request.form if request.method == 'POST' else None,

View File

@@ -66,7 +66,7 @@
<div class="pure-control-group inline-radio"> <div class="pure-control-group inline-radio">
{{ render_checkbox_field(form.notification_muted) }} {{ render_checkbox_field(form.notification_muted) }}
</div> </div>
{% if is_html_webdriver %} {% if 1 %}
<div class="pure-control-group inline-radio"> <div class="pure-control-group inline-radio">
{{ render_checkbox_field(form.notification_screenshot) }} {{ render_checkbox_field(form.notification_screenshot) }}
<span class="pure-form-message-inline"> <span class="pure-form-message-inline">

View File

@@ -19,20 +19,6 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, queuedWatchMe
if tag_uuid in watch.get('tags', []) and (tag.get('include_filters') or tag.get('subtractive_selectors')): if tag_uuid in watch.get('tags', []) and (tag.get('include_filters') or tag.get('subtractive_selectors')):
return True return True
def levenshtein_ratio_recent_history(watch):
try:
from Levenshtein import ratio, distance
k = list(watch.history.keys())
if len(k) >= 2:
a = watch.get_history_snapshot(timestamp=k[0])
b = watch.get_history_snapshot(timestamp=k[1])
distance = distance(a, b)
return distance
except Exception as e:
logger.warning("Unable to calc similarity", e)
return "Unable to calc similarity"
return ''
@edit_blueprint.route("/edit/<string:uuid>", methods=['GET', 'POST']) @edit_blueprint.route("/edit/<string:uuid>", methods=['GET', 'POST'])
@login_optionally_required @login_optionally_required
# https://stackoverflow.com/questions/42984453/wtforms-populate-form-with-data-if-data-exists # https://stackoverflow.com/questions/42984453/wtforms-populate-form-with-data-if-data-exists
@@ -227,9 +213,6 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, queuedWatchMe
if request.method == 'POST' and not form.validate(): if request.method == 'POST' and not form.validate():
flash("An error occurred, please see below.", "error") flash("An error occurred, please see below.", "error")
visualselector_data_is_ready = datastore.visualselector_data_is_ready(uuid)
# JQ is difficult to install on windows and must be manually added (outside requirements.txt) # JQ is difficult to install on windows and must be manually added (outside requirements.txt)
jq_support = True jq_support = True
try: try:
@@ -239,11 +222,12 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, queuedWatchMe
watch = datastore.data['watching'].get(uuid) watch = datastore.data['watching'].get(uuid)
# if system or watch is configured to need a chrome type browser
system_uses_webdriver = datastore.data['settings']['application']['fetch_backend'] == 'html_webdriver' system_uses_webdriver = datastore.data['settings']['application']['fetch_backend'] == 'html_webdriver'
watch_needs_selenium_or_playwright = False
watch_uses_webdriver = False
if (watch.get('fetch_backend') == 'system' and system_uses_webdriver) or watch.get('fetch_backend') == 'html_webdriver' or watch.get('fetch_backend', '').startswith('extra_browser_'): if (watch.get('fetch_backend') == 'system' and system_uses_webdriver) or watch.get('fetch_backend') == 'html_webdriver' or watch.get('fetch_backend', '').startswith('extra_browser_'):
watch_uses_webdriver = True watch_needs_selenium_or_playwright = True
from zoneinfo import available_timezones from zoneinfo import available_timezones
@@ -262,14 +246,16 @@ def construct_blueprint(datastore: ChangeDetectionStore, update_q, queuedWatchMe
'has_extra_headers_file': len(datastore.get_all_headers_in_textfile_for_watch(uuid=uuid)) > 0, 'has_extra_headers_file': len(datastore.get_all_headers_in_textfile_for_watch(uuid=uuid)) > 0,
'has_special_tag_options': _watch_has_tag_options_set(watch=watch), 'has_special_tag_options': _watch_has_tag_options_set(watch=watch),
'jq_support': jq_support, 'jq_support': jq_support,
'lev_info': levenshtein_ratio_recent_history(watch),
'playwright_enabled': os.getenv('PLAYWRIGHT_DRIVER_URL', False), 'playwright_enabled': os.getenv('PLAYWRIGHT_DRIVER_URL', False),
'settings_application': datastore.data['settings']['application'], 'settings_application': datastore.data['settings']['application'],
'system_has_playwright_configured': os.getenv('PLAYWRIGHT_DRIVER_URL'),
'system_has_webdriver_configured': os.getenv('WEBDRIVER_URL'),
'visual_selector_data_ready': datastore.visualselector_data_is_ready(watch_uuid=uuid),
'timezone_default_config': datastore.data['settings']['application'].get('timezone'), 'timezone_default_config': datastore.data['settings']['application'].get('timezone'),
'using_global_webdriver_wait': not default['webdriver_delay'], 'using_global_webdriver_wait': not default['webdriver_delay'],
'uuid': uuid, 'uuid': uuid,
'watch': watch, 'watch': watch,
'watch_uses_webdriver': watch_uses_webdriver, 'watch_needs_selenium_or_playwright': watch_needs_selenium_or_playwright,
} }
included_content = None included_content = None

View File

@@ -94,11 +94,11 @@ def execute_ruleset_against_all_plugins(current_watch_uuid: str, application_dat
EXECUTE_DATA = {} EXECUTE_DATA = {}
result = True result = True
ruleset_settings = application_datastruct['watching'].get(current_watch_uuid) watch = application_datastruct['watching'].get(current_watch_uuid)
if ruleset_settings.get("conditions"): if watch and watch.get("conditions"):
logic_operator = "and" if ruleset_settings.get("conditions_match_logic", "ALL") == "ALL" else "or" logic_operator = "and" if watch.get("conditions_match_logic", "ALL") == "ALL" else "or"
complete_rules = filter_complete_rules(ruleset_settings['conditions']) complete_rules = filter_complete_rules(watch['conditions'])
if complete_rules: if complete_rules:
# Give all plugins a chance to update the data dict again (that we will test the conditions against) # Give all plugins a chance to update the data dict again (that we will test the conditions against)
for plugin in plugin_manager.get_plugins(): for plugin in plugin_manager.get_plugins():

View File

@@ -26,9 +26,11 @@ def capture_full_page(page):
step_size = SCREENSHOT_SIZE_STITCH_THRESHOLD # Size that won't cause GPU to overflow step_size = SCREENSHOT_SIZE_STITCH_THRESHOLD # Size that won't cause GPU to overflow
screenshot_chunks = [] screenshot_chunks = []
y = 0 y = 0
# If page height is larger than current viewport, use a larger viewport for better capturing
if page_height > page.viewport_size['height']: if page_height > page.viewport_size['height']:
if page_height < step_size:
step_size = page_height # Incase page is bigger than default viewport but smaller than proposed step size
logger.debug(f"Setting bigger viewport to step through large page width W{page.viewport_size['width']}xH{step_size} because page_height > viewport_size")
# Set viewport to a larger size to capture more content at once # Set viewport to a larger size to capture more content at once
page.set_viewport_size({'width': page.viewport_size['width'], 'height': step_size}) page.set_viewport_size({'width': page.viewport_size['width'], 'height': step_size})

View File

@@ -46,9 +46,10 @@ async def capture_full_page(page):
screenshot_chunks = [] screenshot_chunks = []
y = 0 y = 0
if page_height > page.viewport['height']: if page_height > page.viewport['height']:
if page_height < step_size:
step_size = page_height # Incase page is bigger than default viewport but smaller than proposed step size
await page.setViewport({'width': page.viewport['width'], 'height': step_size}) await page.setViewport({'width': page.viewport['width'], 'height': step_size})
while y < min(page_height, SCREENSHOT_MAX_TOTAL_HEIGHT): while y < min(page_height, SCREENSHOT_MAX_TOTAL_HEIGHT):
await page.evaluate(f"window.scrollTo(0, {y})") await page.evaluate(f"window.scrollTo(0, {y})")
screenshot_chunks.append(await page.screenshot(type_='jpeg', screenshot_chunks.append(await page.screenshot(type_='jpeg',

View File

@@ -31,33 +31,33 @@ def stitch_images_worker(pipe_conn, chunks_bytes, original_page_height, capture_
# Draw caption on top (overlaid, not extending canvas) # Draw caption on top (overlaid, not extending canvas)
draw = ImageDraw.Draw(stitched) draw = ImageDraw.Draw(stitched)
if original_page_height > capture_height:
caption_text = f"WARNING: Screenshot was {original_page_height}px but trimmed to {capture_height}px because it was too long" caption_text = f"WARNING: Screenshot was {original_page_height}px but trimmed to {capture_height}px because it was too long"
padding = 10 padding = 10
font_size = 35 font_size = 35
font_color = (255, 0, 0) font_color = (255, 0, 0)
background_color = (255, 255, 255) background_color = (255, 255, 255)
# Try to load a proper font # Try to load a proper font
try: try:
font = ImageFont.truetype("arial.ttf", font_size) font = ImageFont.truetype("arial.ttf", font_size)
except IOError: except IOError:
font = ImageFont.load_default() font = ImageFont.load_default()
bbox = draw.textbbox((0, 0), caption_text, font=font) bbox = draw.textbbox((0, 0), caption_text, font=font)
text_width = bbox[2] - bbox[0] text_width = bbox[2] - bbox[0]
text_height = bbox[3] - bbox[1] text_height = bbox[3] - bbox[1]
# Draw white rectangle background behind text # Draw white rectangle background behind text
rect_top = 0 rect_top = 0
rect_bottom = text_height + 2 * padding rect_bottom = text_height + 2 * padding
draw.rectangle([(0, rect_top), (max_width, rect_bottom)], fill=background_color) draw.rectangle([(0, rect_top), (max_width, rect_bottom)], fill=background_color)
# Draw text centered horizontally, 10px padding from top of the rectangle # Draw text centered horizontally, 10px padding from top of the rectangle
text_x = (max_width - text_width) // 2 text_x = (max_width - text_width) // 2
text_y = padding text_y = padding
draw.text((text_x, text_y), caption_text, font=font, fill=font_color) draw.text((text_x, text_y), caption_text, font=font, fill=font_color)
# Encode and send image # Encode and send image
output = io.BytesIO() output = io.BytesIO()

View File

@@ -65,7 +65,17 @@ class fetcher(Fetcher):
# request_body, request_method unused for now, until some magic in the future happens. # request_body, request_method unused for now, until some magic in the future happens.
options = ChromeOptions() options = ChromeOptions()
options.add_argument("--headless")
# Load Chrome options from env
CHROME_OPTIONS = [
line.strip()
for line in os.getenv("CHROME_OPTIONS", "").strip().splitlines()
if line.strip()
]
for opt in CHROME_OPTIONS:
options.add_argument(opt)
if self.proxy: if self.proxy:
options.proxy = self.proxy options.proxy = self.proxy
@@ -80,7 +90,9 @@ class fetcher(Fetcher):
self.quit() self.quit()
raise raise
self.driver.set_window_size(1280, 1024) if not "--window-size" in os.getenv("CHROME_OPTIONS", ""):
self.driver.set_window_size(1280, 1024)
self.driver.implicitly_wait(int(os.getenv("WEBDRIVER_DELAY_BEFORE_CONTENT_READY", 5))) self.driver.implicitly_wait(int(os.getenv("WEBDRIVER_DELAY_BEFORE_CONTENT_READY", 5)))
if self.webdriver_js_execute_code is not None: if self.webdriver_js_execute_code is not None:
@@ -88,6 +100,7 @@ class fetcher(Fetcher):
# Selenium doesn't automatically wait for actions as good as Playwright, so wait again # Selenium doesn't automatically wait for actions as good as Playwright, so wait again
self.driver.implicitly_wait(int(os.getenv("WEBDRIVER_DELAY_BEFORE_CONTENT_READY", 5))) self.driver.implicitly_wait(int(os.getenv("WEBDRIVER_DELAY_BEFORE_CONTENT_READY", 5)))
# @todo - how to check this? is it possible? # @todo - how to check this? is it possible?
self.status_code = 200 self.status_code = 200
# @todo somehow we should try to get this working for WebDriver # @todo somehow we should try to get this working for WebDriver

View File

@@ -98,15 +98,13 @@
{% macro playwright_warning() %} {% macro playwright_warning() %}
<p><strong>Error - Playwright support for Chrome based fetching is not enabled.</strong> Alternatively try our <a href="https://changedetection.io">very affordable subscription based service which has all this setup for you</a>.</p> <p><strong>Error - This watch needs Chrome (with playwright/sockpuppetbrowser), but Chrome based fetching is not enabled.</strong> Alternatively try our <a href="https://changedetection.io">very affordable subscription based service which has all this setup for you</a>.</p>
<p>You may need to <a href="https://github.com/dgtlmoon/changedetection.io/blob/09ebc6ec6338545bdd694dc6eee57f2e9d2b8075/docker-compose.yml#L31">Enable playwright environment variable</a> and uncomment the <strong>sockpuppetbrowser</strong> in the <a href="https://github.com/dgtlmoon/changedetection.io/blob/master/docker-compose.yml">docker-compose.yml</a> file.</p> <p>You may need to <a href="https://github.com/dgtlmoon/changedetection.io/blob/09ebc6ec6338545bdd694dc6eee57f2e9d2b8075/docker-compose.yml#L31">Enable playwright environment variable</a> and uncomment the <strong>sockpuppetbrowser</strong> in the <a href="https://github.com/dgtlmoon/changedetection.io/blob/master/docker-compose.yml">docker-compose.yml</a> file.</p>
<br> <br>
<p>(Also Selenium/WebDriver can not extract full page screenshots reliably so Playwright is recommended here)</p>
{% endmacro %} {% endmacro %}
{% macro only_webdriver_type_watches_warning() %} {% macro only_playwright_type_watches_warning() %}
<p><strong>Sorry, this functionality only works with Playwright/Chrome enabled watches.<br>You need to <a href="#request">Set the fetch method to Playwright/Chrome mode and resave</a> and have the Playwright connection enabled.</strong></p><br> <p><strong>Sorry, this functionality only works with Playwright/Chrome enabled watches.<br>You need to <a href="#request">Set the fetch method to Playwright/Chrome mode and resave</a> and have the SockpuppetBrowser/Playwright or Selenium enabled.</strong></p><br>
{% endmacro %} {% endmacro %}
{% macro render_time_schedule_form(form, available_timezones, timezone_default_config) %} {% macro render_time_schedule_form(form, available_timezones, timezone_default_config) %}

View File

@@ -1,6 +1,6 @@
{% extends 'base.html' %} {% extends 'base.html' %}
{% block content %} {% block content %}
{% from '_helpers.html' import render_field, render_checkbox_field, render_button, render_time_schedule_form, playwright_warning, only_webdriver_type_watches_warning, render_conditions_fieldlist_of_formfields_as_table %} {% from '_helpers.html' import render_field, render_checkbox_field, render_button, render_time_schedule_form, playwright_warning, only_playwright_type_watches_warning, render_conditions_fieldlist_of_formfields_as_table %}
{% from '_common_fields.html' import render_common_settings_form %} {% from '_common_fields.html' import render_common_settings_form %}
<script src="{{url_for('static_content', group='js', filename='tabs.js')}}" defer></script> <script src="{{url_for('static_content', group='js', filename='tabs.js')}}" defer></script>
<script src="{{url_for('static_content', group='js', filename='vis.js')}}" defer></script> <script src="{{url_for('static_content', group='js', filename='vis.js')}}" defer></script>
@@ -204,7 +204,9 @@ Math: {{ 1 + 1 }}") }}
</div> </div>
<div class="tab-pane-inner" id="browser-steps"> <div class="tab-pane-inner" id="browser-steps">
{% if playwright_enabled and watch_uses_webdriver %} {% if watch_needs_selenium_or_playwright %}
{# Only works with playwright #}
{% if system_has_playwright_configured %}
<img class="beta-logo" src="{{url_for('static_content', group='images', filename='beta-logo.png')}}" alt="New beta functionality"> <img class="beta-logo" src="{{url_for('static_content', group='images', filename='beta-logo.png')}}" alt="New beta functionality">
<fieldset> <fieldset>
<div class="pure-control-group"> <div class="pure-control-group">
@@ -223,7 +225,6 @@ Math: {{ 1 + 1 }}") }}
<div class="flex-wrapper" > <div class="flex-wrapper" >
<div id="browser-steps-ui" class="noselect"> <div id="browser-steps-ui" class="noselect">
<div class="noselect" id="browsersteps-selector-wrapper" style="width: 100%"> <div class="noselect" id="browsersteps-selector-wrapper" style="width: 100%">
<span class="loader" > <span class="loader" >
<span id="browsersteps-click-start"> <span id="browsersteps-click-start">
@@ -245,15 +246,16 @@ Math: {{ 1 + 1 }}") }}
</div> </div>
</fieldset> </fieldset>
{% else %} {% else %}
<span class="pure-form-message-inline"> {# it's configured to use selenium or chrome but system says its not configured #}
{% if not watch_uses_webdriver %} {{ playwright_warning() }}
{{ only_webdriver_type_watches_warning() }} {% if system_has_webdriver_configured %}
{% endif %} <strong>Selenium/Webdriver cant be used here because it wont fetch screenshots reliably.</strong>
{% if not playwright_enabled %} {% endif %}
{{ playwright_warning() }}
{% endif %}
</span>
{% endif %} {% endif %}
{% else %}
{# "This functionality needs chrome.." #}
{{ only_playwright_type_watches_warning() }}
{% endif %}
</div> </div>
@@ -262,7 +264,7 @@ Math: {{ 1 + 1 }}") }}
<div class="pure-control-group inline-radio"> <div class="pure-control-group inline-radio">
{{ render_checkbox_field(form.notification_muted) }} {{ render_checkbox_field(form.notification_muted) }}
</div> </div>
{% if watch_uses_webdriver %} {% if watch_needs_selenium_or_playwright %}
<div class="pure-control-group inline-radio"> <div class="pure-control-group inline-radio">
{{ render_checkbox_field(form.notification_screenshot) }} {{ render_checkbox_field(form.notification_screenshot) }}
<span class="pure-form-message-inline"> <span class="pure-form-message-inline">
@@ -379,13 +381,15 @@ Math: {{ 1 + 1 }}") }}
<fieldset> <fieldset>
<div class="pure-control-group"> <div class="pure-control-group">
{% if playwright_enabled and watch_uses_webdriver %} {% if watch_needs_selenium_or_playwright %}
{% if system_has_playwright_configured %}
<span class="pure-form-message-inline" id="visual-selector-heading"> <span class="pure-form-message-inline" id="visual-selector-heading">
The Visual Selector tool lets you select the <i>text</i> elements that will be used for the change detection. It automatically fills-in the filters in the "CSS/JSONPath/JQ/XPath Filters" box of the <a href="#filters-and-triggers">Filters & Triggers</a> tab. Use <strong>Shift+Click</strong> to select multiple items. The Visual Selector tool lets you select the <i>text</i> elements that will be used for the change detection. It automatically fills-in the filters in the "CSS/JSONPath/JQ/XPath Filters" box of the <a href="#filters-and-triggers">Filters & Triggers</a> tab. Use <strong>Shift+Click</strong> to select multiple items.
</span> </span>
<div id="selector-header"> <div id="selector-header">
<a id="clear-selector" class="pure-button button-secondary button-xsmall" style="font-size: 70%">Clear selection</a> <a id="clear-selector" class="pure-button button-secondary button-xsmall" style="font-size: 70%">Clear selection</a>
<!-- visual selector IMG will try to load, it will either replace this or on error replace it with some handy text -->
<i class="fetching-update-notice" style="font-size: 80%;">One moment, fetching screenshot and element information..</i> <i class="fetching-update-notice" style="font-size: 80%;">One moment, fetching screenshot and element information..</i>
</div> </div>
<div id="selector-wrapper" style="display: none"> <div id="selector-wrapper" style="display: none">
@@ -397,13 +401,16 @@ Math: {{ 1 + 1 }}") }}
</div> </div>
<div id="selector-current-xpath" style="overflow-x: hidden"><strong>Currently:</strong>&nbsp;<span class="text">Loading...</span></div> <div id="selector-current-xpath" style="overflow-x: hidden"><strong>Currently:</strong>&nbsp;<span class="text">Loading...</span></div>
{% else %} {% else %}
{% if not watch_uses_webdriver %} {# The watch needed chrome but system says that playwright is not ready #}
{{ only_webdriver_type_watches_warning() }} {{ playwright_warning() }}
{% endif %}
{% if not playwright_enabled %}
{{ playwright_warning() }}
{% endif %}
{% endif %} {% endif %}
{% if system_has_webdriver_configured %}
<strong>Selenium/Webdriver cant be used here because it wont fetch screenshots reliably.</strong>
{% endif %}
{% else %}
{# "This functionality needs chrome.." #}
{{ only_playwright_type_watches_warning() }}
{% endif %}
</div> </div>
</fieldset> </fieldset>
</div> </div>
@@ -443,10 +450,6 @@ Math: {{ 1 + 1 }}") }}
</tr> </tr>
</tbody> </tbody>
</table> </table>
<h4>Text similarity</h4>
<p><strong>Levenshtein Distance</strong> - Last 2 snapshots: {{ lev_info }}</p>
<p style="max-width: 80%; font-size: 80%"><strong>Levenshtein Distance</strong> Calculates the minimum number of insertions, deletions, and substitutions required to change one text into the other.</p>
{% if watch.history_n %} {% if watch.history_n %}
<p> <p>
<a href="{{url_for('ui.ui_edit.watch_get_latest_html', uuid=uuid)}}" class="pure-button button-small">Download latest HTML snapshot</a> <a href="{{url_for('ui.ui_edit.watch_get_latest_html', uuid=uuid)}}" class="pure-button button-small">Download latest HTML snapshot</a>

View File

@@ -74,11 +74,6 @@ def test_check_basic_change_detection_functionality(client, live_server, measure
res = client.get(url_for("ui.ui_edit.watch_get_latest_html", uuid=uuid)) res = client.get(url_for("ui.ui_edit.watch_get_latest_html", uuid=uuid))
assert b'which has this one new line' in res.data assert b'which has this one new line' in res.data
# Check the 'levenshtein' distance calc showed something useful
res = client.get(url_for("ui.ui_edit.edit_page", uuid=uuid))
assert b'Last 2 snapshots: 17' in res.data
# Now something should be ready, indicated by having a 'unviewed' class # Now something should be ready, indicated by having a 'unviewed' class
res = client.get(url_for("watchlist.index")) res = client.get(url_for("watchlist.index"))
assert b'unviewed' in res.data assert b'unviewed' in res.data

View File

@@ -9,15 +9,20 @@ services:
# - ./proxies.json:/datastore/proxies.json # - ./proxies.json:/datastore/proxies.json
# environment: # environment:
# Default listening port, can also be changed with the -p option # Default listening port, can also be changed with the -p option (not to be confused with ports: below)
# - PORT=5000 # - PORT=5000
# #
# Log levels are in descending order. (TRACE is the most detailed one) # Log levels are in descending order. (TRACE is the most detailed one)
# Log output levels: TRACE, DEBUG(default), INFO, SUCCESS, WARNING, ERROR, CRITICAL # Log output levels: TRACE, DEBUG(default), INFO, SUCCESS, WARNING, ERROR, CRITICAL
# - LOGGER_LEVEL=TRACE # - LOGGER_LEVEL=TRACE
# #
# Alternative WebDriver/selenium URL, do not use "'s or 's! #
# - WEBDRIVER_URL=http://browser-chrome:4444/wd/hub # Uncomment below and the "sockpuppetbrowser" to use a real Chrome browser (It uses the "playwright" protocol)
# - PLAYWRIGHT_DRIVER_URL=ws://browser-sockpuppet-chrome:3000
#
#
# Alternative WebDriver/selenium URL, do not use "'s or 's! (old, deprecated, does not support screenshots very well)
# - WEBDRIVER_URL=http://browser-selenium-chrome:4444/wd/hub
# #
# WebDriver proxy settings webdriver_proxyType, webdriver_ftpProxy, webdriver_noProxy, # WebDriver proxy settings webdriver_proxyType, webdriver_ftpProxy, webdriver_noProxy,
# webdriver_proxyAutoconfigUrl, webdriver_autodetect, # webdriver_proxyAutoconfigUrl, webdriver_autodetect,
@@ -25,9 +30,6 @@ services:
# #
# https://selenium-python.readthedocs.io/api.html#module-selenium.webdriver.common.proxy # https://selenium-python.readthedocs.io/api.html#module-selenium.webdriver.common.proxy
# #
# Alternative target "Chrome" Playwright URL, do not use "'s or 's!
# "Playwright" is a driver/librarythat allows changedetection to talk to a Chrome or similar browser.
# - PLAYWRIGHT_DRIVER_URL=ws://sockpuppetbrowser:3000
# #
# Playwright proxy settings playwright_proxy_server, playwright_proxy_bypass, playwright_proxy_username, playwright_proxy_password # Playwright proxy settings playwright_proxy_server, playwright_proxy_bypass, playwright_proxy_username, playwright_proxy_password
# #
@@ -43,7 +45,7 @@ services:
# Base URL of your changedetection.io install (Added to the notification alert) # Base URL of your changedetection.io install (Added to the notification alert)
# - BASE_URL=https://mysite.com # - BASE_URL=https://mysite.com
# Respect proxy_pass type settings, `proxy_set_header Host "localhost";` and `proxy_set_header X-Forwarded-Prefix /app;` # Respect proxy_pass type settings, `proxy_set_header Host "localhost";` and `proxy_set_header X-Forwarded-Prefix /app;`
# More here https://github.com/dgtlmoon/changedetection.io/wiki/Running-changedetection.io-behind-a-reverse-proxy-sub-directory # More here https://github.com/dgtlmoon/changedetection.io/wiki/Running-changedetection.io-behind-a-reverse-proxy
# - USE_X_SETTINGS=1 # - USE_X_SETTINGS=1
# #
# Hides the `Referer` header so that monitored websites can't see the changedetection.io hostname. # Hides the `Referer` header so that monitored websites can't see the changedetection.io hostname.
@@ -86,8 +88,8 @@ services:
# Sockpuppetbrowser is basically chrome wrapped in an API for allowing fast fetching of web-pages. # Sockpuppetbrowser is basically chrome wrapped in an API for allowing fast fetching of web-pages.
# RECOMMENDED FOR FETCHING PAGES WITH CHROME, be sure to enable the "PLAYWRIGHT_DRIVER_URL" env variable in the main changedetection container # RECOMMENDED FOR FETCHING PAGES WITH CHROME, be sure to enable the "PLAYWRIGHT_DRIVER_URL" env variable in the main changedetection container
# sockpuppetbrowser: # browser-sockpuppet-chrome:
# hostname: sockpuppetbrowser # hostname: browser-sockpuppet-chrome
# image: dgtlmoon/sockpuppetbrowser:latest # image: dgtlmoon/sockpuppetbrowser:latest
# cap_add: # cap_add:
# - SYS_ADMIN # - SYS_ADMIN
@@ -102,14 +104,18 @@ services:
# Used for fetching pages via Playwright+Chrome where you need Javascript support. # Used for fetching pages via Playwright+Chrome where you need Javascript support.
# Note: Works well but is deprecated, does not fetch full page screenshots (doesnt work with Visual Selector) # Note: Works well but is deprecated, does not fetch full page screenshots (doesnt work with Visual Selector)
# Does not report status codes (200, 404, 403) and other issues # Does not report status codes (200, 404, 403) and other issues
# browser-chrome: # browser-selenium-chrome:
# hostname: browser-chrome # hostname: browser-selenium-chrome
# image: selenium/standalone-chrome:4 # image: selenium/standalone-chrome:4
# environment: # environment:
# - VNC_NO_PASSWORD=1 # - VNC_NO_PASSWORD=1
# - SCREEN_WIDTH=1920 # - SCREEN_WIDTH=1920
# - SCREEN_HEIGHT=1080 # - SCREEN_HEIGHT=1080
# - SCREEN_DEPTH=24 # - SCREEN_DEPTH=24
# CHROME_OPTIONS: |
# --window-size=1280,1024
# --headless
# --disable-gpu
# volumes: # volumes:
# # Workaround to avoid the browser crashing inside a docker container # # Workaround to avoid the browser crashing inside a docker container
# # See https://github.com/SeleniumHQ/docker-selenium#quick-start # # See https://github.com/SeleniumHQ/docker-selenium#quick-start

File diff suppressed because one or more lines are too long

View File

@@ -5,13 +5,13 @@
<meta name="description" content="Manage your changedetection.io watches via API, requires the `x-api-key` header which is found in the settings UI."> <meta name="description" content="Manage your changedetection.io watches via API, requires the `x-api-key` header which is found in the settings UI.">
<meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<link href="assets/bootstrap.min.css?v=1701595483622" rel="stylesheet" media="screen"> <link href="assets/bootstrap.min.css?v=1744573753999" rel="stylesheet" media="screen">
<link href="assets/prism.css?v=1701595483622" rel="stylesheet" /> <link href="assets/prism.css?v=1744573753999" rel="stylesheet" />
<link href="assets/main.css?v=1701595483622" rel="stylesheet" media="screen, print"> <link href="assets/main.css?v=1744573753999" rel="stylesheet" media="screen, print">
<link href="assets/favicon.ico?v=1701595483622" rel="icon" type="image/x-icon"> <link href="assets/favicon.ico?v=1744573753999" rel="icon" type="image/x-icon">
<link href="assets/apple-touch-icon.png?v=1701595483622" rel="apple-touch-icon" sizes="180x180"> <link href="assets/apple-touch-icon.png?v=1744573753999" rel="apple-touch-icon" sizes="180x180">
<link href="assets/favicon-32x32.png?v=1701595483622" rel="icon" type="image/png" sizes="32x32"> <link href="assets/favicon-32x32.png?v=1744573753999" rel="icon" type="image/png" sizes="32x32">
<link href="assets/favicon-16x16.png?v=1701595483622" rel="icon" type="image/png" sizes="16x16"> <link href="assets/favicon-16x16.png?v=1744573753999" rel="icon" type="image/png" sizes="16x16">
</head> </head>
<body class="container-fluid"> <body class="container-fluid">
@@ -928,6 +928,6 @@
</div> </div>
</div> </div>
<script src="assets/main.bundle.js?v=1701595483622"></script> <script src="assets/main.bundle.js?v=1744573753999"></script>
</body> </body>
</html> </html>

View File

@@ -68,8 +68,6 @@ openpyxl
jq~=1.3; python_version >= "3.8" and sys_platform == "darwin" jq~=1.3; python_version >= "3.8" and sys_platform == "darwin"
jq~=1.3; python_version >= "3.8" and sys_platform == "linux" jq~=1.3; python_version >= "3.8" and sys_platform == "linux"
levenshtein
# playwright is installed at Dockerfile build time because it's not available on all platforms # playwright is installed at Dockerfile build time because it's not available on all platforms
pyppeteer-ng==2.0.0rc9 pyppeteer-ng==2.0.0rc9