Commit Graph

53 Commits

Author SHA1 Message Date
dgtlmoon
ab0b85d088 Unify safe URL checking to the one function, strengthen tests and filters (#3564)
Some checks failed
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (alpine) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/amd64 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v7 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm/v8 (main) (push) Has been cancelled
ChangeDetection.io Container Build Test / Build linux/arm64 (main) (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-10-28 13:24:37 +01:00
dgtlmoon
f304ae19db Adding small amount of cache to common functions (#3565) 2025-10-28 10:43:20 +01:00
dgtlmoon
2116b2cb93 CVE-2025-62780 - Stored XSS in Watch update via API 2025-10-28 10:09:30 +01:00
dgtlmoon
a51614f83d Be sure that default namespaces are registered (#3535) 2025-10-24 18:04:47 +02:00
dgtlmoon
785dabd071 Empty "ignore text" lines could break ignore text and prevent changes from being detected (#3524)
Some checks failed
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-10-21 15:54:13 +02:00
dgtlmoon
b59ce190ac Ensure JSON is always correctly reformatted with padding (#3485 #3482) 2025-10-10 16:00:32 +02:00
dgtlmoon
80be1a30f2 No need to reformat/reprocess content in the case that no filters were found (#3484, #3483)
Some checks failed
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built 📦 package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-10-10 13:44:49 +02:00
dgtlmoon
3b9d19df43 Refactoring text/html difference processor (#3475) 2025-10-09 18:30:53 +02:00
dgtlmoon
9db7fb83eb Always extract page <title>, {{watch_title}} added to notification body tokens (#3415)
Some checks failed
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built 📦 package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
2025-09-10 14:52:41 +02:00
Emmanuel Ferdman
bb7f7f473b Resolve warnings of bs4 library (#3187)
Some checks failed
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
ChangeDetection.io Container Build Test / test-container-build (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built 📦 package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-05-09 14:35:35 +02:00
dgtlmoon
a9ca511004 Revert memory strategy change for html_to_text (Was hanging under high concurrency setups)
Some checks failed
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built 📦 package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-05-09 09:44:02 +02:00
dgtlmoon
3a583a4e5d Memory management - Run HTML to text in sub process, a few more cleanups to playwright (#3110)
Some checks failed
Build and push containers / metadata (push) Has been cancelled
Build and push containers / build-push-containers (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built 📦 package works basically. (push) Has been cancelled
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Has been cancelled
ChangeDetection.io Container Build Test / test-container-build (push) Has been cancelled
ChangeDetection.io App Test / lint-code (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-10 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-11 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-12 (push) Has been cancelled
ChangeDetection.io App Test / test-application-3-13 (push) Has been cancelled
2025-04-11 18:18:29 +02:00
dgtlmoon
5f43d988a3 Fetching - Small improvement memory handling in detecting price information (saves ~10Mb) 2025-04-11 10:11:59 +02:00
MoshiMoshi0
4e6e680d79 Filters - Support multi line regex (#2889) 2025-04-09 15:06:08 +02:00
dgtlmoon
55da48f719 Re #2945 - Handle/Strip UTF-8 ByteOrderMark in JSON strings correctly (fixes "Exception: No parsable JSON found in this document" error) (#2947)
Some checks are pending
Build and push containers / metadata (push) Waiting to run
Build and push containers / build-push-containers (push) Waiting to run
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Build distribution 📦 (push) Waiting to run
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Test the built 📦 package works basically. (push) Blocked by required conditions
Publish Python 🐍distribution 📦 to PyPI and TestPyPI / Publish Python 🐍 distribution 📦 to PyPI (push) Blocked by required conditions
ChangeDetection.io App Test / lint-code (push) Waiting to run
ChangeDetection.io App Test / test-application-3-10 (push) Blocked by required conditions
ChangeDetection.io App Test / test-application-3-11 (push) Blocked by required conditions
ChangeDetection.io App Test / test-application-3-12 (push) Blocked by required conditions
ChangeDetection.io App Test / test-application-3-13 (push) Blocked by required conditions
2025-02-07 22:19:23 +01:00
dgtlmoon
87510becb5 Filters - Process all CSS and XPath 'subtract' selectors in a single pass to prevent index shifting and reference loss during DOM manipulation. (#2754) 2024-10-30 12:00:53 +01:00
dgtlmoon
5a768d7db3 UTF-8 handling fixes, Improvements to whitespace filtering (#2691) 2024-10-10 14:59:39 +02:00
Michael McMillan
dc936a2e8a Filters - Add support for also removing HTML elements using XPath selectors (#2632) 2024-09-17 22:43:04 +02:00
dgtlmoon
7f2fa20318 Small memory allocation fixes (#2625) 2024-09-11 14:51:32 +02:00
dgtlmoon
cffb6d748c Restock & Price monitor - Huge refactor, set upper and lower price alert limits, set % change, follow the prices and restock amounts directly in the watch-overview list 2024-07-12 17:09:42 +02:00
dgtlmoon
99b0935b42 Product checks - Just a basic string check is far more efficient for suggestion price/restock check plugin (#2488) 2024-07-12 14:46:36 +02:00
Hritik Vijay
ffd160ce0e Filters - Implement jqraw: filter (use this to output nicer JSON format when selecting/filtering by JSON) (#2430) 2024-06-21 13:31:03 +02:00
dgtlmoon
3ae9bfa6f9 Bug fix - further work on lxml filter extract (#2313 #2312 #2317) 2024-04-18 11:53:45 +02:00
dgtlmoon
74707909f1 Bug fix for newer lxml module - module 'lxml.etree' has no attribute '_ElementStringResult' - reimplement _ElementStringResult (#2313 #2312) 2024-04-17 19:55:45 +02:00
Constantin Hong
26931e0167 feature: Support XPath2.0 to 3.1 (#1774) 2023-11-13 16:42:21 +01:00
dgtlmoon
7220afab0a RSS fetch - RSS field <title> was not rendering as text correctly, added workaround #1879 2023-10-19 16:42:05 +02:00
dgtlmoon
f707c914b6 RSS Fetching - Handle CDATA (commented out text) in RSS correctly, generally handle RSS better (#1866) 2023-10-17 18:34:19 +02:00
dgtlmoon
ceac8c21e4 LD JSON Price followers - Handle incorrectly created LD-JSON price structures better (#1837) 2023-10-04 15:57:55 +02:00
dgtlmoon
2b948c15c1 Backend - Regular expression / string filtering refactor for Python 3.11 and deprecation warnings since Python 3.6 (#1786) 2023-10-03 17:44:27 +02:00
dgtlmoon
40d01acde9 Fix - Regular Expression text in ignore and trigger were not processing correctly, also refactored for lower CPU usage (#1747) 2023-09-05 13:07:17 +02:00
Maciej Rapacz
a4e6fd1ec3 Fetcher / Parser - Automatically attempt to extract JSON from document when document contains JSON but could be wrapped in HTML (#1593) 2023-05-30 08:57:17 +02:00
dgtlmoon
d5f574ca17 Notifications - Include triggered text token as {{triggered_text}} in notifications, so you can send just the content that matches. (#1485) 2023-03-21 19:16:13 +01:00
dgtlmoon
6f1eec0d5a Fixing bad linebreak definition </br> in notifications and UI (#1465) 2023-03-12 17:05:34 +01:00
dgtlmoon
b58fd995b5 Automatically offer to track LD+JSON product price data (#1204) 2022-12-08 19:28:20 +01:00
dgtlmoon
359fc48fb4 Filters can now accept a list/multiple filters (#1064) #623 2022-11-03 12:13:54 +01:00
dgtlmoon
32ea1a8721 Windows - JQ - Make library optional so it doesnt break Windows pip installs (#1009) 2022-10-12 09:53:16 +02:00
Yusef Ouda
dfa7fc3a81 Adds support for jq JSON path querying engine (#1001) 2022-10-09 16:12:45 +02:00
dgtlmoon
b1c51c0a65 Enhancement - support xPath text() function filter, for example "//title/text()" in RSS feeds (#778) 2022-07-28 11:50:31 +02:00
dgtlmoon
291700554e Bug fix for alerting when xPath based filters are no longer present (#772) 2022-07-23 19:39:52 +02:00
dgtlmoon
a82fad7059 Send notification when CSS/xPath filter is missing after more than 6 (configurable) attempts (#771) 2022-07-23 17:19:00 +02:00
dgtlmoon
dd0eeca056 Handle simple obfuscations - HomeDepot.com style price obfuscation (#764) 2022-07-20 14:02:22 +02:00
dgtlmoon
e17c2121f7 Fix encoding errors with XPath filters from UTF-8 responses (#619) 2022-05-20 18:07:08 +02:00
dgtlmoon
9809af142d Option to render links as [Some Text ](/link), adds the ability to change-detect on hyperlink changes 2022-04-09 10:35:14 +02:00
dgtlmoon
ffad5cca97 JSON diff/preview should use utf-8 encoding where possible (#465) 2022-03-13 11:37:51 +01:00
Tim Loderhose
d4359c2e67 Add filter to remove elements by CSS rule from HTML before change detection is run (#445) 2022-03-12 13:29:30 +01:00
dgtlmoon
b401998030 Ensure string matching on the ignore filter is always case-INsensitive 2022-02-23 12:01:11 +01:00
dgtlmoon
014fda9058 Ability to visualise trigger and filter rules against the current snapshot on the preview page 2022-02-23 10:49:25 +01:00
Michael
85715120e2 XPath RegularExpression support 2022-02-19 13:40:57 +01:00
dgtlmoon
59d31bf76f XPath support (#355)
* XPath support and minor improvements to form validation
2022-01-05 17:58:07 +01:00
dgtlmoon
f87f7077a6 Better handling of EmptyReply exception, always bump 'last_checked' in the case of an error (#354)
* Better handling of EmptyReply exception, always bump 'last_checked' in the case of an error, adds test
2022-01-05 14:13:30 +01:00