lee/changedetection.io
1
0
Fork 0
mirror of https://github.com/dgtlmoon/changedetection.io.git synced 2025-12-11 18:45:34 +00:00
Code Issues Actions 7 Packages Projects Releases Wiki Activity

@todo Comment - BS4's element.get_text() seems to lose the indentation format no-matter what

Browse Source
This commit is contained in:
dgtlmoon
2021-06-23 12:49:53 +10:00
parent 47f7698b32
commit e3a9847f74
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
backend/fetch_site_status.py
View File

@@ -90,6 +90,8 @@ class perform_site_check():
stripped_text_from_html = "" stripped_text_from_html = ""
for item in soup.select(css_filter): for item in soup.select(css_filter):
# By default, bs4's get_text will lump the text together # By default, bs4's get_text will lump the text together
# BS4's element strip() will lose the indentation format, I've tried using a space as separator, setting strip=False etc, but doesnt help
# @todo ideas? if you compare the css_filtered output to non-filtered snapshot it will always lose the indentation/format
text = str(item.get_text(separator="\n", strip=True)).strip() text = str(item.get_text(separator="\n", strip=True)).strip()
# Try to cut back on excessive linefeeds if there are any # Try to cut back on excessive linefeeds if there are any
text = text.replace("\n\n","\n") text = text.replace("\n\n","\n")