lee/Scraperr
1
0
Fork 0
mirror of https://github.com/jaypyles/Scraperr.git synced 2025-12-14 11:46:17 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

feat: add agent mode (#81)

Browse Source 
* chore: wip agent mode

* wip: add agent mode frontend

* wip: add agent mode frontend

* chore: cleanup code

* chore: cleanup code

* chore: cleanup code
This commit is contained in:
Jayden Pyles
2025-05-19 20:44:41 -05:00
committed by GitHub
parent d602d3330a
commit 5ebd96b62b
33 changed files with 869 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
api/backend/job/scraping/scraping_utils.py
View File

@@ -30,3 +30,16 @@ async def scrape_content(
await collect_media_utils(id, page)
return html
def clean_format_characters(text: str) -> str:
text = text.strip()
text = text.replace("\n", " ")
text = text.replace("\t", " ")
text = text.replace("\r", " ")
text = text.replace("\f", " ")
text = text.replace("\v", " ")
text = text.replace("\b", " ")
text = text.replace("\a", " ")
return text