CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-07-12 21:01:56 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #239 from mayurdb/deepScrapeFix

Browse Source
This commit is contained in:
Marco Vinciguerra 2024-05-14 17:08:58 +02:00 committed by GitHub
commit d76badd133
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 10 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
scrapegraphai/nodes/search_link_node.py
View File

@ -71,8 +71,16 @@ class SearchLinkNode(BaseNode):
prompt_relevant_links = """
You are a website scraper and you have just scraped the following content from a website.
Content: {content}
You are now asked to find all relevant links from the extracted webpage content related
to prompt {user_prompt}. Only pick links which are valid and relevant
You are now tasked with identifying all hyper links within the content that are potentially
relevant to the user task: {user_prompt}
Assume relevance broadly, including any links that might be related or potentially useful
in relation to the task.
Please list only valid URLs and make sure to err on the side of inclusion if it's uncertain
whether the content at the link is directly relevant.
Output only a list of relevant links in the format:
[
"link1",