CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-06-25 21:11:11 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update documentation

Browse Source
This commit is contained in:
mayurdb 2024-05-11 16:57:22 +05:30
parent 8f1fbe7e10
commit 9a67a26cd3
1 changed files with 2 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
scrapegraphai/nodes/search_link_node.py
View File

@ -5,7 +5,6 @@ SearchLinkNode Module
# Imports from standard library
from typing import List, Optional
from tqdm import tqdm
from bs4 import BeautifulSoup
# Imports from Langchain
@ -19,15 +18,9 @@ from .base_node import BaseNode
class SearchLinkNode(BaseNode):
"""
A node that can filter out the relevant links in the webpage content.
Node expects the aleready scrapped information and hence it is expected
A node that can filter out the relevant links in the webpage content for the user prompt.
Node expects the aleready scrapped links on the webpage and hence it is expected
that this node be used after the FetchNode.
For the links which are not incomplete and hence in-navigable. the node will complete
the url and return,
For example: link /projects/rotary-pendulum-rl/ on https://perinim.github.io/projects/,
would be augmented to return https://perinim.github.io/projects/rotary-pendulum-rl/
Attributes:
llm_model: An instance of the language model client used for generating answers.