CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-07-12 21:01:56 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #2 from VinciGit00/pre/beta

Browse Source 
fix: refactoring of fetch_node.py
This commit is contained in:
Matteo Vedovati 2024-09-20 21:52:52 +02:00 committed by GitHub
commit 2616289e07
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 16 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
scrapegraphai/nodes/fetch_node.py
View File

@ -316,21 +316,7 @@ class FetchNode(BaseNode):
compressed_document = [
Document(page_content=parsed_content, metadata={"source": "html file"})
]
return self.update_state(state, compressed_document)
def update_state(self, state, compressed_document):
"""
Updates the state with the output data from the node.
Args:
state (dict): The current state of the graph.
compressed_document (List[Document]): The compressed document content fetched
by the node.
Returns:
dict: The updated state with the output data.
"""
state["original_html"] = document
state.update({self.output[0]: compressed_document,})
return state