CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-06-23 21:00:30 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

docs: add diagram showing general structure/flow of the library

Browse Source
This commit is contained in:
roncaglionidaniele 2024-05-11 12:27:04 +02:00
parent 3f0e0694f3
commit 13ae9180ac
3 changed files with 11 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
docs/assets/project_overview_diagram.fig Normal file
View File

Binary file not shown.

BIN
docs/assets/project_overview_diagram.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 74 KiB

11
docs/source/introduction/overview.rst
View File

@ -18,3 +18,14 @@ Flexibility and Adaptability
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Traditional web scraping tools often rely on fixed patterns or manual configuration to extract data from web pages. ScrapegraphAI, leveraging the power of LLMs, adapts to changes in website structures, reducing the need for constant developer intervention.
This flexibility ensures that scrapers remain functional even when website layouts change.
Overview
========
With ScrapegraphAI you first construct a pipeline of steps you want to execute by combining nodes into a graph.
Executing the graph takes care of all the steps that are often part of scraping: fetching, parsing etc...
Finally the scraped and processed data gets fed to an LLM which generates a response.
.. image:: ../../assets/project_overview_diagram.png
:align: center
:alt: ScrapegraphAI Overview