CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-06-28 21:01:55 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,485 Commits 2 Branches 448 Tags 17 MiB
5cdf0550fe
Commit Graph

52 Commits

Author SHA1 Message Date
Michele_Zenoni
7da7bfe338 fix: improved links extraction for parse_node, resolves #822 2024-11-24 14:44:48 +01:00
Marco Vinciguerra
3b7b701a89 feat: refactoring of mdscraper 2024-10-11 08:32:41 +02:00
Marco Vinciguerra
560f079d4c refactoring of the code 2024-09-28 09:02:20 +02:00
Marco Vinciguerra
ceede46673 fix: parse_node 2024-09-24 15:27:20 +02:00
Marco Vinciguerra
c5a3f893f1 refactoring of node names 2024-09-22 10:35:43 +02:00
Lorenzo Paleari
c3d1b7c200
fix: OmniScraerGraph working. 
Added url scraping capability to ParseNode
 2024-09-13 01:47:39 +02:00
Tom Robinson
da9726f738 updates to tokenization for #651 to implement for mistral and ollama 2024-09-12 08:28:30 +01:00
Marco Vinciguerra
1a7f21fbf3 feat: removed semchunk and used tikton 2024-09-10 14:03:52 +02:00
Marco Vinciguerra
380174d490 add chunking functionn 2024-09-10 13:52:15 +02:00
Marco Vinciguerra
947ebd2895 fix: parse node
Some checks failed
/ build (push) Has been cancelled
Details
2024-09-10 08:41:08 +02:00
Marco Vinciguerra
f2bb22d8e9 fix: temporary fix for parse_node 2024-09-09 11:42:33 +02:00
Marco Vinciguerra
fc738cacac Update parse_node.py 2024-09-08 11:54:11 +02:00
Lorenzo Paleari
66a3b6d6a3
fix: Parse Node scraping link and img urls allowing OmniScraper to work 2024-09-02 12:53:10 +02:00
Tom Robinson
a8b0e4a359 updated token calculation on parsenode 2024-09-02 08:01:21 +01:00
Marco Vinciguerra
f7ba1f30de refactoring of the code 2024-08-23 11:33:22 +02:00
Federico Aguzzi
b48ee825ee Merge branch 'pre/beta' into support_structured_output_shema_openai 2024-08-19 13:31:01 +02:00
Federico Aguzzi
683bf57d89 fix(ParseNode): leave room for LLM reply in context window 2024-08-19 11:33:09 +02:00
Marco Vinciguerra
8b8d8f09b7 refactoring of the code according to pylint style
Some checks are pending
/ build (push) Waiting to run
Details
Release / Build (push) Waiting to run
Details
Release / Release (push) Blocked by required conditions
Details
2024-08-18 11:51:34 +02:00
Marco Vinciguerra
faef3186f7 fix: model count 2024-08-16 17:38:55 +02:00
Marco Vinciguerra
de1ec250ef refactoring pyproject.toml
Some checks are pending
/ build (push) Waiting to run
Details
Release / Build (push) Waiting to run
Details
Release / Release (push) Blocked by required conditions
Details 
Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>
 2024-08-11 18:04:31 +02:00
Marco Vinciguerra
8b2c266aff refactoring of the code 
Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>
 2024-08-10 17:44:35 +02:00
Federico Aguzzi
5ec2de9e1a fix(chunking): count tokens from words instead of characters 
closes #513
 2024-08-08 10:50:52 +02:00
Marco Vinciguerra
9355507a2d feat: refactoring of the code 2024-08-02 12:00:00 +02:00
Marco Vinciguerra
09256f7b11 fix: parse node 2024-07-22 16:15:38 +02:00
Marco Vinciguerra
71f894eee3
fix: parse_html node have a bug 2024-07-22 16:01:27 +02:00
Marco Vinciguerra
032a491605 Update parse_node.py
Some checks failed
/ build (push) Has been cancelled
Details
2024-07-17 23:06:05 +02:00
Marco Vinciguerra
07f1e23d23 fix: parse_node 2024-07-17 22:58:21 +02:00
Marco Vinciguerra
68f58cc4dd refactoring of generate answer node 2024-07-17 22:41:49 +02:00
Marco Vinciguerra
ed2af51501 update the chunk size
Some checks are pending
/ build (3.10) (push) Waiting to run
Details
2024-07-02 12:03:08 +02:00
Marco Perini
203de83405 fix(pdf): correctly read .pdf files 2024-06-14 15:20:30 +02:00
Marco Vinciguerra
e6c7940a57 feat: add Parse_Node 2024-06-12 12:29:14 +02:00
Marco Vinciguerra
e1f045b280 feat: add new chunking function 2024-06-08 11:44:09 +02:00
Marco Vinciguerra
b913b51cca Merge branch 'logger-integration' into pre/beta 2024-05-24 12:39:14 +02:00
Federico Minutoli
c251cc45d3 fix(node-logging): use centralized logger in each node for logging 2024-05-24 01:09:49 +02:00
Marco Perini
fc58e2d3a6 feat(smart-scraper-multi): add schema to graphs and created SmartScraperMultiGraph 2024-05-21 13:13:27 +02:00
VinciGit00
29d284e497 Merge branch 'main' into logger-integration 2024-05-15 15:28:20 +02:00
VinciGit00
05890835f5 refactoring of loggers 2024-05-15 10:54:53 +02:00
VinciGit00
e53766b16e feat: add logger integration 2024-05-14 15:20:39 +02:00
Marco Perini
a296927624 feat(omni-scraper): working OmniScraperGraph with images 2024-05-14 13:46:49 +02:00
Eric Page
0683e78e78
Merge branch 'pre/beta' into fix-GenerateScraperGraph 2024-05-11 01:59:28 +02:00
Eric Page
40884747c7 Added parse_html option in parse_node 2024-05-11 00:32:01 +02:00
Marco Perini
186c0d035d fix(examples): openai std examples 2024-05-08 14:56:44 +02:00
Marco Perini
dbb614a8dd feat: multiple graph instances 2024-05-05 23:51:04 +02:00
Marco Perini
1409797475 docs: refactor nodes docstrings 2024-05-01 23:17:57 +02:00
EURAC\marperini
2dd7817cfb feat: added verbose flag to suppress print statements 2024-04-30 15:31:57 +02:00
EURAC\marperini
dee1a42629 fixed token models, added mistral support 2024-04-08 15:21:06 +02:00
VinciGit00
1afb950751 refactoring of code, add example on the readme 2024-03-18 14:35:47 +01:00
Perinim
52934bf007 implemented graph_config, fixed smart_scraper and speech graph 2024-03-17 20:35:04 +01:00
Perinim
875a7cc4f0 refactored RagNode and GenerateAnswerNode 2024-03-17 16:48:58 +01:00
Perinim
3585cd81e5 refactor fetch node 2024-03-17 12:02:35 +01:00