CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-07-12 21:01:56 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,293 Commits 2 Branches 448 Tags 17 MiB
5faac9fb46
Commit Graph

144 Commits

Author SHA1 Message Date
Marco Vinciguerra
8bd8fb0859 refactoring of examples
Some checks are pending
Release / Build (push) Waiting to run
Details
Release / Release (push) Blocked by required conditions
Details
2024-07-10 12:19:41 +02:00
Marco Vinciguerra
30ca15ca28
Merge branch 'md_scraper_integration' into integration_markdown
Some checks failed
/ build (3.10) (push) Has been cancelled
Details
2024-06-30 16:58:37 +02:00
Marco Vinciguerra
2804434a9e feat: add integrations for markdown files
Some checks are pending
/ build (3.10) (push) Waiting to run
Details
2024-06-29 13:35:39 +02:00
Marco Vinciguerra
228a1de2be add new force 2024-06-27 18:57:27 +02:00
Marco Vinciguerra
2f02830c81 refactoring of fetch node
Some checks are pending
/ build (3.10) (push) Waiting to run
Details
2024-06-20 13:44:42 +02:00
Marco Perini
080a318ff6 feat(telemetry): add telemetry module 2024-06-17 13:00:33 +02:00
Marco Perini
9b0e62742b changed source to text 2024-06-14 15:24:50 +02:00
Marco Perini
12f4386552
Merge branch 'pre/beta' into 349-problem-with-scrapegraphaigraphspdf_scraper_graphpy 2024-06-14 15:22:43 +02:00
Marco Perini
203de83405 fix(pdf): correctly read .pdf files 2024-06-14 15:20:30 +02:00
Marco Perini
91c5b5af43 fix(multi): updated multi pdf scraper with schema 2024-06-14 14:59:12 +02:00
Marco Perini
283b61fafc docs: better logging 2024-06-13 18:13:47 +02:00
Marco Perini
5d692bff9e feat(schema): merge scripts to follow pydantic schema 2024-06-12 00:48:08 +02:00
Marco Vinciguerra
cb00c4fb17 changed model 2024-06-08 12:22:50 +02:00
Marco Vinciguerra
1981230e6f add multi scraper integration 2024-06-08 12:13:18 +02:00
Marco Vinciguerra
dd2b3a8f59 add examples
Some checks failed
/ build (3.10) (push) Has been cancelled
Details
Release / Build (push) Has been cancelled
Details
Release / Release (push) Has been cancelled
Details
2024-06-05 21:08:00 +02:00
Marco Vinciguerra
4f53b09bf1 add examples for schema
Some checks are pending
/ build (3.10) (push) Waiting to run
Details
Release / Build (push) Waiting to run
Details
Release / Release (push) Blocked by required conditions
Details
2024-06-05 10:43:57 +02:00
Marco Vinciguerra
74fd530914
Merge branch 'pre/beta' into 332-pydantic-schema-validation 2024-06-05 09:05:19 +02:00
Marco Perini
376f758a76 feat(pydantic): added pydantic output schema 2024-06-04 23:07:49 +02:00
Marco Vinciguerra
1dde43cdeb add new examples 2024-06-03 21:03:13 +02:00
Marco Vinciguerra
743dfe1191 add all possible examples 2024-06-03 12:19:43 +02:00
Marco Vinciguerra
1fe49753b9 add openai and oneapi examples 2024-06-01 13:46:15 +02:00
Marco Vinciguerra
4d42d7bfc6 add example 2024-06-01 11:20:24 +02:00
Marco Vinciguerra
3b90ebd9a8 add new examples 2024-05-29 10:41:37 +02:00
Marco Vinciguerra
58dfe9b658 add examples of usage 2024-05-28 09:11:14 +02:00
Marco Vinciguerra
3c7dedf9c8
Merge pull request #305 from VinciGit00/pdf_fix 
Pdf fix
 2024-05-26 15:39:21 +02:00
arsaboo
a7961691df fix(pdf-example): added pdf example and coauthor 2024-05-26 15:09:49 +02:00
Marco Vinciguerra
ecd98b2a45 add sche,a example 2024-05-26 12:15:48 +02:00
Marco Perini
8d76c4b3cb fix(schema): added schema 2024-05-26 10:51:48 +02:00
Marco Perini
a4ee757507
Merge branch 'pre/beta' into pdf_scraper_refactoring 2024-05-25 00:16:38 +02:00
Marco Perini
8d5eb0bb0d fix(local_file): fixed textual input pdf, csv, json and xml graph 2024-05-25 00:13:47 +02:00
Marco Vinciguerra
b913b51cca Merge branch 'logger-integration' into pre/beta 2024-05-24 12:39:14 +02:00
Marco Perini
fc58e2d3a6 feat(smart-scraper-multi): add schema to graphs and created SmartScraperMultiGraph 2024-05-21 13:13:27 +02:00
Marco Vinciguerra
5701afe927 add new import 2024-05-18 10:36:41 +02:00
Marco Perini
58cc903d55 feat(multiple): quick fix working 2024-05-18 05:57:43 +02:00
Marco Perini
bed3eed50c feat(multiple_search): working multiple example 2024-05-18 01:51:12 +02:00
Marco Vinciguerra
05e511e36f add new prompts 2024-05-18 00:22:52 +02:00
Marco Vinciguerra
3453f72397 add graph 2024-05-17 18:23:50 +02:00
VinciGit00
9483afddd3 revert 2024-05-16 12:34:02 +02:00
Mayur Bhosale
1e0b2f7e70
Merge branch 'pre/beta' into nDeep 2024-05-15 23:36:23 +02:00
mayurdb
d60438cc89 Add a n-level deep search support 2024-05-15 23:32:15 +02:00
VinciGit00
ba8a4f7122 removed duplicates 2024-05-15 21:47:39 +02:00
VinciGit00
befa48c912 update lock 2024-05-15 21:28:56 +02:00
Marco Perini
a6e1813ddd fix(fetch_node): bug in handling local files 2024-05-14 16:51:10 +02:00
Marco Perini
fcb3abb01d feat(omni-search): added omni search graph and updated docs 2024-05-14 16:26:43 +02:00
Marco Perini
a296927624 feat(omni-scraper): working OmniScraperGraph with images 2024-05-14 13:46:49 +02:00
Marco Perini
90955ca52f feat(gpt-4o): image to text single node test 2024-05-14 11:43:21 +02:00
Marco Perini
367dea5cbd
Merge branch 'pre/beta' into feat/parallel-node-execution 2024-05-13 23:50:40 +02:00
Marco Perini
dedc733047 fix(asyncio): replaced deepcopy with copy due to serialization problems 2024-05-13 18:46:34 +02:00
VinciGit00
7c91f9f94c add examples for deepseek 2024-05-13 12:56:57 +02:00
Marco Perini
c7ec114274 docs(refactor): changed example 2024-05-13 12:28:03 +02:00
Marco Perini
0c1594737f fix(fetch-node): removed isSoup from default 2024-05-13 12:09:55 +02:00
Marco Perini
7e8acd8e6a
Merge branch 'pre/beta' into fix/fetch-node-proxybroker 2024-05-13 11:17:37 +02:00
Marco Perini
0c36a7ec1f feat: added proxy rotation 2024-05-13 11:04:56 +02:00
Marco Perini
5d6d996e8f fix(proxy-rotation): removed max_shape duplicate 2024-05-13 07:26:43 +02:00
Marco Perini
1e9a564616 fix(proxy-rotation): removed duplicated arg and passed the loader_kwarhs correctly to the node 2024-05-12 18:39:03 +02:00
mayurdb
8f1fbe7e10 minor changes 2024-05-11 16:52:36 +05:30
mayurdb
df271b6451 Add search link node that can find out relevant links in the webpage 2024-05-11 16:39:55 +05:30
Eric Page
0683e78e78
Merge branch 'pre/beta' into fix-GenerateScraperGraph 2024-05-11 01:59:28 +02:00
Eric Page
67d8fec6b3 Minor typo fix for clarity 2024-05-11 00:05:13 +02:00
Marco Perini
186c0d035d fix(examples): openai std examples 2024-05-08 14:56:44 +02:00
VinciGit00
2258fe5ac0 add new search graph examples 2024-05-06 22:31:48 +02:00
VinciGit00
d9a4ab2d05 Delete custom_search_graph.py 2024-05-06 09:47:29 +02:00
Marco Perini
930adb38f2 feat(node): multiple url search in SearchGraph + fixes 
Implemented GraphIteratorNode and MergeAnswersNode to create multiple istances of a graph and merge the scraped content from multiple pages
 2024-05-06 00:30:09 +02:00
Marco Perini
dbb614a8dd feat: multiple graph instances 2024-05-05 23:51:04 +02:00
Marco Perini
16f53c52fd add example custom search graph 2024-05-05 22:03:55 +02:00
Marco Perini
84fcb44aaa feat: fixed custom_graphs example and robots_node 2024-05-05 22:02:24 +02:00
Eric Page
cc27b21c3f
Merge branch 'pre/beta' into pass-common-params-graph 2024-05-05 16:18:32 +02:00
Eric Page
3ae2ea1dbd Miscellaneous "llm" -> "llm_model" refactors 2024-05-05 15:58:50 +02:00
Marco Perini
aa9271e7bc feat: Azure implementation + embeddings refactoring 2024-05-03 21:40:01 +02:00
Marco Vinciguerra
a2d5c75f10
Merge branch 'pre/beta' into main 2024-05-02 17:55:21 +02:00
Marco Perini
c11fc28896 docs: refactor examples 2024-05-02 00:44:19 +02:00
VinciGit00
276ac8b40c changed name 2024-05-01 20:17:01 +02:00
VinciGit00
7c1b5796d7 add csv scraper 2024-05-01 17:20:04 +02:00
EURAC\marperini
2dd7817cfb feat: added verbose flag to suppress print statements 2024-04-30 15:31:57 +02:00
EURAC\marperini
8fba7e5490 feat(refactor): changed variable names 2024-04-30 14:28:30 +02:00
VinciGit00
da2c82a2a2 add json and xml scraper 2024-04-30 10:52:22 +02:00
VinciGit00
f8917320b3 add xml_example 2024-04-29 16:15:24 +02:00
VinciGit00
deb920a33e fixing json and example 2024-04-29 16:11:57 +02:00
VinciGit00
45b2317ab7 add json examples 2024-04-29 11:16:48 +02:00
VinciGit00
cd8d3e7a4f refactoring of the graphs 2024-04-26 11:08:56 +02:00
VinciGit00
e714a59c2e refactoring of engine 2024-04-25 19:22:12 +02:00
VinciGit00
1056cff50d fixed bug on custom graph 2024-04-25 09:13:48 +02:00
VinciGit00
a73496df9b refactoring of nodes 2024-04-24 21:13:27 +02:00
VinciGit00
3f95801737 Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-04-19 10:36:40 +02:00
VinciGit00
bd004db80a add benchmarks and benchmarks results 2024-04-17 20:30:24 +02:00
VinciGit00
876223da56 add generateScraperBranch 
Co-Authored-By: Matteo Mei <52063123+FattiMei@users.noreply.github.com>
 2024-04-14 13:18:50 +02:00
VinciGit00
e0978c53b3 Merge branch 'main' into ollama_integration 2024-04-12 13:00:35 +02:00
VinciGit00
60a2377bff add inputs 2024-04-12 12:53:54 +02:00
VinciGit00
87d911b7bb removed files and refactoring of actions 2024-04-11 18:18:22 +02:00
EURAC\marperini
74ec928f96 added execution info of the graph, time, price etc 2024-04-11 14:29:32 +02:00
VinciGit00
f3766955fd add examples 2024-04-11 12:17:04 +02:00
VinciGit00
ee533b8d74 add examples for local models and gemini models 2024-04-09 11:43:51 +02:00
VinciGit00
0ffefdd170 refactoring if examples ffolder 2024-04-08 22:28:55 +02:00
VinciGit00
958b2a4a79 add new version + refactoring of the code 2024-04-08 18:24:59 +02:00