CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-07-12 21:01:56 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,293 Commits 2 Branches 448 Tags 17 MiB
5faac9fb46
Commit Graph

427 Commits

Author SHA1 Message Date
VinciGit00
cd8d3e7a4f refactoring of the graphs 2024-04-26 11:08:56 +02:00
VinciGit00
e714a59c2e refactoring of engine 2024-04-25 19:22:12 +02:00
VinciGit00
1056cff50d fixed bug on custom graph 2024-04-25 09:13:48 +02:00
VinciGit00
a73496df9b refactoring of nodes 2024-04-24 21:13:27 +02:00
VinciGit00
9b9a9f204c handling the agent model 2024-04-24 13:13:46 +02:00
Marco Vinciguerra
b8a342ec90
Update Readme.md 2024-04-23 13:21:54 +02:00
Lorenzo Padoan
12dcc5065a ADD benchmarks m2 max llama3 2024-04-23 12:28:51 +02:00
VinciGit00
3c90d2f4fe add info for gemini example 2024-04-22 22:52:40 +02:00
VinciGit00
8aa2cadb00 add integration for llama3 2024-04-21 13:11:10 +02:00
EURAC\marperini
ac099e2763 fixed ollama embedder params 2024-04-21 01:21:09 +02:00
VinciGit00
d3e2eb6ea5 add new benchmarks 2024-04-20 22:49:25 +02:00
VinciGit00
3f95801737 Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-04-19 10:36:40 +02:00
VinciGit00
1107630d27 add new test for script generator 2024-04-18 10:39:53 +02:00
Lorenzo Padoan
625b0d7498 ADD benchmark of my lovely mac 💝💫⭐💖💗💕 † 
💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕💝💫⭐💖💗💕
 2024-04-17 22:25:08 +02:00
VinciGit00
bd004db80a add benchmarks and benchmarks results 2024-04-17 20:30:24 +02:00
Marco Perini
edc439fd01
Merge pull request #64 from VinciGit00/generateScraperbranch 
add generateScraperBranch
 2024-04-17 11:51:06 +02:00
Marco Vinciguerra
9394ee01d5
Update readme.md 2024-04-14 16:40:30 +02:00
VinciGit00
876223da56 add generateScraperBranch 
Co-Authored-By: Matteo Mei <52063123+FattiMei@users.noreply.github.com>
 2024-04-14 13:18:50 +02:00
Marco Vinciguerra
7268ebd7bf
Update readme.md 2024-04-13 21:44:48 +02:00
Marco Vinciguerra
069a42bf07
Update readme.md 2024-04-12 21:47:14 +02:00
Matteo Mei
c660c334dd
Update readme.md with benchmark on ubuntu system 
Added results from running `benchmark.py` on a ubuntu laptop without docker
 2024-04-12 21:44:05 +02:00
VinciGit00
b2b21e55b1 add benchmark comparison 2024-04-12 20:26:24 +02:00
VinciGit00
68f1ca1b78 add embeddings in the example 2024-04-12 14:55:11 +02:00
VinciGit00
a17e35c3de Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-04-12 13:30:25 +02:00
VinciGit00
a8b452cb2c refactoring of the folders 2024-04-12 13:29:58 +02:00
VinciGit00
e0978c53b3 Merge branch 'main' into ollama_integration 2024-04-12 13:00:35 +02:00
VinciGit00
3640434f5c add utils 2024-04-12 12:59:15 +02:00
VinciGit00
60a2377bff add inputs 2024-04-12 12:53:54 +02:00
VinciGit00
8e08652d8a removed unused file and removed unused documentation 2024-04-12 11:05:52 +02:00
VinciGit00
87d911b7bb removed files and refactoring of actions 2024-04-11 18:18:22 +02:00
EURAC\marperini
7d521ef1c8 Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-04-11 14:29:43 +02:00
EURAC\marperini
74ec928f96 added execution info of the graph, time, price etc 2024-04-11 14:29:32 +02:00
VinciGit00
f3766955fd add examples 2024-04-11 12:17:04 +02:00
VinciGit00
4ef1564e18 Update smart_scraper_gemini.py 2024-04-10 12:49:55 +02:00
EURAC\marperini
982f142f20 fixed embedder_model config for local models 2024-04-10 11:00:52 +02:00
EURAC\marperini
8b915e3ec6 fixed issue ollama embeddings 2024-04-10 09:29:03 +02:00
ftoppi
64d8e7c3b6
smart_scraper_local.py: Allow using a remote Ollama #53 
Since you pass `config["llm"]` to langchain ChatOllama(), it's only a matter of adding a new parameter `base_url`.
 2024-04-09 22:28:02 +02:00
VinciGit00
8ef49a03e0 fixed local llm examples 2024-04-09 21:48:19 +02:00
ftoppi
65c572afe1
smart_scraper_local.py: example of arbitrary model_tokens 2024-04-09 21:30:00 +02:00
VinciGit00
ee533b8d74 add examples for local models and gemini models 2024-04-09 11:43:51 +02:00
VinciGit00
f954097387 add avaiable models 2024-04-09 10:34:35 +02:00
VinciGit00
0ffefdd170 refactoring if examples ffolder 2024-04-08 22:28:55 +02:00
VinciGit00
958b2a4a79 add new version + refactoring of the code 2024-04-08 18:24:59 +02:00
EURAC\marperini
dee1a42629 fixed token models, added mistral support 2024-04-08 15:21:06 +02:00
EURAC\marperini
17add20c13 implemented node_config, add embedder model choice, add azure endpoint, refactor graphs and exmaples 2024-04-08 12:18:32 +02:00
EURAC\marperini
6a83c2306d docker compose working ollama implementation 2024-04-07 11:08:08 +02:00
EURAC\marperini
72859d52b8 fixed variable name 2024-04-06 20:08:50 +02:00
VinciGit00
6dc4c6dec4 Merge branch 'main' into research_branch 2024-04-06 14:44:56 +02:00
VinciGit00
a5fc71f34e fixed internet research 2024-04-06 14:39:19 +02:00
EURAC\marperini
57a53bd6af refactor examples 2024-04-06 13:22:42 +02:00
EURAC\marperini
c2709aa50c working search graph 2024-04-06 13:01:04 +02:00
VinciGit00
d1afd2bd69 add xml parsing example 2024-04-05 20:42:37 +02:00
VinciGit00
4c3ea8f4ac fixed image_to_tex_node and refactoring 2024-04-03 12:53:12 +02:00
Marco Vinciguerra
b67af93946
Update scrape_plain_text.py 2024-03-29 19:17:18 +01:00
VinciGit00
5e38c2e298 add superclasse for graphs 2024-03-28 20:46:17 +01:00
VinciGit00
a28ca993de refactoring of examples folder 2024-03-27 21:41:21 +01:00
VinciGit00
587dfa5c68 refactoring of the saving functions 2024-03-26 14:15:36 +01:00
VinciGit00
be36ab8707 add new type of scraping through text 2024-03-24 21:22:26 +01:00
VinciGit00
e25f74ddc8 fixed bug for scraping from node + add example 2024-03-18 22:53:52 +01:00
VinciGit00
1afb950751 refactoring of code, add example on the readme 2024-03-18 14:35:47 +01:00
Perinim
cae38a0d67 fixed bugs after merge 2024-03-18 11:30:14 +01:00
Perinim
b5bdd4e729 Merge remote-tracking branch 'origin/main' into refactoring_nodes 2024-03-18 11:07:42 +01:00
Perinim
7d3190a636 fixed example graphs and utils files 2024-03-18 10:20:23 +01:00
Perinim
b547972968 fixed custom_graph, graph_builder and trulens_evaluator 2024-03-18 09:12:59 +01:00
Perinim
52934bf007 implemented graph_config, fixed smart_scraper and speech graph 2024-03-17 20:35:04 +01:00
Perinim
20d7b69008 refactored BaseNode and ParseNode 2024-03-17 11:40:35 +01:00
VinciGit00
e0c5bfe093 add new example for gemini 2024-03-14 21:03:34 +01:00
Lorenzo Padoan
d0f4b0159f DEV gemini support for simple custom graph 
-Not supported yet smartscrapergrapn with gemini
 2024-03-14 19:35:04 +01:00
VinciGit00
a849315182 add new tests and refactoring of code 2024-03-12 21:40:09 +01:00
Perinim
308c2d28c8 fixed text input example 2024-03-12 12:22:47 +01:00
Perinim
9d31db2d35 working smart scraper example 2024-03-12 11:06:08 +01:00
VinciGit00
ff7f8105bb add multiple merging generate answer node 2024-03-05 14:50:31 +01:00
VinciGit00
064f14d7cb fixed a typo 2024-03-04 17:47:33 +01:00
VinciGit00
ab71a6bc5a fixed documentation and fetch_text_node 2024-03-03 18:50:03 +01:00
VinciGit00
a0c77491bf add new progresbbar and finished the node 2024-03-02 20:51:29 +01:00
VinciGit00
1250aeeb8e add the node for reading the html as a txt 2024-03-02 13:32:34 +01:00
VinciGit00
79387d1617 refactoring for pylint score 2024-02-29 10:49:25 +01:00
VinciGit00
89d193392d refactor of parse_html_node 2024-02-27 17:02:06 +01:00
VinciGit00
f1bada175c refactoring of parse node 2024-02-27 12:24:22 +01:00
VinciGit00
043ab48466 add integration for the RAG for ParseHTMLNode 2024-02-25 11:24:44 +01:00
PeriniM
ed64db24df working speech graph with rag 2024-02-23 14:17:46 +01:00
PeriniM
cb8ed4fc5b fixed eval output 2024-02-23 10:33:25 +01:00
PeriniM
4733e29023 disable dashboard 2024-02-23 09:51:51 +01:00
PeriniM
51a28eb166 Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-02-23 09:45:09 +01:00
PeriniM
284ba5b75d added trulens 2024-02-23 09:39:25 +01:00
VinciGit00
e08ac716df refactoring of the examples folder 2024-02-23 09:21:06 +01:00
Lorenzo Padoan
665a91484b dev trulens_evaluetor 2024-02-22 19:19:10 +01:00
VinciGit00
ae929393b9 refactoring for pylint score 2024-02-22 10:19:10 +01:00
Marco Perini
ed48f273f8
Merge branch 'main' into multi-chunks-implementation 2024-02-22 00:00:01 +01:00
Perinim
62cc9170c8 updated SmartScraperGraph with rag 2024-02-21 23:52:45 +01:00
Perinim
a8b9b9e619 updated dependencies 2024-02-21 19:26:24 +01:00
Perinim
b0c75dc7ed created rag example 2024-02-21 19:15:38 +01:00
Perinim
a17971b7ed implemented parallelrunnables 2024-02-21 18:17:34 +01:00
Perinim
15b468473b working test rag 2024-02-21 17:02:10 +01:00
Perinim
3236f6315a added example multichunks rag 2024-02-21 13:32:14 +01:00
VinciGit00
906fea92e9 add examples + module documentation 2024-02-20 16:50:32 +01:00
VinciGit00
71bd53d254 refactoring of the code, add tests and examples 2024-02-19 11:54:36 +01:00
PeriniM
3dc7bf1de9 fixed typo 2024-02-19 07:23:08 +01:00
PeriniM
4e3005db0a implemented summary graph 2024-02-19 07:18:28 +01:00
PeriniM
da16360b1f working text2speech and image2text 2024-02-19 06:26:50 +01:00
PeriniM
3ed5fd5e48 added openai wrapper and examples 2024-02-19 04:45:43 +01:00
PeriniM
20db549d08 implemented graphbuilder 2024-02-19 04:11:35 +01:00
PeriniM
49c22ae155 implemented graphbuilder class 2024-02-19 03:39:50 +01:00
PeriniM
20398a3059 dev: refactor dict on graph state 2024-02-18 23:36:16 +01:00
VinciGit00
f8bb32919b refactoring of the example and add 2 new nodes 2024-02-18 20:36:45 +01:00
Marco Perini
ef7d575b77
add: example custom_graph.py 2024-02-18 20:16:13 +01:00
VinciGit00
0926a3c439 removed unused files 2024-02-16 19:46:05 +01:00
VinciGit00
e6e87de2da add docstring 2024-02-14 20:37:15 +01:00
VinciGit00
2a234f2e4d refactoring of the code changing the name 2024-02-14 20:35:30 +01:00
VinciGit00
8e5201ee9a add refactoring for pylint standards 2024-02-14 12:53:03 +01:00
Perinim
716c296ada dev: docstrings, nodes folder and fixed setup.py 2024-02-14 11:04:20 +01:00
Marco Perini
dfcad74057
dev: implemented simpler example 2024-02-13 22:00:22 +01:00
Marco Perini
dce571ccda
dev: added scrape graph example 2024-02-13 21:46:04 +01:00
Marco Perini
02aade4ed1
add: example graph application 2024-02-13 20:46:55 +01:00
VinciGit00
bb2f488236 add generation schema 2024-02-11 13:59:47 +01:00
PeriniM
eacda97a1c fix: remover bug 2024-02-07 14:21:07 +01:00
PeriniM
134b964495 merge: temp branch 2024-02-07 13:40:37 +01:00
PeriniM
5312fcaf55 dev: changed library name 2024-02-03 21:42:06 +01:00
VinciGit00
0bc7e61072 refactoring of the functions 2024-02-01 10:29:19 +01:00
PeriniM
63a3be3357 del: removed unnecessary files 2024-01-31 16:56:17 +01:00
PeriniM
bc4e21128b add: readthedocs structure 2024-01-31 16:53:35 +01:00
PeriniM
787fea0e78 add: readthedocs structure 2024-01-31 16:46:05 +01:00
VinciGit00
bad46ffe39 refactoring of classes, removed .env in the class + documentation 2024-01-31 12:43:12 +01:00
VinciGit00
63b0eba188 Merge branch 'main' into pr/3 2024-01-30 15:35:55 +01:00
PeriniM
a9beecde8f add: setup.py for deployment 2024-01-30 15:05:29 +01:00
VinciGit00
f311c3e005 add examples + documentation in the readme 2024-01-30 13:31:39 +01:00
PeriniM
f367c8919d dev: refactor project structure 2024-01-30 10:36:02 +01:00