mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-06-15 21:00:30 +08:00

History

roryhaung 57a58e162e docs: Updated the graph_config in the documentation.		2024-09-12 14:50:02 +08:00
..
inputs	add new benchmarks	2024-04-20 22:49:25 +02:00
.env.example	ADD benchmarks m2 max llama3	2024-04-23 12:28:51 +02:00
benchmark_docker.py	handling the agent model	2024-04-24 13:13:46 +02:00
benchmark_groq.py	fix: script generator and add new benchmarks	2024-04-30 11:51:04 +02:00
benchmark_llama3.py	fix: script generator and add new benchmarks	2024-04-30 11:51:04 +02:00
benchmark_mistral.py	add integration for llama3	2024-04-21 13:11:10 +02:00
benchmark_openai_gpt4.py	docs: Updated the graph_config in the documentation.	2024-09-12 14:50:02 +08:00
benchmark_openai_gpt4o.py	docs: Updated the graph_config in the documentation.	2024-09-12 14:50:02 +08:00
benchmark_openai_gpt35.py	docs: Updated the graph_config in the documentation.	2024-09-12 14:50:02 +08:00
Readme.md	Update Readme.md	2024-06-21 15:00:31 +02:00

Readme.md

Local models

The two websites benchmark are:

Example 1: https://perinim.github.io/projects
Example 2: https://www.wired.com (at 17/4/2024)

Both are strored locally as txt file in .txt format because in this way we do not have to think about the internet connection

Hardware	Model	Example 1	Example 2
Macbook 14' m1 pro	Mistral on Ollama with nomic-embed-text	16.291s	38.74s
Macbook m2 max	Mistral on Ollama with nomic-embed-text
Macbook 14' m1 pro	Llama3 on Ollama with nomic-embed-text	12.88s	13.84s
Macbook m2 max	Llama3 on Ollama with nomic-embed-text

Note: the examples on Docker are not runned on other devices than the Macbook because the performance are to slow (10 times slower than Ollama). Indeed the results are the following:

Hardware	Example 1	Example 2
Macbook 14' m1 pro	139.89	Too long

Performance on APIs services

Example 1: personal portfolio

URL: https://perinim.github.io/projects Task: List me all the projects with their description.

Name	Execution time (seconds)	total_tokens	prompt_tokens	completion_tokens	successful_requests	total_cost_USD
gpt-3.5-turbo	4.132s	438	303	135	1	0.000724
gpt-4-turbo-preview	6.965s	442	303	139	1	0.0072
gpt-4-o	4.446s	444	305	139	1	0
Grooq with nomic-embed-text	1.335s	648	482	166	1	0

Example 2: Wired

URL: https://www.wired.com Task: List me all the articles with their description.

Name	Execution time (seconds)	total_tokens	prompt_tokens	completion_tokens	successful_requests	total_cost_USD
gpt-3.5-turbo	8.836s	1167	726	441	1	0.001971
gpt-4-turbo-preview	21.53s	1205	726	479	1	0.02163
gpt-4-o	15.27s	1400	715	685	1	0
Grooq with nomic-embed-text	3.82s	2459	2192	267	1	0