CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-06-15 21:00:30 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
ea2ff5099b
Scrapegraph-ai/examples/benchmarks/GenerateScraper
History
roryhaung 57a58e162e docs: Updated the graph_config in the documentation.
2024-09-12 14:50:02 +08:00
..
inputs add new benchmarks 2024-04-20 22:49:25 +02:00
.env.example ADD benchmarks m2 max llama3 2024-04-23 12:28:51 +02:00
benchmark_docker.py add new benchmarks 2024-04-20 22:49:25 +02:00
benchmark_groq.py fix: script generator and add new benchmarks 2024-04-30 11:51:04 +02:00
benchmark_llama3.py fix: script generator and add new benchmarks 2024-04-30 11:51:04 +02:00
benchmark_mistral.py add integration for llama3 2024-04-21 13:11:10 +02:00
benchmark_openai_gpt4.py docs: Updated the graph_config in the documentation. 2024-09-12 14:50:02 +08:00
benchmark_openai_gpt35.py docs: Updated the graph_config in the documentation. 2024-09-12 14:50:02 +08:00
Readme.md fix: script generator and add new benchmarks 2024-04-30 11:51:04 +02:00

Readme.md

Local models

Local models

The two websites benchmark are:

Both are strored locally as txt file in .txt format because in this way we do not have to think about the internet connection

The time is measured in seconds

The model runned for this benchmark is Mistral on Ollama with nomic-embed-text

Hardware Model Example 1 Example 2
Macbook 14' m1 pro Mistral on Ollama with nomic-embed-text 30.54s 35.76s
Macbook m2 max Mistral on Ollama with nomic-embed-text
Macbook 14' m1 pro
 Llama3 on Ollama with nomic-embed-text 27.82s 29.986s
Macbook m2 max
 Llama3 on Ollama with nomic-embed-text

Note: the examples on Docker are not runned on other devices than the Macbook because the performance are to slow (10 times slower than Ollama).

Performance on APIs services

Example 1: personal portfolio

URL: https://perinim.github.io/projects Task: List me all the projects with their description.

Name Execution time (seconds) total_tokens prompt_tokens completion_tokens successful_requests total_cost_USD
gpt-3.5-turbo 24.21 1892 1802 90 1 0.002883
gpt-4-turbo-preview 6.614 1936 1802 134 1 0.02204
Grooq with nomic-embed-text 6.71 2201 2024 177 1 0

Example 2: Wired

URL: https://www.wired.com Task: List me all the articles with their description.

Name Execution time (seconds) total_tokens prompt_tokens completion_tokens successful_requests total_cost_USD
gpt-3.5-turbo
gpt-4-turbo-preview
Grooq with nomic-embed-text