Commit Graph

427 Commits

Author SHA1 Message Date
Evan Lin
a7264cebd2 chore: update gemini model to "gemini-pro" 2024-08-09 23:15:01 +08:00
Marco Vinciguerra
157d59d5c0
Merge branch 'pre/beta' into temp 2024-08-09 10:28:59 +02:00
Marco Vinciguerra
b6bff0be33
Merge branch 'pre/beta' into mistral-ai-integration 2024-08-07 21:24:18 +02:00
Marco Vinciguerra
bd5b6863ee
Update search_graph_openai.py 2024-08-07 16:19:04 +02:00
Federico Aguzzi
b0ffc51e54 chore(examples): fix Mistral examples
Some checks are pending
/ build (push) Waiting to run
2024-08-07 15:20:49 +02:00
Marco Vinciguerra
b52e4a390b fix: generate answer node omni 2024-08-07 14:09:02 +02:00
Matteo Vedovati
29ad140fa3 fix: refactoring of fetch_node 2024-08-07 11:56:10 +02:00
Federico Aguzzi
f8ad616e10 chore(mistral): create examples 2024-08-06 18:16:15 +02:00
Marco Vinciguerra
886bab78e7
Merge branch 'main' into temp-1 2024-08-06 10:04:57 +02:00
Marco Vinciguerra
66a29bc5cc refactoring of the openai examples
Some checks are pending
/ build (push) Waiting to run
Release / Build (push) Waiting to run
Release / Release (push) Blocked by required conditions
Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>
2024-08-05 20:54:53 +02:00
Marco Vinciguerra
fe099f9cbc
Merge branch 'pre/beta' into browserbase_integration
Some checks failed
/ build (push) Has been cancelled
2024-08-01 15:14:49 +02:00
Marco Vinciguerra
0b4cfd6522 fix: abstract_graph and removed unused embeddings
Co-Authored-By: Federico Aguzzi <62149513+f-aguzzi@users.noreply.github.com>
2024-08-01 14:38:50 +02:00
Marco Vinciguerra
5ecdbe715f feat: add integration in the abstract grapgh 2024-08-01 12:51:18 +02:00
Marco Vinciguerra
fb87d01ced Create browser_base.py 2024-08-01 11:27:10 +02:00
Marco Vinciguerra
1db164e9e6 feat: fix tests 2024-07-30 16:12:31 +02:00
Marco Vinciguerra
93fc2c1767 Update md_scraper_openai.py
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
Release / Build (push) Waiting to run
Release / Release (push) Blocked by required conditions
2024-07-26 12:35:13 +02:00
Marco Perini
5c08eea189 docs: prev version
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
/ build (push) Has been cancelled
Release / Build (push) Has been cancelled
Release / Release (push) Has been cancelled
2024-07-24 23:37:57 +02:00
Marco Vinciguerra
e08b30495d boh
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
/ build (push) Waiting to run
Release / Build (push) Waiting to run
Release / Release (push) Blocked by required conditions
2024-07-24 14:33:55 +02:00
Marco Vinciguerra
41ea3e0e27 add llama3.1 model 2024-07-24 13:38:27 +02:00
Marco Perini
49b6468fb1
Merge branch 'main' into pre/beta 2024-07-23 16:14:15 +02:00
Marco Perini
12b5eada6e
fix(md_conversion): add absolute links md, added missing dependency 2024-07-23 15:34:12 +02:00
Marco Vinciguerra
8c859f8d72 add nemotron examples 2024-07-22 19:12:50 +02:00
Marco Vinciguerra
09256f7b11 fix: parse node 2024-07-22 16:15:38 +02:00
Marco Perini
61a51d4bce
Merge pull request #466 from ScrapeGraphAI/refactoring_convert_to_md
refactoring of generate answer node
2024-07-20 19:40:57 +02:00
Marco Vinciguerra
cb02740ac9 refactoring of the examples 2024-07-20 19:09:26 +02:00
Marco Perini
de20f30fb3
Merge pull request #474 from DragonelRoland/main
Some checks are pending
/ build (push) Waiting to run
Release / Build (push) Waiting to run
Release / Release (push) Blocked by required conditions
Update smart_scraper_openai.py
2024-07-20 14:28:08 +02:00
DragonelRoland
0b1539341c
Update smart_scraper_openai.py 2024-07-20 14:25:48 +02:00
Marco Vinciguerra
3a9d556e1b
Merge branch 'pre/beta' into add-openai-supported-model-gpt-4o-mini 2024-07-19 10:08:41 +02:00
Marco Vinciguerra
e2ca41e928 add example 2024-07-18 10:24:24 +02:00
Marco Vinciguerra
07f1e23d23 fix: parse_node 2024-07-17 22:58:21 +02:00
Marco Vinciguerra
f42d47e2c3 add examples 2024-07-15 20:27:44 +02:00
Marco Vinciguerra
57fdaf9e3a create search_link_graph
Some checks are pending
/ build (push) Waiting to run
2024-07-15 12:36:11 +02:00
Marco Vinciguerra
c19d83f994
Update fetch_node.py
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Python application / build (push) Has been cancelled
Release / Build (push) Has been cancelled
Release / Release (push) Has been cancelled
2024-07-11 18:03:10 +02:00
Marco Vinciguerra
62f51652eb Create serch_graph_scehma.py
Some checks failed
Release / Build (push) Has been cancelled
Release / Release (push) Has been cancelled
2024-07-10 20:16:25 +02:00
Marco Vinciguerra
8bd8fb0859 refactoring of examples
Some checks are pending
Release / Build (push) Waiting to run
Release / Release (push) Blocked by required conditions
2024-07-10 12:19:41 +02:00
Marco Vinciguerra
8a52914871
Merge pull request #429 from ScrapeGraphAI/421-default-prompt-template-customization
421 default prompt template customization
2024-07-04 19:42:35 +02:00
Federico Aguzzi
720f18729b Merge branch 'fireworks_integration' into support 2024-07-04 18:17:38 +02:00
Marco Vinciguerra
3bf5f570a8 feat: add integration for infos 2024-07-01 21:19:16 +02:00
Marco Vinciguerra
30ca15ca28
Merge branch 'md_scraper_integration' into integration_markdown
Some checks failed
/ build (3.10) (push) Has been cancelled
2024-06-30 16:58:37 +02:00
Marco Vinciguerra
2804434a9e feat: add integrations for markdown files
Some checks are pending
/ build (3.10) (push) Waiting to run
2024-06-29 13:35:39 +02:00
Marco Vinciguerra
9b45ebcdcf modify fetch node with no cut mode 2024-06-28 14:38:36 +02:00
Marco Vinciguerra
228a1de2be add new force 2024-06-27 18:57:27 +02:00
Marco Vinciguerra
4b56604413 add examples + test
Some checks failed
/ build (3.10) (push) Has been cancelled
2024-06-25 10:32:29 +02:00
Marco Vinciguerra
df0e310829 feat: add fireworks integration 2024-06-24 23:11:28 +02:00
Marco Vinciguerra
92cabe1da6 add load examples from a yml file
Some checks failed
/ build (3.10) (push) Has been cancelled
2024-06-23 13:02:35 +02:00
Marco Vinciguerra
d8fcb6ccd1 add new examples 2024-06-22 20:59:53 +02:00
Marco Vinciguerra
6549915962 Update Readme.md
Some checks failed
/ build (3.10) (push) Has been cancelled
2024-06-21 15:00:31 +02:00
Marco Vinciguerra
5d6123847e add new convert function
Some checks are pending
/ build (3.10) (push) Waiting to run
Co-Authored-By: Federico Minutoli <40361744+DiTo97@users.noreply.github.com>
2024-06-20 21:15:16 +02:00
Marco Vinciguerra
2f02830c81 refactoring of fetch node
Some checks are pending
/ build (3.10) (push) Waiting to run
2024-06-20 13:44:42 +02:00
Marco Vinciguerra
23bc6332d0 fixed a bug 2024-06-19 21:46:31 +02:00
Marco Vinciguerra
6d783755ce add benchmark 2024-06-19 21:11:15 +02:00
Marco Perini
0cb46f5f30
Merge pull request #392 from VinciGit00/391-switch-between-search-engines
Some checks failed
/ build (3.10) (push) Has been cancelled
Release / Build (push) Has been cancelled
Release / Release (push) Has been cancelled
feat: add new search engine avaiability and new tests
2024-06-18 23:46:28 +02:00
Jason Vertrees
aedda44868 fix: updated for schema changes
docs: updated for schema changes
2024-06-18 13:31:10 -05:00
Marco Vinciguerra
073d226723 feat: add new search engine avaiability and new tests 2024-06-18 14:35:13 +02:00
Marco Perini
080a318ff6 feat(telemetry): add telemetry module 2024-06-17 13:00:33 +02:00
Marco Vinciguerra
2419003999 fix: fix robot node
Some checks are pending
/ build (3.10) (push) Waiting to run
Release / Build (push) Waiting to run
Release / Release (push) Blocked by required conditions
2024-06-16 14:04:36 +02:00
Marco Perini
9b0e62742b changed source to text 2024-06-14 15:24:50 +02:00
Marco Perini
12f4386552
Merge branch 'pre/beta' into 349-problem-with-scrapegraphaigraphspdf_scraper_graphpy 2024-06-14 15:22:43 +02:00
Marco Perini
203de83405 fix(pdf): correctly read .pdf files 2024-06-14 15:20:30 +02:00
Marco Perini
91c5b5af43 fix(multi): updated multi pdf scraper with schema 2024-06-14 14:59:12 +02:00
Marco Perini
283b61fafc docs: better logging 2024-06-13 18:13:47 +02:00
Marco Vinciguerra
e45f159a31 enhanced performance and readibility 2024-06-12 14:59:10 +02:00
Marco Vinciguerra
58a257f05b update model tokens
Some checks are pending
/ build (3.10) (push) Waiting to run
Release / Build (push) Waiting to run
Release / Release (push) Blocked by required conditions
2024-06-12 12:41:58 +02:00
Marco Perini
a10b060409
Merge pull request #361 from VinciGit00/multi_scraper_implementation
Multi scraper implementation
2024-06-12 00:49:18 +02:00
Marco Perini
5d692bff9e feat(schema): merge scripts to follow pydantic schema 2024-06-12 00:48:08 +02:00
Marco Vinciguerra
9326637cc3 Merge branch 'main' into pre/beta 2024-06-09 17:04:11 +02:00
Marco Vinciguerra
bde02492c0 add examples 2024-06-09 15:26:56 +02:00
Marco Vinciguerra
fe8083fe48 Update pdf_scraper_graph_haiku.py
Some checks failed
/ build (3.10) (push) Has been cancelled
Release / Build (push) Has been cancelled
Release / Release (push) Has been cancelled
2024-06-09 10:02:29 +02:00
Marco Vinciguerra
5dc6165881 add example
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
/ build (3.10) (push) Has been cancelled
Release / Build (push) Has been cancelled
Release / Release (push) Has been cancelled
2024-06-09 09:25:37 +02:00
Marco Vinciguerra
c14fb88fca add examples
Some checks failed
/ build (3.10) (push) Has been cancelled
2024-06-09 08:58:47 +02:00
Marco Vinciguerra
cb00c4fb17 changed model 2024-06-08 12:22:50 +02:00
Marco Vinciguerra
1981230e6f add multi scraper integration 2024-06-08 12:13:18 +02:00
Marco Perini
f41a755519
Merge pull request #356 from VinciGit00/321-integration-with-indexify
fixed pydantic schema
2024-06-07 20:27:25 +02:00
Marco Vinciguerra
dd2b3a8f59 add examples
Some checks failed
/ build (3.10) (push) Has been cancelled
Release / Build (push) Has been cancelled
Release / Release (push) Has been cancelled
2024-06-05 21:08:00 +02:00
Marco Perini
5d1fbf806a feat(indexify-node): add example
Some checks failed
/ build (3.10) (push) Has been cancelled
2024-06-05 18:45:37 +02:00
Marco Vinciguerra
95725789ff add earnie example 2024-06-05 13:21:32 +02:00
Marco Vinciguerra
4f53b09bf1 add examples for schema
Some checks are pending
/ build (3.10) (push) Waiting to run
Release / Build (push) Waiting to run
Release / Release (push) Blocked by required conditions
2024-06-05 10:43:57 +02:00
Marco Vinciguerra
74fd530914
Merge branch 'pre/beta' into 332-pydantic-schema-validation 2024-06-05 09:05:19 +02:00
Marco Perini
376f758a76 feat(pydantic): added pydantic output schema 2024-06-04 23:07:49 +02:00
Marco Vinciguerra
fff89f431f feat: refactoring of abstract graph 2024-06-04 19:41:11 +02:00
Marco Vinciguerra
8de720d379 feat: removed a bug 2024-06-03 21:45:37 +02:00
Marco Vinciguerra
1dde43cdeb add new examples 2024-06-03 21:03:13 +02:00
Marco Perini
79ace115c7
Merge pull request #323 from VinciGit00/refactoring-pdf_scraper
Refactoring pdf scraper and json scrape
2024-06-03 13:26:11 +02:00
Marco Vinciguerra
743dfe1191 add all possible examples 2024-06-03 12:19:43 +02:00
Marco Vinciguerra
b4086550cc feat: add csv scraper and xml scraper multi 2024-06-02 22:57:33 +02:00
Marco Vinciguerra
fa9722d2b9 add examples 2024-06-02 14:43:02 +02:00
Marco Vinciguerra
5cfc10178a feat: add forcing format as json 2024-06-02 12:24:54 +02:00
Marco Vinciguerra
1fe49753b9 add openai and oneapi examples 2024-06-01 13:46:15 +02:00
Marco Vinciguerra
5bda918a39 feat: add json multiscraper 2024-06-01 12:04:19 +02:00
Marco Vinciguerra
4d42d7bfc6 add example 2024-06-01 11:20:24 +02:00
Marco Vinciguerra
f5cbd80c97 feat: add pdf scraper multi graph 2024-06-01 09:52:21 +02:00
Marco Vinciguerra
1aa8c86b61 removed unused file 2024-05-29 19:15:56 +02:00
Marco Vinciguerra
3b90ebd9a8 add new examples 2024-05-29 10:41:37 +02:00
Marco Vinciguerra
58dfe9b658 add examples of usage 2024-05-28 09:11:14 +02:00
Marco Vinciguerra
004d03a0ff add examples 2024-05-27 12:50:51 +02:00
Marco Vinciguerra
f4a253b513 removed unused file 2024-05-27 11:40:51 +02:00
Marco Perini
54e82163f0 fix: fixed typo 2024-05-26 16:38:10 +02:00
Marco Vinciguerra
3c7dedf9c8
Merge pull request #305 from VinciGit00/pdf_fix
Pdf fix
2024-05-26 15:39:21 +02:00
arsaboo
a7961691df fix(pdf-example): added pdf example and coauthor 2024-05-26 15:09:49 +02:00
Marco Vinciguerra
fb74a5207e update one_api example with schema 2024-05-26 12:22:53 +02:00
Marco Vinciguerra
1d958be44d
Merge pull request #303 from VinciGit00/295-scrapegraph-ai接入oneapi模型qwen-turbo
add OneAPI integration
2024-05-26 12:17:34 +02:00
Marco Vinciguerra
ecd98b2a45 add sche,a example 2024-05-26 12:15:48 +02:00
Marco Vinciguerra
40a99fa2f9 Update pdf_scraper_ollama.py 2024-05-26 11:02:08 +02:00
Marco Vinciguerra
a22be474f5 add example 2024-05-26 11:01:10 +02:00
Marco Perini
8d76c4b3cb fix(schema): added schema 2024-05-26 10:51:48 +02:00
Marco Perini
5684578fab fix(kg): removed unused nodes and utils 2024-05-25 00:44:14 +02:00
Marco Perini
a4ee757507
Merge branch 'pre/beta' into pdf_scraper_refactoring 2024-05-25 00:16:38 +02:00
Marco Perini
8d5eb0bb0d fix(local_file): fixed textual input pdf, csv, json and xml graph 2024-05-25 00:13:47 +02:00
Marco Vinciguerra
b6f1766bd1 add OneAPI integration
Co-Authored-By: Federico Aguzzi <62149513+f-aguzzi@users.noreply.github.com>
Co-Authored-By: wangdongpeng1 <74647183+wangdongpeng1@users.noreply.github.com>
2024-05-24 13:46:40 +02:00
Marco Vinciguerra
b913b51cca Merge branch 'logger-integration' into pre/beta 2024-05-24 12:39:14 +02:00
Marco Vinciguerra
b377467b29 add info 2024-05-23 12:51:08 +02:00
Marco Vinciguerra
be4237a04d
Merge branch 'pre/beta' into multi_scraper_graph 2024-05-21 13:44:06 +02:00
Marco Perini
fc58e2d3a6 feat(smart-scraper-multi): add schema to graphs and created SmartScraperMultiGraph 2024-05-21 13:13:27 +02:00
JGalego
60584921ca Updated JSON scraper example prompt 2024-05-20 16:55:28 +01:00
JGalego
7e5ff4e410 Added bedrock examples 2024-05-20 16:49:20 +01:00
Marco Vinciguerra
5701afe927 add new import 2024-05-18 10:36:41 +02:00
Marco Perini
c75e6a06b1 feat(kg): working rag kg 2024-05-18 10:26:25 +02:00
Marco Perini
58cc903d55 feat(multiple): quick fix working 2024-05-18 05:57:43 +02:00
Marco Perini
bed3eed50c feat(multiple_search): working multiple example 2024-05-18 01:51:12 +02:00
Marco Vinciguerra
05e511e36f add new prompts 2024-05-18 00:22:52 +02:00
Marco Perini
0196423bde feat(knowledgegraph): add knowledge graph node 2024-05-17 23:41:44 +02:00
Marco Perini
8c33ea3fbc feat(node): knowledge graph node 2024-05-17 18:40:58 +02:00
Marco Vinciguerra
3453f72397 add graph 2024-05-17 18:23:50 +02:00
Marco Vinciguerra
02745a4f63 Merge branch 'main' into pre/beta 2024-05-17 10:40:19 +02:00
VinciGit00
9483afddd3 revert 2024-05-16 12:34:02 +02:00
Mayur Bhosale
1e0b2f7e70
Merge branch 'pre/beta' into nDeep 2024-05-15 23:36:23 +02:00
mayurdb
d60438cc89 Add a n-level deep search support 2024-05-15 23:32:15 +02:00
VinciGit00
ba8a4f7122 removed duplicates 2024-05-15 21:47:39 +02:00
VinciGit00
befa48c912 update lock 2024-05-15 21:28:56 +02:00
VinciGit00
cc5adefd29 fix: come back to the old version 2024-05-15 15:54:00 +02:00
VinciGit00
29d284e497 Merge branch 'main' into logger-integration 2024-05-15 15:28:20 +02:00
Marco Vinciguerra
cffcf80a75
Merge branch '88-blockscraper-implementation' into main 2024-05-15 15:20:12 +02:00
VinciGit00
a4700bfc75 add robot node 2024-05-15 12:00:32 +02:00
Marco Perini
a6e1813ddd fix(fetch_node): bug in handling local files 2024-05-14 16:51:10 +02:00
Marco Perini
fcb3abb01d feat(omni-search): added omni search graph and updated docs 2024-05-14 16:26:43 +02:00
Marco Perini
a296927624 feat(omni-scraper): working OmniScraperGraph with images 2024-05-14 13:46:49 +02:00
Marco Perini
90955ca52f feat(gpt-4o): image to text single node test 2024-05-14 11:43:21 +02:00
VinciGit00
d6f5ca874f Merge branch 'main' into pre/beta 2024-05-14 10:24:29 +02:00
Marco Perini
367dea5cbd
Merge branch 'pre/beta' into feat/parallel-node-execution 2024-05-13 23:50:40 +02:00
VinciGit00
c0d26d61d7 ad bedrocl 2024-05-13 20:17:03 +02:00
Marco Perini
dedc733047 fix(asyncio): replaced deepcopy with copy due to serialization problems 2024-05-13 18:46:34 +02:00
VinciGit00
7c91f9f94c add examples for deepseek 2024-05-13 12:56:57 +02:00
Marco Perini
c7ec114274 docs(refactor): changed example 2024-05-13 12:28:03 +02:00
Marco Perini
0c1594737f fix(fetch-node): removed isSoup from default 2024-05-13 12:09:55 +02:00
Marco Perini
7e8acd8e6a
Merge branch 'pre/beta' into fix/fetch-node-proxybroker 2024-05-13 11:17:37 +02:00
Marco Perini
0c36a7ec1f feat: added proxy rotation 2024-05-13 11:04:56 +02:00
Marco Perini
5d6d996e8f fix(proxy-rotation): removed max_shape duplicate 2024-05-13 07:26:43 +02:00
VinciGit00
30758b4b99 Create smart_scarper_deepseek.py 2024-05-12 20:27:35 +02:00
Marco Perini
1e9a564616 fix(proxy-rotation): removed duplicated arg and passed the loader_kwarhs correctly to the node 2024-05-12 18:39:03 +02:00
mayurdb
8f1fbe7e10 minor changes 2024-05-11 16:52:36 +05:30