1007 Commits

Author	SHA1	Message	Date
smith peng	553527a269	fix: fix pydantic object copy	2024-09-01 16:42:48 +08:00
Marco Vinciguerra	a73573d87a	update version	2024-08-31 12:32:25 +02:00
smith peng	71b22d4880	feat: add deepcopy error	2024-08-31 17:55:14 +08:00
smith peng	36818b1fb3	feat:adjust uncopiable obj raise error and remove memo	2024-08-31 17:39:33 +08:00
smith peng	cd07418474	fix: deepcopy fail for coping model_instance config	2024-08-31 16:26:11 +08:00
smith peng	4f4d091b82	feat:add deepcopy tool	2024-08-31 11:30:20 +08:00
Marco Vinciguerra	388630c0ff	fix: screenshot scraper	2024-08-30 13:02:13 +02:00
Marco Vinciguerra	a0d21137b7	refactoring of folders	2024-08-30 12:17:21 +02:00
Marco Vinciguerra	735120d0a3	Merge branch 'screenshot-scraper-fix' into pre/beta	2024-08-30 11:41:45 +02:00
Marco Vinciguerra	c348f674ad	fix: update generate answernode	2024-08-30 11:37:44 +02:00
elijahbenizzy	25d8fd2e72	Fixes node expression validator error message to make it easier to debug missing keys	2024-08-29 13:24:47 -07:00
Federico Aguzzi	34942deca5	chore(examples): create Together AI examples ... Co-Authored-By: Marco Vinciguerra <88108002+VinciGit00@users.noreply.github.com>	2024-08-28 22:39:05 +02:00
Marco Vinciguerra	8f615adef3	feat: add togheterai	2024-08-28 17:02:48 +02:00
Marco Vinciguerra	08fa257407	Merge pull request #602 from ScrapeGraphAI/593-abstract-graph-fix-round-4 ... Abstract graph fix	2024-08-28 16:10:36 +02:00
Federico Aguzzi	f7a85c266a	fix(models): better DeepSeek and OneApi integration	2024-08-28 12:51:50 +02:00
Federico Aguzzi	4f120e29c5	fix(AbstractGraph): model selection bug	2024-08-28 12:36:03 +02:00
Santabot123	90d75498fc	updated requirements.txt	2024-08-28 11:46:49 +03:00
Santabot123	aa9e85fdac	remove some comments and image	2024-08-28 11:38:56 +03:00
ajenkins	bda30a9ecb	swapped failing imports (local Gemini and non-imported Ernie) for langchain chat models in graph_builder.py	2024-08-28 16:48:00 +10:00
Marco Vinciguerra	df70b4f75d	Update abstract_graph.py	2024-08-27 18:08:53 +02:00
Marco Vinciguerra	cf73883451	fix: bug for abstract graph	2024-08-27 18:05:34 +02:00
Marco Vinciguerra	d45064b09a	Merge pull request #597 from ScrapeGraphAI/593-abstract-graph-fix-round-4 ... Abbstract graph fix, round 4	2024-08-27 17:54:04 +02:00
Federico Aguzzi	7789663338	fix(BurrBrige): dynamic imports	2024-08-27 17:17:51 +02:00
Federico Aguzzi	83e71df2e2	fix: set up dynamic imports correctly	2024-08-27 16:54:57 +02:00
Federico Aguzzi	5c16ee985b	fix(docloaders): BrowserBase dynamic import	2024-08-27 16:44:05 +02:00
Federico Aguzzi	f6df9b7512	chore(examples): update model names	2024-08-27 12:41:18 +02:00
Federico Aguzzi	f73343f193	fix(AbstractGraph): correct and simplify instancing logic	2024-08-27 12:22:53 +02:00
Marco Vinciguerra	04128e7e9f	fix: abstract graph local model	2024-08-26 18:00:17 +02:00
Marco Vinciguerra	2a602a157e	Merge branch 'pre/beta' into jamie-beck-patch-1	2024-08-26 17:54:31 +02:00
Jamie Beck	35b994a8cd	fix model_tokens not being used for ollama ... I am passing in the explicit model_tokens from user config as the default_token so it will correctly fallback to the users setting if the model is not found	2024-08-26 11:30:09 -04:00
Santabot123	0cf7c44409	correct the typo and updated select_area_with_ipywidget()	2024-08-26 14:51:28 +03:00
Santabot123	b4f8ea420b	add __init__.py and docstrings	2024-08-26 13:53:19 +03:00
Marco Vinciguerra	e3fbf01c5b	Revert "Anthropic refactoring"	2024-08-25 10:54:19 +02:00
Marco Vinciguerra	9df4b14bc0	refacttoring of the anthropic example	2024-08-25 10:53:01 +02:00
Marco Vinciguerra	86fe5fcaf1	fix: update abstract graph	2024-08-24 21:57:04 +02:00
Marco Vinciguerra	ee8f8b31ec	fix: add claude3.5 sonnet	2024-08-24 21:25:55 +02:00
Santabot123	e11f0cd28d	Added text_detection.py and updated screenshot_preparation.py	2024-08-24 18:40:54 +03:00
Marco Vinciguerra	cf1fada36a	fix: abstract graph ... Co-Authored-By: Federico Aguzzi <62149513+f-aguzzi@users.noreply.github.com>	2024-08-23 18:43:49 +02:00
Santabot123	49ae56ceee	Added screenshot preparation script for screenshot scraping	2024-08-23 19:00:14 +03:00
Marco Vinciguerra	26de5dd623	Merge branch 'pre/beta' into ligthweigthing_library	2024-08-23 12:22:31 +02:00
Marco Vinciguerra	f7ba1f30de	refactoring of the code	2024-08-23 11:33:22 +02:00
Marco Vinciguerra	0d2b7b37e0	update dependenceis	2024-08-21 20:48:47 +02:00
aziz-ullah-khan	a92b9c6970	Fix: Azure OpenAI issue	2024-08-21 16:18:51 +05:00
Federico Aguzzi	b05ec16b25	fix(models_tokens): add llama2 and llama3 sizes explicitly	2024-08-21 09:32:59 +02:00
Federico Aguzzi	dd406dff0c	style(models_tokens): enforce formatting	2024-08-21 09:29:38 +02:00
Marco Vinciguerra	860fde8a2c	Merge pull request #558 from ScrapeGraphAI/screenshot_scraper ... Screenshot scraper integration	2024-08-20 11:33:51 +02:00
Federico Aguzzi	d2486469af	Merge branch 'pre/beta' into screenshot_scraper	2024-08-20 09:58:34 +02:00
Federico Aguzzi	052f7d5e66	fix(SearchNode): prompt ... Co-Authored-By: Andrew Masek <8219702+portoaj@users.noreply.github.com>	2024-08-20 09:53:35 +02:00
Marco Vinciguerra	fee77d1fe6	Update screenshot_scraper_graph.py	2024-08-19 14:35:13 +02:00
Marco Vinciguerra	f774fe40e5	add try catch and robust integration	2024-08-19 14:24:27 +02:00
Federico Aguzzi	d29338b7c2	feat(GenerateAnswerNode): built-in structured output through LangChain ... Co-Authored-By: Marco Vinciguerra <88108002+VinciGit00@users.noreply.github.com>	2024-08-19 13:45:37 +02:00
Federico Aguzzi	b48ee825ee	Merge branch 'pre/beta' into support_structured_output_shema_openai	2024-08-19 13:31:01 +02:00
Federico Aguzzi	683bf57d89	fix(ParseNode): leave room for LLM reply in context window	2024-08-19 11:33:09 +02:00
Marco Vinciguerra	f60aa3acde	feat: add async call	2024-08-19 11:22:40 +02:00
ekinsenler	08e9d9d6a0	feat: Implemented a filter logic in search_link_node.py ... feat: Added dict entry for Llama3.1:8b	2024-08-19 11:05:31 +03:00
Marco Vinciguerra	0bf79b5926	Update generate_answer_from_image_node.py	2024-08-19 02:21:45 +02:00
Marco Vinciguerra	79fa3f6bd6	add if and cool stuff	2024-08-19 02:01:41 +02:00
Marco Vinciguerra	c72c077eb6	refactoring of the nodes	2024-08-19 01:15:45 +02:00
Marco Vinciguerra	8d6c0b7288	Update generate_answer_node.py	2024-08-18 22:01:39 +02:00
Marco Vinciguerra	103c21c86e	Update generate_answer_from_image_node.py	2024-08-18 20:54:35 +02:00
Marco Vinciguerra	5eb3cff64f	feat: refactoring of the code	2024-08-18 20:53:35 +02:00
Marco Vinciguerra	8e3d5deaaa	add screenshot scraper	2024-08-18 19:39:49 +02:00
Marco Vinciguerra	8b8d8f09b7	refactoring of the code according to pylint style	2024-08-18 11:51:34 +02:00
Marco Vinciguerra	ebdb74967d	add new model for gpt	2024-08-17 15:00:44 +02:00
Marco Vinciguerra	609316c8e7	Update abstract_graph.py	2024-08-17 14:39:39 +02:00
Marco Vinciguerra	391d051a5e	updated	2024-08-17 14:39:19 +02:00
Marco Vinciguerra	0aca28732b	feat: update model tokens dict	2024-08-17 14:34:06 +02:00
Marco Vinciguerra	c2c2e6582f	Update generate_answer_node.py	2024-08-17 13:30:44 +02:00
Marco Vinciguerra	f0587908e2	Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegraph-ai into pre/beta	2024-08-17 12:21:03 +02:00
Marco Vinciguerra	1d7f30b65b	fix: browser-base integration	2024-08-17 12:21:00 +02:00
Federico Aguzzi	5eed0a7654	fix(FetchNode) add OpenAI optimization to GPT models on Azure ... closes #547	2024-08-17 10:33:54 +02:00
Marco Vinciguerra	faef3186f7	fix: model count	2024-08-16 17:38:55 +02:00
Marco Vinciguerra	5100368b5a	Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegraph-ai into pre/beta	2024-08-16 09:01:30 +02:00
Marco Vinciguerra	b115f9612f	refactoring of code and documentation	2024-08-16 09:01:27 +02:00
Marco Vinciguerra	333f09a289	Merge pull request #552 from sandeepchittilla/550-add-support-gpt4o-variants ... Add new feature to support gpt-4o variant models with different pricing	2024-08-16 09:00:31 +02:00
Federico Aguzzi	ea72d7609c	Merge pull request #536 from ScrapeGraphAI/refactoring-of-abstract-graph ... Refactoring of abstract graph	2024-08-15 16:46:03 +02:00
sandeepchittilla	855144876d	feat: Add new feature to support gpt-4o variant models with different pricing	2024-08-15 14:59:48 +01:00
Marco Vinciguerra	203ee2c186	removed unused imports	2024-08-14 11:07:11 +02:00
Marco Vinciguerra	597f2acd07	fixed doc bugs and removed unused rag node	2024-08-14 10:59:53 +02:00
Marco Vinciguerra	c5aebd73d3	Merge pull request #541 from ScrapeGraphAI/496-not-match-gemini-embeddings-model	2024-08-13 20:54:57 +02:00
Marco Vinciguerra	9be44742d6	Update abstract_graph.py	2024-08-13 19:29:03 +02:00
Marco Vinciguerra	795c9325cc	Merge branch 'pre/beta' into refactoring-of-abstract-graph	2024-08-12 18:35:55 +02:00
Marco Vinciguerra	c105c26b2f	Update abstract_graph.py	2024-08-12 18:32:41 +02:00
Federico Aguzzi	cb6b35397e	fix(models_tokens): incorrect provider names	2024-08-12 10:52:13 +02:00
Federico Aguzzi	bae89e832e	Merge branch 'pre/beta' into 530-probable-bug	2024-08-12 10:37:50 +02:00
Federico Aguzzi	e6bedb6701	fix(AbstractGraph): pass kwargs to Ernie and Nvidia models ... Co-Authored-By: Marco Vinciguerra <88108002+VinciGit00@users.noreply.github.com>	2024-08-12 10:11:45 +02:00
Marco Vinciguerra	8cece1d31b	Merge branch 'pre/beta' into refactoring-of-abstract-graph	2024-08-11 19:20:26 +02:00
Marco Vinciguerra	cef2fdb420	Update abstract_graph.py ... Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>	2024-08-11 19:19:17 +02:00
Marco Vinciguerra	c77231c983	feat: update abstract graph ... Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>	2024-08-11 19:18:24 +02:00
Marco Vinciguerra	de1ec250ef	refactoring pyproject.toml ... Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>	2024-08-11 18:04:31 +02:00
Marco Vinciguerra	cec5537f2a	add new tests ... Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>	2024-08-11 17:10:55 +02:00
Matteo Vedovati	a3b7181f95	quick fix typo	2024-08-11 11:52:50 +02:00
Matteo Vedovati	734b740bf9	Template name refactoring	2024-08-11 11:46:01 +02:00
Matteo Vedovati	98779d193b	search_link_node prompts moved	2024-08-11 11:43:22 +02:00
Matteo Vedovati	ef966525c3	search_link_node prompt moved	2024-08-11 11:34:49 +02:00
Matteo Vedovati	86a4903e99	search_internet_node prompt moved	2024-08-11 11:31:51 +02:00
Matteo Vedovati	f455fcbc5f	robot_node prompt moved	2024-08-11 11:27:27 +02:00
Matteo Vedovati	9814b6dd85	fixed versioning errors	2024-08-11 10:04:15 +02:00
Matteo Vedovati	7c6210019e	Merge branch 'pre/beta' of https://github.com/vedovati-matteo/Scrapegraph-ai into pre/beta	2024-08-11 09:42:38 +02:00
Matteo Vedovati	f17cef94bb	fix: merge_anwser prompt import	2024-08-11 09:41:31 +02:00
Matteo Vedovati	9fa3df2390	Changed prompts import from helpers to prompts	2024-08-11 09:33:21 +02:00
Matteo Vedovati	3b5b24d6f8	Added new folder for prompts	2024-08-11 09:32:00 +02:00
Matteo Vedovati	4ca606cf7a	remove promps form helpers folders	2024-08-11 09:31:26 +02:00
Marco Vinciguerra	0cda020d48	Merge branch 'pre/beta' into temp	2024-08-10 17:45:52 +02:00
Marco Vinciguerra	8b2c266aff	refactoring of the code ... Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>	2024-08-10 17:44:35 +02:00
Marco Vinciguerra	778efd4c87	fix: conditional node	2024-08-10 12:21:09 +02:00
Marco Vinciguerra	c3f1520240	Update abstract_graph.py	2024-08-10 12:13:09 +02:00
Marco Vinciguerra	6c3b37ab00	feat: add refactoring of default temperature	2024-08-10 11:51:37 +02:00
Marco Vinciguerra	f01b55e89b	fix: fetch node	2024-08-10 11:41:51 +02:00
Marco Vinciguerra	ce00345495	fix: conditional node	2024-08-09 23:38:26 +02:00
Marco Vinciguerra	1272273499	fix: broken node	2024-08-09 17:35:43 +02:00
Marco Vinciguerra	157d59d5c0	Merge branch 'pre/beta' into temp	2024-08-09 10:28:59 +02:00
Federico Aguzzi	50edbcc7f8	fix(FetchNode): missing bracket syntax error	2024-08-09 09:37:51 +02:00
Marco Vinciguerra	0048ba1542	Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegraph-ai into pre/beta	2024-08-08 12:09:39 +02:00
Marco Vinciguerra	71ae3845b4	Update generate_answer_node_prompts.py	2024-08-08 12:09:38 +02:00
Federico Aguzzi	5ec2de9e1a	fix(chunking): count tokens from words instead of characters ... closes #513	2024-08-08 10:50:52 +02:00
Marco Vinciguerra	7f03926432	Merge pull request #515 from vedovati-matteo/pre/beta ... fix: refactoring of merge_answer_node	2024-08-07 21:50:14 +02:00
Matteo Vedovati	898e5a7af5	fix: refactoring of merge_answer_node	2024-08-07 21:42:54 +02:00
Marco Vinciguerra	b6bff0be33	Merge branch 'pre/beta' into mistral-ai-integration	2024-08-07 21:24:18 +02:00
Marco Vinciguerra	a7db47ecad	refatoring	2024-08-07 21:23:42 +02:00
Federico Aguzzi	786af992f8	fix(AbstractGraph): LangChain warnings handling, Mistral tokens	2024-08-07 15:16:10 +02:00
Federico Aguzzi	07720b6e0c	fix(FetchNode): handling of missing browser_base key	2024-08-07 15:15:13 +02:00
Marco Vinciguerra	b52e4a390b	fix: generate answer node omni	2024-08-07 14:09:02 +02:00
Marco Vinciguerra	625ca9f22a	fix: generate answer node pdf has a bug	2024-08-07 14:06:53 +02:00
Matteo Vedovati	bfc6852b77	fix: refactoring of fetch_node adding comment	2024-08-07 13:02:02 +02:00
Matteo Vedovati	1ea2ad8e79	fix: refactoring of fetch_node qixed error	2024-08-07 12:04:54 +02:00
Matteo Vedovati	29ad140fa3	fix: refactoring of fetch_node	2024-08-07 11:56:10 +02:00
Federico Aguzzi	f8ad616e10	chore(mistral): create examples	2024-08-06 18:16:15 +02:00
Federico Aguzzi	17f2707313	feat: add mistral support	2024-08-06 17:51:50 +02:00
Federico Aguzzi	5e824327c3	chore(models_tokens): add mistral models	2024-08-06 17:22:39 +02:00
Marco Vinciguerra	82e63213ae	Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegraph-ai into pre/beta	2024-08-06 17:07:35 +02:00
Marco Vinciguerra	37dd6e9ba9	update reqs ... Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>	2024-08-06 17:07:33 +02:00
Federico Aguzzi	bc9e70031f	Merge branch 'main' into pre/beta	2024-08-06 16:50:46 +02:00
Marco Vinciguerra	6e4d04450f	Update base_graph.py	2024-08-06 16:29:59 +02:00
Marco Vinciguerra	fa651d4cd9	feat: add grok integration	2024-08-06 15:17:49 +02:00
Marco Vinciguerra	0571b6da55	feat: update base_graph	2024-08-06 14:01:11 +02:00
Marco Vinciguerra	886bab78e7	Merge branch 'main' into temp-1	2024-08-06 10:04:57 +02:00
Marco Vinciguerra	66a29bc5cc	refactoring of the openai examples ... Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>	2024-08-05 20:54:53 +02:00
Marco Vinciguerra	1fb711ae51	Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegraph-ai into pre/beta	2024-08-05 14:54:16 +02:00
Marco Vinciguerra	a3d7f1b71c	Update csv_scraper_graph.py	2024-08-05 14:54:14 +02:00
Marco Vinciguerra	ddb3135109	Merge pull request #503 from ScrapeGraphAI/497-abstract-graph-refactor ... fix(AbstractGraph): instantiation of Azure GPT models	2024-08-02 14:32:29 +02:00
Federico Aguzzi	ade28fca2c	fix(AbstractGraph): instantiation of Azure GPT models ... Closes #498	2024-08-02 12:15:43 +02:00
Marco Vinciguerra	9355507a2d	feat: refactoring of the code	2024-08-02 12:00:00 +02:00
Marco Vinciguerra	fe099f9cbc	Merge branch 'pre/beta' into browserbase_integration	2024-08-01 15:14:49 +02:00
Lorenzo Padoan	abd0ea13f9	Merge pull request #500 from ScrapeGraphAI/497-abstract-graph-refactor ... 497 abstract graph refactor	2024-08-01 14:51:36 +02:00
Marco Vinciguerra	53dc06a2cb	Update speech_graph.py	2024-08-01 14:48:19 +02:00
Marco Vinciguerra	9ac74de622	Update speech_graph.py	2024-08-01 14:47:49 +02:00
Marco Vinciguerra	0b4cfd6522	fix: abstract_graph and removed unused embeddings ... Co-Authored-By: Federico Aguzzi <62149513+f-aguzzi@users.noreply.github.com>	2024-08-01 14:38:50 +02:00
Marco Vinciguerra	be870a4316	Update fetch_node.py ... Co-Authored-By: Federico Minutoli <40361744+DiTo97@users.noreply.github.com>	2024-08-01 13:24:48 +02:00
Marco Vinciguerra	6d8e02cd62	Update browser_base.py ... Co-Authored-By: Federico Minutoli <40361744+DiTo97@users.noreply.github.com>	2024-08-01 13:24:32 +02:00
Marco Vinciguerra	968c69e217	fix: fixed bug on fetch_node ... Co-Authored-By: Federico Minutoli <40361744+DiTo97@users.noreply.github.com>	2024-08-01 13:23:54 +02:00
Marco Vinciguerra	e21d461710	push	2024-08-01 13:16:49 +02:00
Marco Vinciguerra	d03eedccd7	Update chromium.py	2024-08-01 13:05:13 +02:00
Marco Vinciguerra	65f9e3a24c	Delete browser_base.py	2024-08-01 12:58:35 +02:00
Marco Vinciguerra	4cd5986516	Merge branch 'pre/beta' into browserbase_integration	2024-08-01 12:52:36 +02:00
Marco Vinciguerra	5ecdbe715f	feat: add integration in the abstract grapgh	2024-08-01 12:51:18 +02:00
Federico Aguzzi	a94ebcde00	refactor: move embeddings code from AbstractGraph to RAGNode	2024-08-01 11:53:17 +02:00
Marco Vinciguerra	7076ab12d3	allignment	2024-08-01 11:31:27 +02:00
Marco Vinciguerra	4caed545e5	feat: intregration of firebase	2024-07-31 19:49:59 +02:00
Federico Aguzzi	bb73d916a1	refactor: reuse code for common interface models	2024-07-31 13:41:09 +02:00
Federico Aguzzi	b17756d934	style: enforce pylint styling	2024-07-31 13:07:19 +02:00
Federico Aguzzi	88710f1a7c	chore: remove unused import	2024-07-30 15:57:08 +02:00
Federico Aguzzi	bc2c9967d2	refactor: remove redundant wrappers for Ernie and Nvidia	2024-07-30 10:59:12 +02:00
Federico Aguzzi	9275486240	refactor: remove redundant LangChain wrappers	2024-07-29 21:59:39 +02:00
Marco Vinciguerra	5007167af1	removed unused models ... Co-Authored-By: Federico Aguzzi <62149513+f-aguzzi@users.noreply.github.com>	2024-07-29 17:59:16 +02:00
Marco Vinciguerra	f6b7c6a430	refactoring ... Co-Authored-By: Federico Aguzzi <62149513+f-aguzzi@users.noreply.github.com>	2024-07-29 17:56:32 +02:00
Federico Aguzzi	25066b2bc5	refactor: remove LangChain wrappers for Ollama	2024-07-29 17:49:21 +02:00
Federico Aguzzi	2c5f934f10	refactor: remove LangChain wrappers	2024-07-29 17:46:17 +02:00
Marco Vinciguerra	9e795f4e35	removed unused init files ... Co-Authored-By: Federico Aguzzi <62149513+f-aguzzi@users.noreply.github.com>	2024-07-29 17:35:10 +02:00
Federico Aguzzi	5e3eb6e43d	refactor(OpenAI): integrate new LangChain chat init	2024-07-29 11:26:20 +02:00
Federico Aguzzi	d177afb68b	refactor(Ollama): integrate new LangChain chat init	2024-07-29 11:24:34 +02:00
amosdinh	05cf9adaa7	Ollama: Use no json format when creating the search query	2024-07-26 12:10:14 +02:00
Marco Vinciguerra	5137b8aa5b	Update models_tokens.py	2024-07-26 10:18:08 +02:00
Federico Aguzzi	d745564ad1	Merge pull request #485 from ScrapeGraphAI/generate_answer_parallel ... merge: generate answer parallel	2024-07-25 11:49:43 +02:00
Marco Perini	5c08eea189	docs: prev version	2024-07-24 23:37:57 +02:00
Marco Vinciguerra	377d679eec	feat: pdate models_tokens.py	2024-07-24 15:17:48 +02:00
Marco Vinciguerra	e08b30495d	boh	2024-07-24 14:33:55 +02:00
Marco Vinciguerra	41ea3e0e27	add llama3.1 model	2024-07-24 13:38:27 +02:00
Federico Aguzzi	ddbfbe1205	Merge branch 'pre/beta' into generate_answer_parallel	2024-07-23 20:51:35 +02:00
Marco Perini	1d41f6eafe	fix: md conversion	2024-07-23 19:27:18 +02:00
Marco Perini	5a45e9f2d8	fix: md conversion	2024-07-23 18:09:15 +02:00
Marco Perini	49b6468fb1	Merge branch 'main' into pre/beta	2024-07-23 16:14:15 +02:00
Marco Perini	0ebc387446	Merge pull request #487 from ScrapeGraphAI/update-generate-answer-node ... Update generate answer node	2024-07-23 15:45:20 +02:00
Marco Perini	12b5eada6e	fix(md_conversion): add absolute links md, added missing dependency	2024-07-23 15:34:12 +02:00
Marco Vinciguerra	df1ecc0019	Update generate_answer_node.py	2024-07-23 13:06:59 +02:00
Marco Vinciguerra	cf2734392c	removed unused function	2024-07-23 13:05:50 +02:00
Marco Vinciguerra	8dd941e39e	update generate answer nodes ... Co-Authored-By: Federico Aguzzi <62149513+f-aguzzi@users.noreply.github.com>	2024-07-23 12:40:11 +02:00
Marco Vinciguerra	879a554a78	Update telemetry.py	2024-07-23 11:15:44 +02:00
Marco Vinciguerra	70911f83be	Update generate_answer_pdf_node.py	2024-07-23 11:07:35 +02:00
Marco Vinciguerra	0c4b2908d9	feat: add generate_answer node paralellization ... Co-Authored-By: Federico Minutoli <40361744+DiTo97@users.noreply.github.com>	2024-07-22 19:58:33 +02:00
Marco Vinciguerra	2ae19aee56	update function	2024-07-22 16:50:58 +02:00
Marco Vinciguerra	09256f7b11	fix: parse node	2024-07-22 16:15:38 +02:00
Marco Vinciguerra	71f894eee3	fix: parse_html node have a bug	2024-07-22 16:01:27 +02:00
Marco Vinciguerra	fc0dadb8f8	feat: add nvidia connection	2024-07-22 11:56:33 +02:00
Marco Vinciguerra	b49f9866fd	fixed telemetry version	2024-07-21 19:20:19 +02:00
Marco Vinciguerra	ce6be37fbc	fix: abstract_graph moel token bug	2024-07-21 18:37:15 +02:00
Marco Vinciguerra	7a3c21b58d	Update generate_answer_node.py	2024-07-20 20:19:16 +02:00
Marco Vinciguerra	6a9205920f	Merge branch 'pre/beta' into temp	2024-07-20 20:03:20 +02:00
Marco Perini	99dc8497d8	docs(gpt-4o-mini): added new gpt, fixed chromium lazy loading, ... add documentation and metrics	2024-07-20 20:02:26 +02:00
Marco Perini	61a51d4bce	Merge pull request #466 from ScrapeGraphAI/refactoring_convert_to_md ... refactoring of generate answer node	2024-07-20 19:40:57 +02:00
Marco Vinciguerra	19af78c7d8	Merge branch 'pre/beta' into main	2024-07-20 18:14:36 +02:00
DragonelRoland	4b8360658e	Update models_tokens.py ... added gpt4o mini to the models	2024-07-20 16:51:24 +02:00
DragonelRoland	e9ecf5f0dc	Update typo in conditional_node.py ... "thar" -> "that"	2024-07-20 15:06:52 +02:00
Marco Vinciguerra	3cf7089e3f	Merge branch 'pre/beta' into temp	2024-07-20 13:27:37 +02:00
Marco Vinciguerra	77777c898d	fix: add gpt o mini for azure	2024-07-20 13:20:16 +02:00
Marco Vinciguerra	03f4a3aa29	fix: azure models	2024-07-20 13:19:21 +02:00
Marco Vinciguerra	3a9d556e1b	Merge branch 'pre/beta' into add-openai-supported-model-gpt-4o-mini	2024-07-19 10:08:41 +02:00
Kernick	4f079cfc74	Fixed a typo in the model name	2024-07-18 21:06:16 +03:00
Kernick	b91020ce56	Update models_tokens.py with gpt-4o-mini ... OpenAI added a new available model - gpt-4o-mini. Adding this model to the supported models, so it would be possible to use it with the Scrapegraph-ai.	2024-07-18 21:03:55 +03:00
Federico Aguzzi	dfa759e834	style(models): fix module docstrings	2024-07-18 11:02:51 +02:00
Marco Vinciguerra	032a491605	Update parse_node.py	2024-07-17 23:06:05 +02:00
Marco Vinciguerra	07f1e23d23	fix: parse_node	2024-07-17 22:58:21 +02:00
Marco Vinciguerra	68f58cc4dd	refactoring of generate answer node	2024-07-17 22:41:49 +02:00
Marco Vinciguerra	602dd00209	feat: refactoring_to_md function	2024-07-16 12:39:48 +02:00
Federico Aguzzi	dccb893fdf	Merge pull request #456 from ScrapeGraphAI/refactoring-of-search_link_node ... fix: search link node	2024-07-15 23:06:03 +02:00
Marco Vinciguerra	830daee1f3	Update search_link_node.py	2024-07-15 20:47:09 +02:00
Marco Vinciguerra	57fdaf9e3a	create search_link_graph	2024-07-15 12:36:11 +02:00
Federico Aguzzi	789c16fb76	Merge pull request #458 from ScrapeGraphAI/searchngx_integration ... merge: update research_web.py	2024-07-14 22:55:52 +02:00
Federico Aguzzi	7ba2f6ae0b	chore: correct search engine name	2024-07-14 22:39:34 +02:00
Federico Aguzzi	fd1b7cb24a	chore: remove unused import	2024-07-14 22:36:43 +02:00
Marco Vinciguerra	7e5789baa3	Update research_web.py	2024-07-14 19:31:22 +02:00
Marco Vinciguerra	cf3ab5564a	fix: search link node	2024-07-14 16:49:29 +02:00
Marco Vinciguerra	a7249685cb	removed rag node	2024-07-14 10:24:38 +02:00
salamanderxing	4b65392cca	Resolves #447	2024-07-12 10:56:25 +02:00
Marco Vinciguerra	765b548c0e	Merge pull request #417 from ScrapeGraphAI/md_scraper_integration ... feat: add integrations for markdown files	2024-07-04 19:42:53 +02:00
Marco Vinciguerra	8a52914871	Merge pull request #429 from ScrapeGraphAI/421-default-prompt-template-customization ... 421 default prompt template customization	2024-07-04 19:42:35 +02:00
Federico Aguzzi	104d8692d6	Merge branch 'pre/beta' into support	2024-07-04 18:36:34 +02:00
Federico Aguzzi	1bbd25a63f	Merge pull request #407 from ScrapeGraphAI/404-split-unit-testing-from-src ... Merge #404 (split unit testing from src)	2024-07-04 18:33:45 +02:00
Federico Aguzzi	d77a622b05	Merge branch '423-add-vertex-ai-integration' into support	2024-07-04 18:23:22 +02:00
Federico Aguzzi	720f18729b	Merge branch 'fireworks_integration' into support	2024-07-04 18:17:38 +02:00
Federico Aguzzi	2ab7ddc95a	Merge pull request #405 from ScrapeGraphAI/integration_markdown ... Integration markdown	2024-07-04 18:13:12 +02:00
Marco Vinciguerra	3ee1743687	update prompts	2024-07-04 14:22:45 +02:00
Marco Vinciguerra	ed2af51501	update the chunk size	2024-07-02 12:03:08 +02:00
Marco Vinciguerra	3bf5f570a8	feat: add integration for infos	2024-07-01 21:19:16 +02:00
AmosDinh	3f9bf5b4d4	fixed typo	2024-07-01 13:42:10 +00:00
Marco Vinciguerra	119514bdfc	feat: add vertexai integration	2024-07-01 12:21:47 +02:00
Marco Vinciguerra	5fe694b6b4	feat: improve md prompt recognition	2024-06-30 18:10:00 +02:00
Marco Vinciguerra	30ca15ca28	Merge branch 'md_scraper_integration' into integration_markdown	2024-06-30 16:58:37 +02:00
Marco Vinciguerra	b6a78e926f	Merge branch 'main' into temp	2024-06-30 16:54:15 +02:00
Marco Vinciguerra	2804434a9e	feat: add integrations for markdown files	2024-06-29 13:35:39 +02:00
Marco Vinciguerra	9b45ebcdcf	modify fetch node with no cut mode	2024-06-28 14:38:36 +02:00
Marco Vinciguerra	f3cbbcee92	Merge pull request #412 from oviniciusfeitosa/patch-1 ... 🐛 Rename user_prompt parameter to prompt	2024-06-28 11:09:25 +02:00
Marco Vinciguerra	5d2e592a67	add new claude models	2024-06-28 10:53:40 +02:00
Marco Vinciguerra	4d936410cc	fix: add new claude model	2024-06-28 10:50:01 +02:00
Vinícius Feitosa da Silva	a3c43c9a99	🐛 Rename user_prompt parameter to prompt ... This adjustment makes the class consistent with the definition of the arguments.	2024-06-27 17:53:20 -03:00
Marco Vinciguerra	228a1de2be	add new force	2024-06-27 18:57:27 +02:00
Marco Vinciguerra	ff4ccb94a1	add claude 3.5 integration	2024-06-25 11:21:06 +02:00
Marco Vinciguerra	0dfa5bf07e	Merge branch 'pre/beta' into main	2024-06-25 10:47:15 +02:00
Maorsg	f75e0835fc	added a function to the search_graph class to allow user to return URLs considered in the search	2024-06-24 21:31:28 -04:00
Marco Vinciguerra	df0e310829	feat: add fireworks integration	2024-06-24 23:11:28 +02:00
Marco Vinciguerra	3a537eec6f	fix: add test	2024-06-23 17:54:09 +02:00
Djamel Feddad	2b4f4f12be	doc: Fixed a small typo in the warning message ... This PR fixes a small type in the warning message	2024-06-23 15:34:48 +08:00
Marco Vinciguerra	9917972c11	fixed request	2024-06-22 21:39:37 +02:00
Marco Vinciguerra	afd46ac77b	fixed generate_answer_node	2024-06-22 11:31:54 +02:00
Marco Vinciguerra	d1c3de777f	fixed a bug	2024-06-21 14:14:43 +02:00
Marco Vinciguerra	599512d2e5	fix: add new model for claude	2024-06-21 14:08:10 +02:00
Marco Vinciguerra	7af411aa99	add trigger ... Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>	2024-06-21 13:36:27 +02:00
Marco Vinciguerra	5d6123847e	add new convert function ... Co-Authored-By: Federico Minutoli <40361744+DiTo97@users.noreply.github.com>	2024-06-20 21:15:16 +02:00
Marco Vinciguerra	2f02830c81	refactoring of fetch node	2024-06-20 13:44:42 +02:00
Marco Vinciguerra	5664eb292b	Update generate_answer_node_prompts.py	2024-06-20 11:57:11 +02:00
Marco Vinciguerra	23bc6332d0	fixed a bug	2024-06-19 21:46:31 +02:00
Marco Vinciguerra	8bb560a489	add convert function	2024-06-19 20:17:45 +02:00
Marco Vinciguerra	88f9def69d	fix: reduced model tokens	2024-06-19 19:55:13 +02:00
Marco Vinciguerra	79a2f51c34	add new models tokens	2024-06-19 10:42:41 +02:00
Marco Vinciguerra	40b4b2d939	add new models tokens	2024-06-19 10:42:00 +02:00
Marco Perini	0cb46f5f30	Merge pull request #392 from VinciGit00/391-switch-between-search-engines ... feat: add new search engine avaiability and new tests	2024-06-18 23:46:28 +02:00
Marco Perini	c7870905e1	fix: total tokens and docs	2024-06-18 23:22:23 +02:00
Marco Vinciguerra	aa2160c108	feat: add research with bing + test function	2024-06-18 21:28:29 +02:00
Jason Vertrees	aedda44868	fix: updated for schema changes ... docs: updated for schema changes	2024-06-18 13:31:10 -05:00
Marco Vinciguerra	073d226723	feat: add new search engine avaiability and new tests	2024-06-18 14:35:13 +02:00
Marco Vinciguerra	1d0cbbc6d6	fix: add new embedding models	2024-06-18 09:13:15 +02:00
Marco Vinciguerra	48ff87f20a	Merge branch 'pre/beta' of https://github.com/VinciGit00/Scrapegraph-ai into pre/beta	2024-06-17 14:17:46 +02:00
Marco Vinciguerra	03ffebc52d	fix: add chinese embedding model	2024-06-17 14:17:44 +02:00
Marco Perini	080a318ff6	feat(telemetry): add telemetry module	2024-06-17 13:00:33 +02:00
Marco Vinciguerra	da93162b53	Merge branch 'pre/beta' into main	2024-06-17 11:02:56 +02:00
shubihu	d8d5cd267a	Update abstract_graph.py ... fix: Incorrect API Key Error with OpenAI Proxy	2024-06-17 16:26:33 +08:00
Marco Perini	4c8becc721	overwrite common params to affect nodes config	2024-06-16 15:19:40 +02:00
Marco Vinciguerra	2419003999	fix: fix robot node	2024-06-16 14:04:36 +02:00
Marco Vinciguerra	3d34ea65df	Merge pull request #384 from liaoliaojun/fix/merge ... fix: TypeError ‘Model provided by the configuration not supported’	2024-06-16 13:34:43 +02:00
Marco Vinciguerra	dcd216e345	fix: removed duplicate from ollama dictionary ... Co-Authored-By: Matteo Vedovati <68272450+vedovati-matteo@users.noreply.github.com>	2024-06-15 19:51:20 +02:00
liaoliaojun	62b372b675	fix: shallow copy config of create_embedder	2024-06-15 17:53:13 +08:00
Marco Perini	09cb6e964e	refactor: add missing schemas and renamed files	2024-06-14 15:38:46 +02:00
Marco Perini	12f4386552	Merge branch 'pre/beta' into 349-problem-with-scrapegraphaigraphspdf_scraper_graphpy	2024-06-14 15:22:43 +02:00
Marco Perini	203de83405	fix(pdf): correctly read .pdf files	2024-06-14 15:20:30 +02:00
Marco Perini	91c5b5af43	fix(multi): updated multi pdf scraper with schema	2024-06-14 14:59:12 +02:00
Marco Vinciguerra	7a34562d50	refactoring of merging answers nodes	2024-06-13 21:41:54 +02:00
Marco Perini	283b61fafc	docs: better logging	2024-06-13 18:13:47 +02:00
Marco Vinciguerra	a6757aca57	Merge pull request #380 from Kshitij-Jande/main ... Strip out the scheme from the proxy server address URI.	2024-06-13 13:23:26 +02:00
Kshitij Jande	2a9ab69f4b	Strip out the scheme from the server address URI	2024-06-13 12:18:41 +05:30
Marco Vinciguerra	1705046cc7	Update pdf_scraper_graph.py	2024-06-12 18:00:12 +02:00
Marco Vinciguerra	dc1340e302	Update generate_answer_pdf_node.py	2024-06-12 15:47:04 +02:00
Marco Vinciguerra	e45f159a31	enhanced performance and readibility	2024-06-12 14:59:10 +02:00
Marco Vinciguerra	58a257f05b	update model tokens	2024-06-12 12:41:58 +02:00
Marco Vinciguerra	e6c7940a57	feat: add Parse_Node	2024-06-12 12:29:14 +02:00
Marco Vinciguerra	5065aa00e1	Merge branch 'pre/beta' into supercoder-327	2024-06-12 12:12:15 +02:00
supercoder-dev	d0e300af72	Update fetch_node.py	2024-06-12 14:32:01 +05:30
supercoder-dev	879c94a2b5	Update cleanup_html.py	2024-06-12 14:30:02 +05:30
supercoder-dev	828bdeedce	Update smart_scraper_graph.py	2024-06-12 14:28:33 +05:30
Marco Perini	6b4cdf92b8	fix: common params	2024-06-12 03:06:21 +02:00
Marco Perini	ab00f23d85	fix(node): fixed generate answer node pydantic schema	2024-06-12 01:40:49 +02:00