1175 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Marco Vinciguerra	9dd2a63857	Merge pull request #379 from iamgodot/test ... test: fix tests for fetch node with proper mock&refactor	2024-06-13 11:03:46 +02:00
iamgodot	17dd936af7	test: fix tests for fetch node with proper mock&refactor	2024-06-12 22:45:43 -07:00
Marco Vinciguerra	cc9f5cc723	Merge pull request #375 from iamgodot/docs	2024-06-12 20:35:57 +02:00
iamgodot	071f3d1906	docs: fix label&logo for github action badges	2024-06-12 11:25:49 -07:00
Marco Vinciguerra	58a257f05b	update model tokens	2024-06-12 12:41:58 +02:00
semantic-release-bot	79b8326b5b	ci(release): 1.7.0-beta.5 [skip ci] ... ## [1.7.0-beta.5](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.7.0-beta.4...v1.7.0-beta.5) (2024-06-12) ### Features * update fetch node ([1e7f334](1e7f3349f3))	2024-06-12 10:14:40 +00:00
Marco Vinciguerra	9952d98190	Merge branch 'pre/beta' of https://github.com/VinciGit00/Scrapegraph-ai into pre/beta	2024-06-12 12:13:22 +02:00
Marco Vinciguerra	1e7f3349f3	feat: update fetch node	2024-06-12 12:13:07 +02:00
Marco Vinciguerra	0145b8f0e8	Merge pull request #372 from supercoder-dev/supercoder-327 ... Issue 327 Resolved	2024-06-12 12:12:28 +02:00
Marco Vinciguerra	5065aa00e1	Merge branch 'pre/beta' into supercoder-327	2024-06-12 12:12:15 +02:00
supercoder-dev	d0e300af72	Update fetch_node.py	2024-06-12 14:32:01 +05:30
supercoder-dev	879c94a2b5	Update cleanup_html.py	2024-06-12 14:30:02 +05:30
supercoder-dev	828bdeedce	Update smart_scraper_graph.py	2024-06-12 14:28:33 +05:30
semantic-release-bot	b4d7532c6c	ci(release): 1.7.0-beta.4 [skip ci] ... ## [1.7.0-beta.4](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.7.0-beta.3...v1.7.0-beta.4) (2024-06-12) ### Bug Fixes * common params ([6b4cdf9](6b4cdf92b8))	2024-06-12 01:07:46 +00:00
Marco Perini	6b4cdf92b8	fix: common params	2024-06-12 03:06:21 +02:00
semantic-release-bot	85a75c893a	ci(release): 1.7.0-beta.3 [skip ci] ... ## [1.7.0-beta.3](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.7.0-beta.2...v1.7.0-beta.3) (2024-06-11) ### Features * add caching ([d790361](d79036149a)) * add dynamic caching ([7ed2fe8](7ed2fe8ef0)) * add new chunking function ([e1f045b](e1f045b280)) * **merge:** add scriptcreatormulti, rag cache and semchunk ([15421ef](15421eff70)) * **schema:** merge scripts to follow pydantic schema ([5d692bf](5d692bff9e)) * refactoring of rag node ([7a13a68](7a13a6819f)) ### Bug Fixes * **cache:** correctly pass the node arguments and logging ([c881f64](c881f64209)) * **node:** fixed generate answer node pydantic schema ([ab00f23](ab00f23d85)) ### Docs * **cache:** added cache_path param ([edddb68](edddb682d0)) * **scriptcreator:** enhance documentation ([650c3aa](650c3aaa60))	2024-06-11 23:55:32 +00:00
Marco Perini	6f994cef8c	Merge pull request #370 from VinciGit00/dev ... Multiple Branches	2024-06-12 01:54:17 +02:00
Marco Perini	ab00f23d85	fix(node): fixed generate answer node pydantic schema	2024-06-12 01:40:49 +02:00
Marco Perini	15421eff70	feat(merge): add scriptcreatormulti, rag cache and semchunk	2024-06-12 01:23:01 +02:00
Marco Perini	650c3aaa60	docs(scriptcreator): enhance documentation	2024-06-12 01:16:50 +02:00
Marco Perini	a10b060409	Merge pull request #361 from VinciGit00/multi_scraper_implementation ... Multi scraper implementation	2024-06-12 00:49:18 +02:00
Marco Perini	5d692bff9e	feat(schema): merge scripts to follow pydantic schema	2024-06-12 00:48:08 +02:00
Marco Perini	893aaddc5c	Merge pull request #359 from VinciGit00/semchunk_integration ... feat: add new chunking function	2024-06-11 23:16:00 +02:00
Marco Perini	589da1d695	Merge pull request #351 from VinciGit00/faiss_integration ... Faiss integration	2024-06-11 23:15:23 +02:00
Marco Perini	edddb682d0	docs(cache): added cache_path param	2024-06-11 23:01:31 +02:00
Marco Perini	c881f64209	fix(cache): correctly pass the node arguments and logging	2024-06-11 22:56:09 +02:00
Marco Vinciguerra	fa951b4c8b	Merge pull request #368 from stevenmichaelthomas/wait-for-network-idle ... Add the ability to specify load state	2024-06-11 19:59:23 +02:00
Steven Thomas	8f405ff87a	Add the ability to specify load state	2024-06-11 11:22:39 -04:00
Marco Vinciguerra	81f89d88be	Update search_link_node.py	2024-06-10 16:14:24 +02:00
semantic-release-bot	e5bb5ae473	ci(release): 1.7.0-beta.2 [skip ci] ... ## [1.7.0-beta.2](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.7.0-beta.1...v1.7.0-beta.2) (2024-06-10) ### Features * Add tests for RobotsNode and update test setup ([b0511ae](b0511aeaaa)) * Add tests for SmartScraperGraph using sample text and configuration fixtures ([@tejhande](https://github.com/tejhande)) ([c927145](c927145bd0)) * Add tests for SmartScraperGraph using sample text and configuration fixtures ([@tejhande](https://github.com/tejhande)) ([9e7038c](9e7038c596)) * Add tests for SmartScraperGraph using sample text and configuration fixtures ([@tejhande](https://github.com/tejhande)) ([c286b16](c286b1649e)) * Add tests for SmartScraperGraph using sample text and configuration fixtures ([@tejhande](https://github.com/tejhande)) ([08f1be6](08f1be682b))	2024-06-10 08:41:23 +00:00
Marco Vinciguerra	0748e10e3a	Merge pull request #365 from tejhande/main	2024-06-10 10:40:16 +02:00
Tejas Amol Hande	40747c3e01	Merge branch 'main' into main	2024-06-10 13:36:44 +05:30
Tejas Amol Hande	c927145bd0	feat: Add tests for SmartScraperGraph using sample text and configuration fixtures (@tejhande) ... - Added pytest fixture to provide sample text from a file. - Added pytest fixture to provide graph configuration. - Implemented test_scraping_pipeline to test the execution of SmartScraperGraph. - Added assertions to verify the result is not None and to check the expected structure of the result. Contributed by @tejhande	2024-06-10 13:33:05 +05:30
Tejas Amol Hande	9e7038c596	feat: Add tests for SmartScraperGraph using sample text and configuration fixtures (@tejhande) ... - Added pytest fixture to provide sample text from a file. - Added pytest fixture to provide graph configuration. - Implemented test_scraping_pipeline to test the execution of SmartScraperGraph. - Added assertions to verify the result is not None and to check the expected structure of the result. Contributed by @tejhande	2024-06-10 13:28:48 +05:30
Tejas Amol Hande	c286b1649e	feat: Add tests for SmartScraperGraph using sample text and configuration fixtures (@tejhande) ... - Added pytest fixture to provide sample text from a file. - Added pytest fixture to provide graph configuration. - Implemented test_scraping_pipeline to test the execution of SmartScraperGraph. - Added assertions to verify the result is not None and to check the expected structure of the result. Contributed by @tejhande	2024-06-10 13:27:31 +05:30
Tejas Amol Hande	08f1be682b	feat: Add tests for SmartScraperGraph using sample text and configuration fixtures (@tejhande) ... - Added pytest fixture to provide sample text from a file. - Added pytest fixture to provide graph configuration. - Implemented test_scraping_pipeline to test the execution of SmartScraperGraph. - Added assertions to verify the result is not None and to check the expected structure of the result. Contributed by @your-github-username	2024-06-10 13:25:10 +05:30
Tejas Amol Hande	b0511aeaaa	feat: Add tests for RobotsNode and update test setup ... - Added pytest fixture to set up the RobotsNode with the initial state. - Implemented test_robots_node to test the execution of RobotsNode. - Used unittest.mock.patch to mock the execute method, ensuring faster and more reliable tests without actual network calls. - Added assertions to verify the correctness of the result and ensure the execute method is called once with the correct arguments.	2024-06-10 13:21:56 +05:30
semantic-release-bot	84a74b2f79	ci(release): 1.7.0-beta.1 [skip ci] ... ## [1.7.0-beta.1](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.6.0...v1.7.0-beta.1) (2024-06-09) ### Features * add csv scraper and xml scraper multi ([b408655](b4086550cc)) * **indexify-node:** add example ([5d1fbf8](5d1fbf806a)) * add forcing format as json ([5cfc101](5cfc10178a)) * add json as output ([5d20186](5d20186bf2)) * add json multiscraper ([5bda918](5bda918a39)) * add pdf scraper multi graph ([f5cbd80](f5cbd80c97)) * **pydantic:** added pydantic output schema ([376f758](376f758a76)) * **append_node:** append node to existing graph ([f8b08e0](f8b08e0b33)) * fix an if ([c8d556d](c8d556da4e)) * refactoring of abstract graph ([fff89f4](fff89f431f)) * refactoring of an in if ([244aada](244aada2de)) * removed a bug ([8de720d](8de720d379)) * removed rag node ([930f673](930f673747)) * **version:** update burr version ([cfa1336](cfa13368f4)) ### Bug Fixes * **schema:** fixed json output ([5c9843f](5c9843f141)) * oneapi model ([4fcb990](4fcb9902fe)) * typo in prompt ([4639f0c](4639f0cac5)) ### Docs * stylize badges in readme ([8696ade](8696adede7)) ### CI * **release:** 1.5.3-beta.1 [skip ci] ([6ea1d2c](6ea1d2c4d0)) * **release:** 1.5.3-beta.2 [skip ci] ([b57bcef](b57bcef5c1)) * **release:** 1.5.5-beta.1 [skip ci] ([38d138e](38d138e36f)) * **release:** 1.6.0-beta.1 [skip ci] ([1d217e4](1d217e4ae6)) * **release:** 1.6.0-beta.10 [skip ci] ([4d0d8fa](4d0d8fa453)) * **release:** 1.6.0-beta.11 [skip ci] ([3453ac0](3453ac01f5)) * **release:** 1.6.0-beta.2 [skip ci] ([ed1dc0b](ed1dc0be08)) * **release:** 1.6.0-beta.3 [skip ci] ([b70cb37](b70cb37c62)) * **release:** 1.6.0-beta.4 [skip ci] ([08a14ef](08a14efdd3)) * **release:** 1.6.0-beta.5 [skip ci] ([dde0c7e](dde0c7e27d)) * **release:** 1.6.0-beta.6 [skip ci] ([ac8e7c1](ac8e7c12fe)) * **release:** 1.6.0-beta.7 [skip ci] ([cab5f68](cab5f6828c)) * **release:** 1.6.0-beta.8 [skip ci] ([7a6f016](7a6f016f92)) * **release:** 1.6.0-beta.9 [skip ci] ([ca8aff8](ca8aff8d88))	2024-06-09 15:07:07 +00:00
Marco Vinciguerra	2f0568c83a	Merge pull request #364 from VinciGit00/all ... allignment	2024-06-09 17:05:59 +02:00
Marco Vinciguerra	8bec47eac2	Merge branch 'pre/beta' into all	2024-06-09 17:05:51 +02:00
semantic-release-bot	3453ac01f5	ci(release): 1.6.0-beta.11 [skip ci] ... ## [1.6.0-beta.11](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.6.0-beta.10...v1.6.0-beta.11) (2024-06-09) ### Bug Fixes * bug on generate_answer_node ([1d38ed1](1d38ed146a)) * getter ([67d83cf](67d83cff46)) * update openai tts class ([10672d6](10672d6ebb)) ### Docs * add Japanese README ([4559ab6](4559ab6db8)) * update japanese.md ([f0042a8](f0042a8e33)) * update README.md ([871e398](871e398a26)) ### Test * Enhance JSON scraping pipeline test ([d845a1b](d845a1ba7d)) ### CI * **release:** 1.5.5 [skip ci] ([3629215](36292150da)) * **release:** 1.5.6 [skip ci] ([49cdadf](49cdadf117)) * **release:** 1.5.7 [skip ci] ([c17daca](c17daca409))	2024-06-09 15:05:20 +00:00
Marco Vinciguerra	9326637cc3	Merge branch 'main' into pre/beta	2024-06-09 17:04:11 +02:00
semantic-release-bot	58086eef47	ci(release): 1.6.0 [skip ci] ... ## [1.6.0](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.5.7...v1.6.0) (2024-06-09) ### Features * Add tests for RobotsNode and update test setup ([dedfa2e](dedfa2eaf0)) ### Test * Enhance JSON scraping pipeline test ([d845a1b](d845a1ba7d))	2024-06-09 15:00:59 +00:00
Marco Vinciguerra	e688480771	Merge pull request #362 from tejhande/patch-4 ... feat: Add tests for RobotsNode and update test setup	2024-06-09 16:59:50 +02:00
Marco Vinciguerra	2781c3c46b	Merge pull request #355 from tejhande/patch-3 ... Test ScriptCreatorGraph and print execution info	2024-06-09 16:57:36 +02:00
tejhande	13f8ca56b2	"Refactor SearchLinkNode test: simplify setup, add patching for execute method, and enhance assertions"	2024-06-09 19:48:33 +05:30
Tejas Amol Hande	dedfa2eaf0	feat: Add tests for RobotsNode and update test setup ... - Added pytest fixture to set up the RobotsNode with the initial state. - Implemented test_robots_node to test the execution of RobotsNode. - Used unittest.mock.patch to mock the execute method, ensuring faster and more reliable tests without actual network calls. - Added assertions to verify the correctness of the result and ensure the execute method is called once with the correct arguments.	2024-06-09 19:39:11 +05:30
Marco Vinciguerra	14d10117b7	Merge pull request #354 from tejhande/patch-2 ... Enhance tests for FetchNode with mocking	2024-06-09 15:50:27 +02:00
Marco Vinciguerra	bde02492c0	add examples	2024-06-09 15:26:56 +02:00
Marco Vinciguerra	fe8083fe48	Update pdf_scraper_graph_haiku.py	2024-06-09 10:02:29 +02:00