CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-06-28 21:01:55 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,951 Commits 2 Branches 448 Tags 17 MiB
ceede46673
Commit Graph

1951 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Marco Vinciguerra
ceede46673 fix: parse_node 2024-09-24 15:27:20 +02:00
Marco Vinciguerra
bc881b4aaa refctoring of the code
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Details
/ build (push) Has been cancelled
Details
Release / Build (push) Has been cancelled
Details
Release / Release (push) Has been cancelled
Details
2024-09-23 08:37:52 +02:00
semantic-release-bot
bfdd86fb96 ci(release): 1.23.0 [skip ci] 
## [1.23.0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.22.0...v1.23.0) (2024-09-23)

### Features

* update search_link_graph ([de10b28](de10b281ba))
 2024-09-23 06:28:13 +00:00
Marco Vinciguerra
de10b281ba feat: update search_link_graph 2024-09-23 08:26:36 +02:00
semantic-release-bot
369332b39c ci(release): 1.22.0 [skip ci] 
## [1.22.0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.21.1...v1.22.0) (2024-09-22)

### Features

* update search_link graph ([e724ae4](e724ae4492))
 2024-09-22 09:34:03 +00:00
Marco Vinciguerra
e724ae4492 feat: update search_link graph 2024-09-22 11:32:46 +02:00
Marco Vinciguerra
9b3695db53 Update search_link_node.py 2024-09-22 10:37:25 +02:00
Marco Vinciguerra
c5a3f893f1 refactoring of node names 2024-09-22 10:35:43 +02:00
semantic-release-bot
2ffeb3653c ci(release): 1.21.1 [skip ci] 
## [1.21.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.21.0...v1.21.1) (2024-09-21)

### Bug Fixes

* removed faiss ([86f6877](86f68770e9))
 2024-09-21 06:26:52 +00:00
Marco Vinciguerra
86f68770e9 fix: removed faiss 2024-09-21 08:25:22 +02:00
Marco Vinciguerra
6f0b9d852a update examples 2024-09-20 15:09:30 +02:00
Marco Vinciguerra
c234c23f5c
Merge pull request #683 from kashindra-mahato/patch-1 
Update abstract_graph.py
 2024-09-19 12:37:25 +02:00
Marco Vinciguerra
72873c1b38 refactoring of teh code 2024-09-19 12:34:25 +02:00
kashindra-mahato
8746958880
Update abstract_graph.py 
fix spelling error 'togetherai'
 2024-09-19 15:56:40 +05:45
semantic-release-bot
7fea150a96 ci(release): 1.21.0 [skip ci] 
## [1.21.0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.1...v1.21.0) (2024-09-19)

### Features

* **AbstractGraph:** add adjustable rate limit ([2859fb7](2859fb72d6))
* add copy for smart_scraper_multi_concat ([9e3171b](9e3171b9fa))
* add scrape_do_integration ([94e69a0](94e69a0515))
* add togheterai ([8f615ad](8f615adef3))
* added Bedrock and Mistral to exec info ([8a37c6b](8a37c6b793))
* ConcatNode.py added for heavy merge operations ([bd4b26d](bd4b26d7d7))
* fetch_node improved ([167f970](167f97040f))
* refactoring of the tokenization function ([ec6b164](ec6b164653))
* removed semchunk and used tikton ([1a7f21f](1a7f21fbf3))
* return urls in searchgraph ([afb6eb7](afb6eb7e47))
* updated pydantic to v2 ([eb89549](eb89549248))

### Bug Fixes

* Add mistral-common dependency ([7681a45](7681a4586a))
* Added support for nested structure ([66ea166](66ea166438))
* **AbstractGraph:** Bedrock init issues ([63a5d18](63a5d18486)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* correctly parsing output when using structured_output ([8e74ac5](8e74ac55a1))
* Error in pyproject dependencies ([5b5cb5b](5b5cb5b861))
* fetch_node condition ([3f45c17](3f45c17022))
* Fixed pydantic error on SearchGraphs ([039ba2e](039ba2e95a))
* **ScreenshotScraper:** impose dynamic imports ([b8ef937](b8ef93738e))
* **Ollama:** instance model from correct package ([398b2c5](398b2c556f))
* OmniScraerGraph working. ([c3d1b7c](c3d1b7c200))
* parse node ([947ebd2](947ebd2895))
* Parse Node scraping link and img urls allowing OmniScraper to work ([66a3b6d](66a3b6d6a3))
* **SmartScraper:** pass llm_model to ParseNode ([5242166](5242166575))
* **DeepSeek:** proper model initialization ([74dfc69](74dfc693f6))
* pyproject.toml ([812c73d](812c73d8aa))
* pyproject.toml dependencies ([b805aea](b805aea1de))
* Refactor code to use CustomOpenAiCallbackManager for exclusive access to get_openai_callback ([e657113](e657113ebc))
* Removed link_urls and img_ulrs from FetchNode output ([57337a0](57337a0a8c))
* screenshot scraper ([388630c](388630c0ff))
* screenshot_scraper ([ef7a589](ef7a5891dc))
* **ScreenShotScraper:** static import of optional dependencies ([52fe441](52fe441c5a))
* temporary fix for parse_node ([f2bb22d](f2bb22d8e9))
* update all nodes that were using MergeNode or IteratorNode ([a92dddb](a92dddb3e0))
* update generate answernode ([c348f67](c348f674ad))
* update pyproject.toml ([932412e](932412e325))

### chore

* **examples:** create Together AI examples ([34942de](34942deca5))

### Docs

* Updated the graph_config in the documentation. ([57a58e1](57a58e162e))

### Refactor

* Output parser code ([28b85a3](28b85a3b16))

### CI

* **release:** 1.16.0-beta.1 [skip ci] ([d7f6036](d7f6036f90))
* **release:** 1.16.0-beta.2 [skip ci] ([1c37d5d](1c37d5db1c))
* **release:** 1.16.0-beta.3 [skip ci] ([886c987](886c987172))
* **release:** 1.16.0-beta.4 [skip ci] ([ba5c7ad](ba5c7adcea))
* **release:** 1.17.0-beta.1 [skip ci] ([13efd4e](13efd4e3a4))
* **release:** 1.17.0-beta.10 [skip ci] ([af28885](af2888539e))
* **release:** 1.17.0-beta.11 [skip ci] ([a73fec5](a73fec5a98))
* **release:** 1.17.0-beta.2 [skip ci] ([08afc92](08afc9292e))
* **release:** 1.17.0-beta.3 [skip ci] ([fc55418](fc55418a45))
* **release:** 1.17.0-beta.4 [skip ci] ([5e99071](5e990719cf))
* **release:** 1.17.0-beta.5 [skip ci] ([16ab1bf](16ab1bf3d9))
* **release:** 1.17.0-beta.6 [skip ci] ([50c9c6b](50c9c6bd8c))
* **release:** 1.17.0-beta.7 [skip ci] ([4347afb](4347afb8d4)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* **release:** 1.17.0-beta.8 [skip ci] ([85c374e](85c374e4b3))
* **release:** 1.17.0-beta.9 [skip ci] ([77d0fd3](77d0fd3dba))
* **release:** 1.19.0-beta.1 [skip ci] ([eddcb79](eddcb79486)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633) [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* **release:** 1.19.0-beta.10 [skip ci] ([92f5df2](92f5df2828))
* **release:** 1.19.0-beta.11 [skip ci] ([edfb185](edfb1850ed))
* **release:** 1.19.0-beta.12 [skip ci] ([bd2afef](bd2afef87e))
* **release:** 1.19.0-beta.2 [skip ci] ([23a260c](23a260c51e))
* **release:** 1.19.0-beta.3 [skip ci] ([38cba96](38cba96ea3))
* **release:** 1.19.0-beta.4 [skip ci] ([24c38f9](24c38f945a))
* **release:** 1.19.0-beta.5 [skip ci] ([7621a7c](7621a7c7b7))
* **release:** 1.19.0-beta.6 [skip ci] ([ed8e173](ed8e1738c3))
* **release:** 1.19.0-beta.7 [skip ci] ([4ab26a2](4ab26a24a3))
* **release:** 1.19.0-beta.8 [skip ci] ([88b2c46](88b2c469ae))
* **release:** 1.19.0-beta.9 [skip ci] ([7ad6f21](7ad6f21ee2))
* **release:** 1.20.0-beta.1 [skip ci] ([cc8392e](cc8392e032)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633) [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633) [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633) [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* **release:** 1.20.0-beta.2 [skip ci] ([4f8b55d](4f8b55d747))
* **release:** 1.20.0-beta.3 [skip ci] ([cca783c](cca783cfeb))
* **release:** 1.20.0-beta.4 [skip ci] ([c81f970](c81f970196))
* **release:** 1.20.0-beta.5 [skip ci] ([b0fef3f](b0fef3fda8))
 2024-09-19 08:12:41 +00:00
Marco Vinciguerra
812c73d8aa fix: pyproject.toml 2024-09-19 10:11:11 +02:00
Marco Vinciguerra
8b51d50499
Merge pull request #682 from ScrapeGraphAI/temp 
Pre/beta
 2024-09-19 10:07:48 +02:00
Marco Vinciguerra
0cdd47ee06
Merge branch 'main' into temp 2024-09-19 10:07:40 +02:00
semantic-release-bot
b0fef3fda8 ci(release): 1.20.0-beta.5 [skip ci] 
## [1.20.0-beta.5](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.0-beta.4...v1.20.0-beta.5) (2024-09-18)

### Features

* added Bedrock and Mistral to exec info ([8a37c6b](8a37c6b793))
 2024-09-18 08:07:45 +00:00
Marco Vinciguerra
6ae2028387 Update CONTRIBUTING.md 2024-09-18 10:06:22 +02:00
Marco Vinciguerra
95a5ee2d35
Merge pull request #680 from LorenzoPaleari/exec-info-enhanced 
feat: added Bedrock and Mistral to exec info
 2024-09-18 10:05:45 +02:00
semantic-release-bot
c81f970196 ci(release): 1.20.0-beta.4 [skip ci] 
## [1.20.0-beta.4](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.0-beta.3...v1.20.0-beta.4) (2024-09-18)

### Bug Fixes

* update pyproject.toml ([932412e](932412e325))
 2024-09-18 08:00:39 +00:00
Marco Vinciguerra
107955357a Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegraph-ai into pre/beta 2024-09-18 09:59:15 +02:00
Marco Vinciguerra
932412e325 fix: update pyproject.toml 2024-09-18 09:59:14 +02:00
semantic-release-bot
cca783cfeb ci(release): 1.20.0-beta.3 [skip ci] 
## [1.20.0-beta.3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.0-beta.2...v1.20.0-beta.3) (2024-09-18)

### Features

* updated pydantic to v2 ([eb89549](eb89549248))

### Refactor

* Output parser code ([28b85a3](28b85a3b16))
 2024-09-18 07:55:58 +00:00
Marco Vinciguerra
e6e2ce6db4
Merge pull request #679 from LorenzoPaleari/output-parser-update 
feat: output parser and pydantic update
 2024-09-18 09:54:37 +02:00
Lorenzo Paleari
8a37c6b793
feat: added Bedrock and Mistral to exec info 2024-09-18 02:02:38 +02:00
Lorenzo Paleari
eb89549248
feat: updated pydantic to v2 2024-09-17 23:08:56 +02:00
Lorenzo Paleari
28b85a3b16
refactor: Output parser code 2024-09-17 23:07:16 +02:00
semantic-release-bot
4f8b55d747 ci(release): 1.20.0-beta.2 [skip ci] 
## [1.20.0-beta.2](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.0-beta.1...v1.20.0-beta.2) (2024-09-17)

### Bug Fixes

* Add mistral-common dependency ([7681a45](7681a4586a))
* Error in pyproject dependencies ([5b5cb5b](5b5cb5b861))
* fetch_node condition ([3f45c17](3f45c17022))
 2024-09-17 13:12:33 +00:00
Marco Vinciguerra
1b85f54455
Merge pull request #677 from LorenzoPaleari/fix-pyproject-error 
fix: Error in pyproject dependencies
 2024-09-17 15:11:08 +02:00
Lorenzo Paleari
5b5cb5b861
fix: Error in pyproject dependencies 2024-09-17 14:02:23 +02:00
semantic-release-bot
31f3f36408 ci(release): 1.20.1 [skip ci] 
## [1.20.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.0...v1.20.1) (2024-09-16)

### Bug Fixes

* fetch_node ([9e46b46](9e46b468c1))
 2024-09-16 16:13:20 +00:00
Marco Vinciguerra
a9d21edd86 Merge branch 'main' of https://github.com/ScrapeGraphAI/Scrapegraph-ai 2024-09-16 18:11:49 +02:00
Marco Vinciguerra
9e46b468c1 fix: fetch_node 2024-09-16 18:11:47 +02:00
semantic-release-bot
67625598e8 ci(release): 1.20.0 [skip ci] 
## [1.20.0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.19.0...v1.20.0) (2024-09-16)

### Features

* add grok integration for ollama ([59aa251](59aa2510e1))
 2024-09-16 13:44:46 +00:00
Marco Vinciguerra
59aa2510e1 feat: add grok integration for ollama 2024-09-16 15:43:02 +02:00
Marco Vinciguerra
3f45c17022 fix: fetch_node condition
Some checks failed
/ build (push) Has been cancelled
Details
Release / Build (push) Has been cancelled
Details
Release / Release (push) Has been cancelled
Details
2024-09-16 09:11:59 +02:00
Marco Vinciguerra
c717bb6d8e Merge branch 'main' into pre/beta 2024-09-16 09:08:37 +02:00
Marco Vinciguerra
438b8127db refctoring of the code 2024-09-15 11:20:08 +02:00
Marco Vinciguerra
cb505ceb67
Merge pull request #673 from LorenzoPaleari/fix-no-mistral_common-error 
fix: Add mistral-common dependency
 2024-09-15 10:47:55 +02:00
Lorenzo Paleari
7681a4586a
fix: Add mistral-common dependency 2024-09-14 22:49:12 +02:00
semantic-release-bot
cc8392e032 ci(release): 1.20.0-beta.1 [skip ci] 
## [1.20.0-beta.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.19.0...v1.20.0-beta.1) (2024-09-14)

### Features

* **AbstractGraph:** add adjustable rate limit ([2859fb7](2859fb72d6))
* add copy for smart_scraper_multi_concat ([9e3171b](9e3171b9fa))
* add scrape_do_integration ([94e69a0](94e69a0515))
* add togheterai ([8f615ad](8f615adef3))
* ConcatNode.py added for heavy merge operations ([bd4b26d](bd4b26d7d7))
* fetch_node improved ([167f970](167f97040f))
* refactoring of the tokenization function ([ec6b164](ec6b164653))
* removed semchunk and used tikton ([1a7f21f](1a7f21fbf3))
* return urls in searchgraph ([afb6eb7](afb6eb7e47))

### Bug Fixes

* Added support for nested structure ([66ea166](66ea166438))
* **AbstractGraph:** Bedrock init issues ([63a5d18](63a5d18486)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* correctly parsing output when using structured_output ([8e74ac5](8e74ac55a1))
* Fixed pydantic error on SearchGraphs ([039ba2e](039ba2e95a))
* **ScreenshotScraper:** impose dynamic imports ([b8ef937](b8ef93738e))
* **Ollama:** instance model from correct package ([398b2c5](398b2c556f))
* OmniScraerGraph working. ([c3d1b7c](c3d1b7c200))
* parse node ([947ebd2](947ebd2895))
* Parse Node scraping link and img urls allowing OmniScraper to work ([66a3b6d](66a3b6d6a3))
* **SmartScraper:** pass llm_model to ParseNode ([5242166](5242166575))
* **DeepSeek:** proper model initialization ([74dfc69](74dfc693f6))
* pyproject.toml dependencies ([b805aea](b805aea1de))
* Refactor code to use CustomOpenAiCallbackManager for exclusive access to get_openai_callback ([e657113](e657113ebc))
* Removed link_urls and img_ulrs from FetchNode output ([57337a0](57337a0a8c))
* screenshot scraper ([388630c](388630c0ff))
* screenshot_scraper ([ef7a589](ef7a5891dc))
* **ScreenShotScraper:** static import of optional dependencies ([52fe441](52fe441c5a))
* temporary fix for parse_node ([f2bb22d](f2bb22d8e9))
* update all nodes that were using MergeNode or IteratorNode ([a92dddb](a92dddb3e0))
* update generate answernode ([c348f67](c348f674ad))

### chore

* **examples:** create Together AI examples ([34942de](34942deca5))

### Docs

* Updated the graph_config in the documentation. ([57a58e1](57a58e162e))

### CI

* **release:** 1.16.0-beta.1 [skip ci] ([d7f6036](d7f6036f90))
* **release:** 1.16.0-beta.2 [skip ci] ([1c37d5d](1c37d5db1c))
* **release:** 1.16.0-beta.3 [skip ci] ([886c987](886c987172))
* **release:** 1.16.0-beta.4 [skip ci] ([ba5c7ad](ba5c7adcea))
* **release:** 1.17.0-beta.1 [skip ci] ([13efd4e](13efd4e3a4))
* **release:** 1.17.0-beta.10 [skip ci] ([af28885](af2888539e))
* **release:** 1.17.0-beta.11 [skip ci] ([a73fec5](a73fec5a98))
* **release:** 1.17.0-beta.2 [skip ci] ([08afc92](08afc9292e))
* **release:** 1.17.0-beta.3 [skip ci] ([fc55418](fc55418a45))
* **release:** 1.17.0-beta.4 [skip ci] ([5e99071](5e990719cf))
* **release:** 1.17.0-beta.5 [skip ci] ([16ab1bf](16ab1bf3d9))
* **release:** 1.17.0-beta.6 [skip ci] ([50c9c6b](50c9c6bd8c))
* **release:** 1.17.0-beta.7 [skip ci] ([4347afb](4347afb8d4)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* **release:** 1.17.0-beta.8 [skip ci] ([85c374e](85c374e4b3))
* **release:** 1.17.0-beta.9 [skip ci] ([77d0fd3](77d0fd3dba))
* **release:** 1.19.0-beta.1 [skip ci] ([eddcb79](eddcb79486)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633) [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* **release:** 1.19.0-beta.10 [skip ci] ([92f5df2](92f5df2828))
* **release:** 1.19.0-beta.11 [skip ci] ([edfb185](edfb1850ed))
* **release:** 1.19.0-beta.12 [skip ci] ([bd2afef](bd2afef87e))
* **release:** 1.19.0-beta.2 [skip ci] ([23a260c](23a260c51e))
* **release:** 1.19.0-beta.3 [skip ci] ([38cba96](38cba96ea3))
* **release:** 1.19.0-beta.4 [skip ci] ([24c38f9](24c38f945a))
* **release:** 1.19.0-beta.5 [skip ci] ([7621a7c](7621a7c7b7))
* **release:** 1.19.0-beta.6 [skip ci] ([ed8e173](ed8e1738c3))
* **release:** 1.19.0-beta.7 [skip ci] ([4ab26a2](4ab26a24a3))
* **release:** 1.19.0-beta.8 [skip ci] ([88b2c46](88b2c469ae))
* **release:** 1.19.0-beta.9 [skip ci] ([7ad6f21](7ad6f21ee2))
 2024-09-14 08:55:00 +00:00
Marco Vinciguerra
0eec93e324
Merge pull request #672 from ScrapeGraphAI/temp 
allignment
 2024-09-14 10:53:49 +02:00
Marco Vinciguerra
5d1fe68dea
Merge branch 'pre/beta' into temp 2024-09-14 10:53:43 +02:00
Marco Vinciguerra
dcef172e03 Update copy.py
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Details
/ build (push) Has been cancelled
Details
Release / Build (push) Has been cancelled
Details
Release / Release (push) Has been cancelled
Details
2024-09-14 10:43:47 +02:00
Marco Vinciguerra
3327312731
Merge pull request #671 from goasleep/fix/fix_boto3_client_copy 
fix:fix boto3 client copy
 2024-09-14 10:04:55 +02:00
semantic-release-bot
bd2afef87e ci(release): 1.19.0-beta.12 [skip ci] 
## [1.19.0-beta.12](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.19.0-beta.11...v1.19.0-beta.12) (2024-09-14)

### Bug Fixes

* Refactor code to use CustomOpenAiCallbackManager for exclusive access to get_openai_callback ([e657113](e657113ebc))

### Docs

* added telemetry info ([62912c2](62912c263e))
 2024-09-14 07:28:10 +00:00
Marco Vinciguerra
d7afdb1d1f
Merge pull request #670 from LorenzoPaleari/576-exec-info-misses-nested-graphs 
Added CustomOpenaiCallback to ensure exclusive access to nested data.
 2024-09-14 09:26:56 +02:00
smith peng
7e3f66e180 fix:fix boto3 client copy 2024-09-14 11:15:31 +08:00