CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-07-01 21:00:48 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,957 Commits 2 Branches 448 Tags 17 MiB
1ffdd3f4e9
Commit Graph

1957 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Marco Vinciguerra
1ffdd3f4e9 Update models_tokens.py
Some checks failed
/ build (push) Has been cancelled
Details
Release / Build (push) Has been cancelled
Details
Release / Release (push) Has been cancelled
Details
2024-09-23 20:56:53 +02:00
Marco Vinciguerra
d0969feb7c removed unused imports and comments + removed dead code 2024-09-23 09:25:13 +02:00
semantic-release-bot
ba4e863f14 ci(release): 1.21.2-beta.2 [skip ci] 
## [1.21.2-beta.2](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.21.2-beta.1...v1.21.2-beta.2) (2024-09-23)

### Bug Fixes

* graph Iterator node ([8ce08ba](8ce08baf01))
* issue about parser ([7eda6bc](7eda6bc06b))
 2024-09-23 06:23:22 +00:00
Marco Vinciguerra
390ad8282d
Merge pull request #689 from ScrapeGraphAI/687-smartscrapermulticoncatgraph-error-with-bedrock 
687 smartscrapermulticoncatgraph error with bedrock
 2024-09-23 08:22:04 +02:00
Marco Vinciguerra
8ce08baf01 fix: graph Iterator node 2024-09-22 22:25:01 +02:00
Marco Vinciguerra
69880b680a Update generate_answer_node.py 2024-09-22 21:16:16 +02:00
Marco Vinciguerra
65b8675586 Update smart_scraper_multi_concat_graph.py 2024-09-22 21:10:57 +02:00
Marco Vinciguerra
ec9f8ff496 Merge branch '687-smartscrapermulticoncatgraph-error-with-bedrock' of https://github.com/ScrapeGraphAI/Scrapegraph-ai into 687-smartscrapermulticoncatgraph-error-with-bedrock 2024-09-22 18:55:09 +02:00
Marco Vinciguerra
7eda6bc06b fix: issue about parser 2024-09-22 18:55:05 +02:00
semantic-release-bot
dd0f260e75 ci(release): 1.21.2-beta.1 [skip ci] 
## [1.21.2-beta.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.21.1...v1.21.2-beta.1) (2024-09-22)

### Bug Fixes

* chat for bedrock ([f9b121f](f9b121f765))
 2024-09-22 14:51:41 +00:00
Marco Vinciguerra
b28e12ba54
Merge pull request #688 from ScrapeGraphAI/687-smartscrapermulticoncatgraph-error-with-bedrock 
fix: chat for bedrock
 2024-09-22 16:50:23 +02:00
Marco Vinciguerra
f9b121f765 fix: chat for bedrock 2024-09-22 16:47:19 +02:00
Marco Vinciguerra
1f03354b1c
Merge pull request #686 from ScrapeGraphAI/main
Some checks failed
/ build (push) Has been cancelled
Details
Release / Build (push) Has been cancelled
Details
Release / Release (push) Has been cancelled
Details 
allginement
 2024-09-21 08:30:11 +02:00
semantic-release-bot
2ffeb3653c ci(release): 1.21.1 [skip ci] 
## [1.21.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.21.0...v1.21.1) (2024-09-21)

### Bug Fixes

* removed faiss ([86f6877](86f68770e9))
 2024-09-21 06:26:52 +00:00
Marco Vinciguerra
86f68770e9 fix: removed faiss 2024-09-21 08:25:22 +02:00
Marco Vinciguerra
6f0b9d852a update examples 2024-09-20 15:09:30 +02:00
Marco Vinciguerra
5ab090de2e
Merge pull request #684 from ScrapeGraphAI/main 
allignment
 2024-09-19 12:39:35 +02:00
Marco Vinciguerra
c234c23f5c
Merge pull request #683 from kashindra-mahato/patch-1 
Update abstract_graph.py
 2024-09-19 12:37:25 +02:00
Marco Vinciguerra
72873c1b38 refactoring of teh code 2024-09-19 12:34:25 +02:00
kashindra-mahato
8746958880
Update abstract_graph.py 
fix spelling error 'togetherai'
 2024-09-19 15:56:40 +05:45
semantic-release-bot
7fea150a96 ci(release): 1.21.0 [skip ci] 
## [1.21.0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.1...v1.21.0) (2024-09-19)

### Features

* **AbstractGraph:** add adjustable rate limit ([2859fb7](2859fb72d6))
* add copy for smart_scraper_multi_concat ([9e3171b](9e3171b9fa))
* add scrape_do_integration ([94e69a0](94e69a0515))
* add togheterai ([8f615ad](8f615adef3))
* added Bedrock and Mistral to exec info ([8a37c6b](8a37c6b793))
* ConcatNode.py added for heavy merge operations ([bd4b26d](bd4b26d7d7))
* fetch_node improved ([167f970](167f97040f))
* refactoring of the tokenization function ([ec6b164](ec6b164653))
* removed semchunk and used tikton ([1a7f21f](1a7f21fbf3))
* return urls in searchgraph ([afb6eb7](afb6eb7e47))
* updated pydantic to v2 ([eb89549](eb89549248))

### Bug Fixes

* Add mistral-common dependency ([7681a45](7681a4586a))
* Added support for nested structure ([66ea166](66ea166438))
* **AbstractGraph:** Bedrock init issues ([63a5d18](63a5d18486)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* correctly parsing output when using structured_output ([8e74ac5](8e74ac55a1))
* Error in pyproject dependencies ([5b5cb5b](5b5cb5b861))
* fetch_node condition ([3f45c17](3f45c17022))
* Fixed pydantic error on SearchGraphs ([039ba2e](039ba2e95a))
* **ScreenshotScraper:** impose dynamic imports ([b8ef937](b8ef93738e))
* **Ollama:** instance model from correct package ([398b2c5](398b2c556f))
* OmniScraerGraph working. ([c3d1b7c](c3d1b7c200))
* parse node ([947ebd2](947ebd2895))
* Parse Node scraping link and img urls allowing OmniScraper to work ([66a3b6d](66a3b6d6a3))
* **SmartScraper:** pass llm_model to ParseNode ([5242166](5242166575))
* **DeepSeek:** proper model initialization ([74dfc69](74dfc693f6))
* pyproject.toml ([812c73d](812c73d8aa))
* pyproject.toml dependencies ([b805aea](b805aea1de))
* Refactor code to use CustomOpenAiCallbackManager for exclusive access to get_openai_callback ([e657113](e657113ebc))
* Removed link_urls and img_ulrs from FetchNode output ([57337a0](57337a0a8c))
* screenshot scraper ([388630c](388630c0ff))
* screenshot_scraper ([ef7a589](ef7a5891dc))
* **ScreenShotScraper:** static import of optional dependencies ([52fe441](52fe441c5a))
* temporary fix for parse_node ([f2bb22d](f2bb22d8e9))
* update all nodes that were using MergeNode or IteratorNode ([a92dddb](a92dddb3e0))
* update generate answernode ([c348f67](c348f674ad))
* update pyproject.toml ([932412e](932412e325))

### chore

* **examples:** create Together AI examples ([34942de](34942deca5))

### Docs

* Updated the graph_config in the documentation. ([57a58e1](57a58e162e))

### Refactor

* Output parser code ([28b85a3](28b85a3b16))

### CI

* **release:** 1.16.0-beta.1 [skip ci] ([d7f6036](d7f6036f90))
* **release:** 1.16.0-beta.2 [skip ci] ([1c37d5d](1c37d5db1c))
* **release:** 1.16.0-beta.3 [skip ci] ([886c987](886c987172))
* **release:** 1.16.0-beta.4 [skip ci] ([ba5c7ad](ba5c7adcea))
* **release:** 1.17.0-beta.1 [skip ci] ([13efd4e](13efd4e3a4))
* **release:** 1.17.0-beta.10 [skip ci] ([af28885](af2888539e))
* **release:** 1.17.0-beta.11 [skip ci] ([a73fec5](a73fec5a98))
* **release:** 1.17.0-beta.2 [skip ci] ([08afc92](08afc9292e))
* **release:** 1.17.0-beta.3 [skip ci] ([fc55418](fc55418a45))
* **release:** 1.17.0-beta.4 [skip ci] ([5e99071](5e990719cf))
* **release:** 1.17.0-beta.5 [skip ci] ([16ab1bf](16ab1bf3d9))
* **release:** 1.17.0-beta.6 [skip ci] ([50c9c6b](50c9c6bd8c))
* **release:** 1.17.0-beta.7 [skip ci] ([4347afb](4347afb8d4)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* **release:** 1.17.0-beta.8 [skip ci] ([85c374e](85c374e4b3))
* **release:** 1.17.0-beta.9 [skip ci] ([77d0fd3](77d0fd3dba))
* **release:** 1.19.0-beta.1 [skip ci] ([eddcb79](eddcb79486)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633) [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* **release:** 1.19.0-beta.10 [skip ci] ([92f5df2](92f5df2828))
* **release:** 1.19.0-beta.11 [skip ci] ([edfb185](edfb1850ed))
* **release:** 1.19.0-beta.12 [skip ci] ([bd2afef](bd2afef87e))
* **release:** 1.19.0-beta.2 [skip ci] ([23a260c](23a260c51e))
* **release:** 1.19.0-beta.3 [skip ci] ([38cba96](38cba96ea3))
* **release:** 1.19.0-beta.4 [skip ci] ([24c38f9](24c38f945a))
* **release:** 1.19.0-beta.5 [skip ci] ([7621a7c](7621a7c7b7))
* **release:** 1.19.0-beta.6 [skip ci] ([ed8e173](ed8e1738c3))
* **release:** 1.19.0-beta.7 [skip ci] ([4ab26a2](4ab26a24a3))
* **release:** 1.19.0-beta.8 [skip ci] ([88b2c46](88b2c469ae))
* **release:** 1.19.0-beta.9 [skip ci] ([7ad6f21](7ad6f21ee2))
* **release:** 1.20.0-beta.1 [skip ci] ([cc8392e](cc8392e032)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633) [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633) [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633) [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* **release:** 1.20.0-beta.2 [skip ci] ([4f8b55d](4f8b55d747))
* **release:** 1.20.0-beta.3 [skip ci] ([cca783c](cca783cfeb))
* **release:** 1.20.0-beta.4 [skip ci] ([c81f970](c81f970196))
* **release:** 1.20.0-beta.5 [skip ci] ([b0fef3f](b0fef3fda8))
 2024-09-19 08:12:41 +00:00
Marco Vinciguerra
812c73d8aa fix: pyproject.toml 2024-09-19 10:11:11 +02:00
Marco Vinciguerra
8b51d50499
Merge pull request #682 from ScrapeGraphAI/temp 
Pre/beta
 2024-09-19 10:07:48 +02:00
Marco Vinciguerra
0cdd47ee06
Merge branch 'main' into temp 2024-09-19 10:07:40 +02:00
semantic-release-bot
b0fef3fda8 ci(release): 1.20.0-beta.5 [skip ci] 
## [1.20.0-beta.5](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.0-beta.4...v1.20.0-beta.5) (2024-09-18)

### Features

* added Bedrock and Mistral to exec info ([8a37c6b](8a37c6b793))
 2024-09-18 08:07:45 +00:00
Marco Vinciguerra
6ae2028387 Update CONTRIBUTING.md 2024-09-18 10:06:22 +02:00
Marco Vinciguerra
95a5ee2d35
Merge pull request #680 from LorenzoPaleari/exec-info-enhanced 
feat: added Bedrock and Mistral to exec info
 2024-09-18 10:05:45 +02:00
semantic-release-bot
c81f970196 ci(release): 1.20.0-beta.4 [skip ci] 
## [1.20.0-beta.4](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.0-beta.3...v1.20.0-beta.4) (2024-09-18)

### Bug Fixes

* update pyproject.toml ([932412e](932412e325))
 2024-09-18 08:00:39 +00:00
Marco Vinciguerra
107955357a Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegraph-ai into pre/beta 2024-09-18 09:59:15 +02:00
Marco Vinciguerra
932412e325 fix: update pyproject.toml 2024-09-18 09:59:14 +02:00
semantic-release-bot
cca783cfeb ci(release): 1.20.0-beta.3 [skip ci] 
## [1.20.0-beta.3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.0-beta.2...v1.20.0-beta.3) (2024-09-18)

### Features

* updated pydantic to v2 ([eb89549](eb89549248))

### Refactor

* Output parser code ([28b85a3](28b85a3b16))
 2024-09-18 07:55:58 +00:00
Marco Vinciguerra
e6e2ce6db4
Merge pull request #679 from LorenzoPaleari/output-parser-update 
feat: output parser and pydantic update
 2024-09-18 09:54:37 +02:00
Lorenzo Paleari
8a37c6b793
feat: added Bedrock and Mistral to exec info 2024-09-18 02:02:38 +02:00
Lorenzo Paleari
eb89549248
feat: updated pydantic to v2 2024-09-17 23:08:56 +02:00
Lorenzo Paleari
28b85a3b16
refactor: Output parser code 2024-09-17 23:07:16 +02:00
semantic-release-bot
4f8b55d747 ci(release): 1.20.0-beta.2 [skip ci] 
## [1.20.0-beta.2](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.0-beta.1...v1.20.0-beta.2) (2024-09-17)

### Bug Fixes

* Add mistral-common dependency ([7681a45](7681a4586a))
* Error in pyproject dependencies ([5b5cb5b](5b5cb5b861))
* fetch_node condition ([3f45c17](3f45c17022))
 2024-09-17 13:12:33 +00:00
Marco Vinciguerra
1b85f54455
Merge pull request #677 from LorenzoPaleari/fix-pyproject-error 
fix: Error in pyproject dependencies
 2024-09-17 15:11:08 +02:00
Lorenzo Paleari
5b5cb5b861
fix: Error in pyproject dependencies 2024-09-17 14:02:23 +02:00
semantic-release-bot
31f3f36408 ci(release): 1.20.1 [skip ci] 
## [1.20.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.20.0...v1.20.1) (2024-09-16)

### Bug Fixes

* fetch_node ([9e46b46](9e46b468c1))
 2024-09-16 16:13:20 +00:00
Marco Vinciguerra
a9d21edd86 Merge branch 'main' of https://github.com/ScrapeGraphAI/Scrapegraph-ai 2024-09-16 18:11:49 +02:00
Marco Vinciguerra
9e46b468c1 fix: fetch_node 2024-09-16 18:11:47 +02:00
semantic-release-bot
67625598e8 ci(release): 1.20.0 [skip ci] 
## [1.20.0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.19.0...v1.20.0) (2024-09-16)

### Features

* add grok integration for ollama ([59aa251](59aa2510e1))
 2024-09-16 13:44:46 +00:00
Marco Vinciguerra
59aa2510e1 feat: add grok integration for ollama 2024-09-16 15:43:02 +02:00
Marco Vinciguerra
3f45c17022 fix: fetch_node condition
Some checks failed
/ build (push) Has been cancelled
Details
Release / Build (push) Has been cancelled
Details
Release / Release (push) Has been cancelled
Details
2024-09-16 09:11:59 +02:00
Marco Vinciguerra
c717bb6d8e Merge branch 'main' into pre/beta 2024-09-16 09:08:37 +02:00
Marco Vinciguerra
438b8127db refctoring of the code 2024-09-15 11:20:08 +02:00
Marco Vinciguerra
cb505ceb67
Merge pull request #673 from LorenzoPaleari/fix-no-mistral_common-error 
fix: Add mistral-common dependency
 2024-09-15 10:47:55 +02:00
Lorenzo Paleari
7681a4586a
fix: Add mistral-common dependency 2024-09-14 22:49:12 +02:00
semantic-release-bot
cc8392e032 ci(release): 1.20.0-beta.1 [skip ci] 
## [1.20.0-beta.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.19.0...v1.20.0-beta.1) (2024-09-14)

### Features

* **AbstractGraph:** add adjustable rate limit ([2859fb7](2859fb72d6))
* add copy for smart_scraper_multi_concat ([9e3171b](9e3171b9fa))
* add scrape_do_integration ([94e69a0](94e69a0515))
* add togheterai ([8f615ad](8f615adef3))
* ConcatNode.py added for heavy merge operations ([bd4b26d](bd4b26d7d7))
* fetch_node improved ([167f970](167f97040f))
* refactoring of the tokenization function ([ec6b164](ec6b164653))
* removed semchunk and used tikton ([1a7f21f](1a7f21fbf3))
* return urls in searchgraph ([afb6eb7](afb6eb7e47))

### Bug Fixes

* Added support for nested structure ([66ea166](66ea166438))
* **AbstractGraph:** Bedrock init issues ([63a5d18](63a5d18486)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* correctly parsing output when using structured_output ([8e74ac5](8e74ac55a1))
* Fixed pydantic error on SearchGraphs ([039ba2e](039ba2e95a))
* **ScreenshotScraper:** impose dynamic imports ([b8ef937](b8ef93738e))
* **Ollama:** instance model from correct package ([398b2c5](398b2c556f))
* OmniScraerGraph working. ([c3d1b7c](c3d1b7c200))
* parse node ([947ebd2](947ebd2895))
* Parse Node scraping link and img urls allowing OmniScraper to work ([66a3b6d](66a3b6d6a3))
* **SmartScraper:** pass llm_model to ParseNode ([5242166](5242166575))
* **DeepSeek:** proper model initialization ([74dfc69](74dfc693f6))
* pyproject.toml dependencies ([b805aea](b805aea1de))
* Refactor code to use CustomOpenAiCallbackManager for exclusive access to get_openai_callback ([e657113](e657113ebc))
* Removed link_urls and img_ulrs from FetchNode output ([57337a0](57337a0a8c))
* screenshot scraper ([388630c](388630c0ff))
* screenshot_scraper ([ef7a589](ef7a5891dc))
* **ScreenShotScraper:** static import of optional dependencies ([52fe441](52fe441c5a))
* temporary fix for parse_node ([f2bb22d](f2bb22d8e9))
* update all nodes that were using MergeNode or IteratorNode ([a92dddb](a92dddb3e0))
* update generate answernode ([c348f67](c348f674ad))

### chore

* **examples:** create Together AI examples ([34942de](34942deca5))

### Docs

* Updated the graph_config in the documentation. ([57a58e1](57a58e162e))

### CI

* **release:** 1.16.0-beta.1 [skip ci] ([d7f6036](d7f6036f90))
* **release:** 1.16.0-beta.2 [skip ci] ([1c37d5d](1c37d5db1c))
* **release:** 1.16.0-beta.3 [skip ci] ([886c987](886c987172))
* **release:** 1.16.0-beta.4 [skip ci] ([ba5c7ad](ba5c7adcea))
* **release:** 1.17.0-beta.1 [skip ci] ([13efd4e](13efd4e3a4))
* **release:** 1.17.0-beta.10 [skip ci] ([af28885](af2888539e))
* **release:** 1.17.0-beta.11 [skip ci] ([a73fec5](a73fec5a98))
* **release:** 1.17.0-beta.2 [skip ci] ([08afc92](08afc9292e))
* **release:** 1.17.0-beta.3 [skip ci] ([fc55418](fc55418a45))
* **release:** 1.17.0-beta.4 [skip ci] ([5e99071](5e990719cf))
* **release:** 1.17.0-beta.5 [skip ci] ([16ab1bf](16ab1bf3d9))
* **release:** 1.17.0-beta.6 [skip ci] ([50c9c6b](50c9c6bd8c))
* **release:** 1.17.0-beta.7 [skip ci] ([4347afb](4347afb8d4)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* **release:** 1.17.0-beta.8 [skip ci] ([85c374e](85c374e4b3))
* **release:** 1.17.0-beta.9 [skip ci] ([77d0fd3](77d0fd3dba))
* **release:** 1.19.0-beta.1 [skip ci] ([eddcb79](eddcb79486)), closes [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633) [#633](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/633)
* **release:** 1.19.0-beta.10 [skip ci] ([92f5df2](92f5df2828))
* **release:** 1.19.0-beta.11 [skip ci] ([edfb185](edfb1850ed))
* **release:** 1.19.0-beta.12 [skip ci] ([bd2afef](bd2afef87e))
* **release:** 1.19.0-beta.2 [skip ci] ([23a260c](23a260c51e))
* **release:** 1.19.0-beta.3 [skip ci] ([38cba96](38cba96ea3))
* **release:** 1.19.0-beta.4 [skip ci] ([24c38f9](24c38f945a))
* **release:** 1.19.0-beta.5 [skip ci] ([7621a7c](7621a7c7b7))
* **release:** 1.19.0-beta.6 [skip ci] ([ed8e173](ed8e1738c3))
* **release:** 1.19.0-beta.7 [skip ci] ([4ab26a2](4ab26a24a3))
* **release:** 1.19.0-beta.8 [skip ci] ([88b2c46](88b2c469ae))
* **release:** 1.19.0-beta.9 [skip ci] ([7ad6f21](7ad6f21ee2))
 2024-09-14 08:55:00 +00:00
Marco Vinciguerra
0eec93e324
Merge pull request #672 from ScrapeGraphAI/temp 
allignment
 2024-09-14 10:53:49 +02:00