CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-07-12 21:01:56 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
853 Commits 2 Branches 448 Tags 17 MiB
cfaf7eebd8
Commit Graph

853 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Marco Perini
cfaf7eebd8
Merge pull request #284 from DAGWorks-Inc/burr_integration 
Updates Burr bridge to use class-based API
 2024-05-21 22:03:59 +02:00
elijahbenizzy
d96840fa65 Updates Burr bridge to use class-based API 
(also makes the App ID a unique one for debugging, it kept getting
confused)
 2024-05-21 12:57:47 -07:00
Marco Perini
6cbd84f254 feat(burr-bridge): BurrBridge class to integrate inside BaseGraph 2024-05-21 19:57:10 +02:00
Marco Vinciguerra
0b5cdd48fd
Merge pull request #246 from VinciGit00/main 
reallignment
 2024-05-15 15:19:05 +02:00
semantic-release-bot
51200d6918 ci(release): 1.1.0 [skip ci] 
## [1.1.0](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.0.1...v1.1.0) (2024-05-15)

### Features

* add turboscraper (alfa) ([51aa109](51aa109e42))
* new search_graph ([67d5fbf](67d5fbf816))

### Docs

* **rye:** replaced poetry with rye ([efb781f](efb781f950))
 2024-05-15 13:18:52 +00:00
Marco Vinciguerra
22cd9e3605
Merge branch 'search_link_context' into main 2024-05-15 15:16:57 +02:00
Marco Perini
efb781f950
docs(rye): replaced poetry with rye 2024-05-15 15:15:17 +02:00
VinciGit00
694d3ab0f3 Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-05-15 15:10:53 +02:00
VinciGit00
7ccd51a879 add rye update script bash 2024-05-15 15:10:51 +02:00
semantic-release-bot
a81d2b7a80 ci(release): 1.0.1 [skip ci] 
## [1.0.1](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.0.0...v1.0.1) (2024-05-15)

### Bug Fixes

* **searchgraph:** used shallow copy to serialize obj ([096b665](096b665c01))
 2024-05-15 13:07:36 +00:00
Marco Perini
096b665c01 fix(searchgraph): used shallow copy to serialize obj 2024-05-15 15:05:47 +02:00
Marco Vinciguerra
24d56af97c
Update pyproject.toml 2024-05-15 14:10:56 +02:00
semantic-release-bot
c0b6f0260f ci(release): 1.0.0 [skip ci] 
## [1.0.0](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.11.1...v1.0.0) (2024-05-15)

### ⚠ BREAKING CHANGES

* **package manager:** move from poetry to rye

### chore

* **package manager:** move from poetry to rye ([8fc2510](8fc2510b37)), closes [#198](https://github.com/VinciGit00/Scrapegraph-ai/issues/198)

### Docs

* **main-readme:** fixed some typos ([78d1940](78d1940235))
 2024-05-15 11:45:54 +00:00
Marco Vinciguerra
672bd29b65
Merge pull request #244 from f-aguzzi/main 
chore(package manager)!: move from poetry to rye
 2024-05-15 13:43:58 +02:00
Federico Aguzzi
8fc2510b37 chore(package manager)!: move from poetry to rye 
closes #198
 2024-05-15 13:42:58 +02:00
Marco Perini
78d1940235
docs(main-readme): fixed some typos 2024-05-15 12:09:14 +02:00
semantic-release-bot
6effe25900 ci(release): 0.11.1 [skip ci] 
## [0.11.1](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.11.0...v0.11.1) (2024-05-14)

### Bug Fixes

* **docs:** requirements-dev ([b0a67ba](b0a67ba387))
 2024-05-14 16:14:46 +00:00
Marco Perini
b0a67ba387 fix(docs): requirements-dev 2024-05-14 18:13:24 +02:00
semantic-release-bot
c55a3b1162 ci(release): 0.11.0 [skip ci] 
## [0.11.0](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.10.1...v0.11.0) (2024-05-14)

### Features

* **parallel-exeuction:** add asyncio event loop dispatcher with semaphore for parallel graph instances ([627cbee](627cbeeb20))
* **webdriver-backend:** add dynamic import scripts from module and file ([db2234b](db2234bf5d))
* add gpt-4o ([52a4a3b](52a4a3b22d)), closes [#232](https://github.com/VinciGit00/Scrapegraph-ai/issues/232)
* add new prompt info ([e2350ed](e2350eda62))
* **proxy-rotation:** add parse (IP address) or search (from broker) functionality for proxy rotation ([2170131](217013181d))
* add support for deepseek-chat ([156b67b](156b67b91e)), closes [#222](https://github.com/VinciGit00/Scrapegraph-ai/issues/222)
* Add support for passing pdf path as source ([f10f3b1](f10f3b1438))
* **omni-search:** added omni search graph and updated docs ([fcb3abb](fcb3abb01d))
* added proxy rotation ([0c36a7e](0c36a7ec1f))
* **safe-web-driver:** enchanced the original `AsyncChromiumLoader` web driver with proxy protection and flexible kwargs and backend ([768719c](768719cce8))
* **gpt-4o:** image to text single node test ([90955ca](90955ca52f))
* revert fetch_node ([864aa91](864aa91326))
* **batchsize:** tested different batch sizes and systems ([a8d5e7d](a8d5e7db05))
* update info ([4ed0fb8](4ed0fb89c3))
* **omni-scraper:** working OmniScraperGraph with images ([a296927](a296927624))

### Bug Fixes

* **pytest:** add dependency for mocking testing functions ([2f4fd45](2f4fd45700))
* add json integration ([0ab31c3](0ab31c3fdb))
* Augment the information getting fetched from a webpage ([f8ce3d5](f8ce3d5916))
* bug for claude ([d0167de](d0167dee71))
* **fetch_node:** bug in handling local files ([a6e1813](a6e1813ddd))
* **chromium-loader:** ensure it subclasses langchain's base loader ([b54d984](b54d984c13))
* fixed bugs for csv and xml ([324e977](324e977b85))
* limit python version to < 3.12 ([a37fbbc](a37fbbcbcf))
* **proxy-rotation:** removed duplicated arg and passed the loader_kwarhs correctly to the node ([1e9a564](1e9a564616))
* **fetch-node:** removed isSoup from default ([0c15947](0c1594737f))
* **proxy-rotation:** removed max_shape duplicate ([5d6d996](5d6d996e8f))
* **asyncio:** replaced deepcopy with copy due to serialization problems ([dedc733](dedc733047))

### chore

* update models_tokens.py with new model configurations ([d9752b1](d9752b1619))

### Docs

* add diagram showing general structure/flow of the library ([13ae918](13ae9180ac))
* **refactor:** added proxy-rotation usage and refactor readthedocs ([e256b75](e256b758b2))
* **refactor:** changed example ([c7ec114](c7ec114274))
* **concurrent:** refactor theme and added benchmarck searchgraph ([ced2bbc](ced2bbcdc9))
* update overview diagram with more models ([b441b30](b441b30a5c))

### CI

* **release:** 0.10.0-beta.3 [skip ci] ([ad32298](ad32298e70))
* **release:** 0.10.0-beta.4 [skip ci] ([548bff9](548bff9d77))
* **release:** 0.10.0-beta.5 [skip ci] ([28c9dce](28c9dce7cb))
* **release:** 0.10.0-beta.6 [skip ci] ([460d292](460d292af2))
* **release:** 0.11.0-beta.1 [skip ci] ([63c0dd9](63c0dd9372))
* **release:** 0.11.0-beta.10 [skip ci] ([218b8ed](218b8ede8a)), closes [#232](https://github.com/VinciGit00/Scrapegraph-ai/issues/232)
* **release:** 0.11.0-beta.11 [skip ci] ([8727d03](8727d03384))
* **release:** 0.11.0-beta.2 [skip ci] ([7ae50c0](7ae50c035e))
* **release:** 0.11.0-beta.3 [skip ci] ([106fb12](106fb12531)), closes [#222](https://github.com/VinciGit00/Scrapegraph-ai/issues/222)
* **release:** 0.11.0-beta.4 [skip ci] ([4ccddda](4ccddda5eb))
* **release:** 0.11.0-beta.5 [skip ci] ([353382b](353382b4d3))
* **release:** 0.11.0-beta.6 [skip ci] ([2724d3d](2724d3dd5f))
* **release:** 0.11.0-beta.7 [skip ci] ([f0f7373](f0f73736f7))
* **release:** 0.11.0-beta.8 [skip ci] ([fa4edb4](fa4edb4703))
* **release:** 0.11.0-beta.9 [skip ci] ([d2877d8](d2877d89e5))
 2024-05-14 15:14:12 +00:00
Marco Vinciguerra
2a57940008
Merge pull request #234 from VinciGit00/pre/beta 
Pre/beta
 2024-05-14 17:12:53 +02:00
semantic-release-bot
8727d03384 ci(release): 0.11.0-beta.11 [skip ci] 
## [0.11.0-beta.11](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.11.0-beta.10...v0.11.0-beta.11) (2024-05-14)

### Features

* **omni-search:** added omni search graph and updated docs ([fcb3abb](fcb3abb01d))
* **gpt-4o:** image to text single node test ([90955ca](90955ca52f))
* **omni-scraper:** working OmniScraperGraph with images ([a296927](a296927624))

### Bug Fixes

* **fetch_node:** bug in handling local files ([a6e1813](a6e1813ddd))
 2024-05-14 15:10:47 +00:00
Marco Vinciguerra
932df8d491
Merge pull request #238 from VinciGit00/gpt4-omni 2024-05-14 17:09:23 +02:00
Marco Vinciguerra
d76badd133
Merge pull request #239 from mayurdb/deepScrapeFix 2024-05-14 17:08:58 +02:00
mayurdb
a458ec4b9f Update the prompt for the search_link_node 2024-05-14 16:59:16 +02:00
Marco Perini
a6e1813ddd fix(fetch_node): bug in handling local files 2024-05-14 16:51:10 +02:00
Marco Perini
fcb3abb01d feat(omni-search): added omni search graph and updated docs 2024-05-14 16:26:43 +02:00
Marco Perini
a296927624 feat(omni-scraper): working OmniScraperGraph with images 2024-05-14 13:46:49 +02:00
Marco Perini
90955ca52f feat(gpt-4o): image to text single node test 2024-05-14 11:43:21 +02:00
semantic-release-bot
218b8ede8a ci(release): 0.11.0-beta.10 [skip ci] 
## [0.11.0-beta.10](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.11.0-beta.9...v0.11.0-beta.10) (2024-05-14)

### Features

* add gpt-4o ([52a4a3b](52a4a3b22d)), closes [#232](https://github.com/VinciGit00/Scrapegraph-ai/issues/232)
 2024-05-14 09:29:03 +00:00
Marco Perini
8e46799515
Merge pull request #235 from f-aguzzi/pre/beta 
feat: add gpt-4o to the model dictionary
 2024-05-14 11:27:41 +02:00
Federico Aguzzi
52a4a3b22d feat: add gpt-4o 
closes #232
 2024-05-14 11:11:34 +02:00
semantic-release-bot
d2877d89e5 ci(release): 0.11.0-beta.9 [skip ci] 
## [0.11.0-beta.9](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.11.0-beta.8...v0.11.0-beta.9) (2024-05-14)

### Bug Fixes

* crash asyncio due dependency version ([2563773](2563773447))

### chore

* update models_tokens.py with new model configurations ([d9752b1](d9752b1619))

### Docs

* fixed speechgraphexample ([4bf90f3](4bf90f32a8))
* fixed unused param and install ([cc28d5a](cc28d5a64f))
* **readme:** improve main readme ([ae5655f](ae5655fdde))
* **concurrent:** refactor theme and added benchmarck searchgraph ([ced2bbc](ced2bbcdc9))
* update instructions to use with LocalAI ([198420c](198420c505))
* Update README.md ([772e064](772e064c55))
* updated sponsor logo ([f8d8d71](f8d8d71589))

### CI

* **release:** 0.10.1 [skip ci] ([d359814](d359814c4a))
 2024-05-14 08:37:42 +00:00
VinciGit00
5914fa8c64 Update poetry.lock 2024-05-14 10:36:21 +02:00
VinciGit00
d6f5ca874f Merge branch 'main' into pre/beta 2024-05-14 10:24:29 +02:00
Marco Perini
4fd8a39d5b Merge branch 'pre/beta' of https://github.com/VinciGit00/Scrapegraph-ai into pre/beta 2024-05-14 02:21:57 +02:00
Marco Perini
ced2bbcdc9 docs(concurrent): refactor theme and added benchmarck searchgraph 2024-05-14 02:21:46 +02:00
semantic-release-bot
fa4edb4703 ci(release): 0.11.0-beta.8 [skip ci] 
## [0.11.0-beta.8](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.11.0-beta.7...v0.11.0-beta.8) (2024-05-13)

### Features

* **parallel-exeuction:** add asyncio event loop dispatcher with semaphore for parallel graph instances ([627cbee](627cbeeb20))
* **batchsize:** tested different batch sizes and systems ([a8d5e7d](a8d5e7db05))

### Bug Fixes

* **asyncio:** replaced deepcopy with copy due to serialization problems ([dedc733](dedc733047))
 2024-05-13 21:52:03 +00:00
Marco Perini
df918fa8e6
Merge pull request #231 from arsaboo/models 
chore: update models_tokens.py with new model configurations
 2024-05-13 23:51:45 +02:00
Marco Perini
62a74a5737
Merge pull request #213 from DiTo97/feat/parallel-node-execution 
feat(parallel-exeuction): parallel asyncio event loop dispatcher
 2024-05-13 23:50:48 +02:00
Marco Perini
367dea5cbd
Merge branch 'pre/beta' into feat/parallel-node-execution 2024-05-13 23:50:40 +02:00
Marco Perini
a8d5e7db05 feat(batchsize): tested different batch sizes and systems 2024-05-13 23:49:48 +02:00
Alok Saboo
d9752b1619 chore: update models_tokens.py with new model configurations 2024-05-13 17:22:32 -04:00
VinciGit00
c0d26d61d7 ad bedrocl 2024-05-13 20:17:03 +02:00
Marco Vinciguerra
28ab8da1fd
Merge pull request #229 from JGalego/feat/custom-aws-creds 
feat: Custom AWS credentials
 2024-05-13 20:14:28 +02:00
JGalego
859c5d5ba5 Refactored to include custom AWS client for bedrock; Added missing Anthropic class 2024-05-13 17:57:53 +01:00
Marco Perini
dedc733047 fix(asyncio): replaced deepcopy with copy due to serialization problems 2024-05-13 18:46:34 +02:00
Marco Vinciguerra
f3d44c09ae
Merge pull request #228 from JGalego/fix/bedrock-support 
fix: Amazon Bedrock Support
 2024-05-13 14:57:23 +02:00
semantic-release-bot
f0f73736f7 ci(release): 0.11.0-beta.7 [skip ci] 
## [0.11.0-beta.7](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.11.0-beta.6...v0.11.0-beta.7) (2024-05-13)

### Bug Fixes

* bug for claude ([d0167de](d0167dee71))

### Docs

* **refactor:** changed example ([c7ec114](c7ec114274))
 2024-05-13 11:44:57 +00:00
VinciGit00
d0167dee71 fix: bug for claude 2024-05-13 13:43:42 +02:00
JGalego
39be38f292 Fixed anthropic/bedrock conflict; Removed duplicate class Claude; Updated BedrockEmbeddings class 2024-05-13 12:35:42 +01:00