CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-06-25 21:11:11 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
888 Commits 2 Branches 448 Tags 17 MiB
2e1c79d960
Commit Graph

888 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Marco Vinciguerra
2e1c79d960 Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-05-21 10:41:08 +02:00
Marco Vinciguerra
1ef63217dd add deepseek embeddings 2024-05-21 10:41:01 +02:00
semantic-release-bot
f15fbc0d52 ci(release): 1.3.1 [skip ci] 
## [1.3.1](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.3.0...v1.3.1) (2024-05-21)

### Bug Fixes

* add deepseek embeddings ([659fad7](659fad770a))
 2024-05-21 08:38:36 +00:00
Marco Vinciguerra
659fad770a fix: add deepseek embeddings 2024-05-21 10:36:40 +02:00
Marco Vinciguerra
481106d923 Update README.md 2024-05-21 09:19:21 +02:00
Marco Vinciguerra
6f51f461d2 Update README.md 2024-05-21 09:18:23 +02:00
Marco Vinciguerra
7c4a26fd0d Update README.md 2024-05-21 09:18:05 +02:00
Marco Vinciguerra
75bb389cf7 add stat logo parternship 2024-05-21 09:15:02 +02:00
semantic-release-bot
7fd06a6ac5 ci(release): 1.3.0 [skip ci] 
## [1.3.0](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.2.4...v1.3.0) (2024-05-19)

### Features

* add new model ([8c7afa7](8c7afa7570))
 2024-05-19 20:06:05 +00:00
Marco Vinciguerra
3c3c7577d8 Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-05-19 22:03:45 +02:00
Marco Vinciguerra
8c7afa7570 feat: add new model 2024-05-19 22:03:21 +02:00
Marco Vinciguerra
ae9986a113
Merge pull request #264 from MobinX/patch-1 
Added gemini flash and pro support
 2024-05-19 08:45:25 +02:00
Mobin Chowdhury
7f30da9930
Update models_tokens.py 2024-05-18 23:32:27 +06:00
Marco Vinciguerra
a0125b7e90 add orca mini integration 2024-05-18 09:31:40 +02:00
semantic-release-bot
746e7bd343 ci(release): 1.2.4 [skip ci] 
## [1.2.4](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.2.3...v1.2.4) (2024-05-17)

### Bug Fixes

* **deepcopy:** switch whether we have obj in the config ([d4d913c](d4d913c8a3))
 2024-05-17 08:38:38 +00:00
Marco Perini
d4d913c8a3 fix(deepcopy): switch whether we have obj in the config 2024-05-17 10:36:39 +02:00
Marco Vinciguerra
b4cfcb100d
Update README.md 2024-05-16 16:32:14 +02:00
VinciGit00
52fe108e48 Update README.md 2024-05-15 21:10:43 +02:00
VinciGit00
e35c5ac33c Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-05-15 19:27:45 +02:00
VinciGit00
a718145f46 rye installation 2024-05-15 19:27:42 +02:00
semantic-release-bot
5a557f9295 ci(release): 1.2.3 [skip ci] 
## [1.2.3](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.2.2...v1.2.3) (2024-05-15)

### Bug Fixes

* **deepcopy:** reaplced to shallow copy ([999c930](999c930f42))
 2024-05-15 15:36:04 +00:00
VinciGit00
753f8cf194 add lock 2024-05-15 17:34:06 +02:00
VinciGit00
5e42e2e2dd Update .gitignore 2024-05-15 16:56:54 +02:00
Marco Perini
0b7ef252df Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-05-15 16:00:50 +02:00
Marco Perini
999c930f42 fix(deepcopy): reaplced to shallow copy 2024-05-15 16:00:42 +02:00
semantic-release-bot
3531a68f5b ci(release): 1.2.2 [skip ci] 
## [1.2.2](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.2.1...v1.2.2) (2024-05-15)

### Bug Fixes

* come back to the old version ([cc5adef](cc5adefd29))
 2024-05-15 13:57:23 +00:00
VinciGit00
3ee3b82bc7 Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-05-15 15:54:38 +02:00
VinciGit00
cc5adefd29 fix: come back to the old version 2024-05-15 15:54:00 +02:00
semantic-release-bot
7614c811a9 ci(release): 1.2.1 [skip ci] 
## [1.2.1](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.2.0...v1.2.1) (2024-05-15)

### Bug Fixes

* removed unused ([5587a64](5587a64d23))
 2024-05-15 13:53:31 +00:00
VinciGit00
5587a64d23 fix: removed unused 2024-05-15 15:51:27 +02:00
VinciGit00
7ced0d0564 remove asdt 2024-05-15 15:49:09 +02:00
semantic-release-bot
d5045abf17 ci(release): 1.2.0 [skip ci] 
## [1.2.0](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.1.0...v1.2.0) (2024-05-15)

### Features

* add finalize_node() ([6e7283e](6e7283ed8f))
 2024-05-15 13:22:20 +00:00
Marco Vinciguerra
cffcf80a75
Merge branch '88-blockscraper-implementation' into main 2024-05-15 15:20:12 +02:00
semantic-release-bot
51200d6918 ci(release): 1.1.0 [skip ci] 
## [1.1.0](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.0.1...v1.1.0) (2024-05-15)

### Features

* add turboscraper (alfa) ([51aa109](51aa109e42))
* new search_graph ([67d5fbf](67d5fbf816))

### Docs

* **rye:** replaced poetry with rye ([efb781f](efb781f950))
 2024-05-15 13:18:52 +00:00
Marco Vinciguerra
22cd9e3605
Merge branch 'search_link_context' into main 2024-05-15 15:16:57 +02:00
Marco Perini
efb781f950
docs(rye): replaced poetry with rye 2024-05-15 15:15:17 +02:00
VinciGit00
694d3ab0f3 Merge branch 'main' of https://github.com/VinciGit00/Scrapegraph-ai 2024-05-15 15:10:53 +02:00
VinciGit00
7ccd51a879 add rye update script bash 2024-05-15 15:10:51 +02:00
semantic-release-bot
a81d2b7a80 ci(release): 1.0.1 [skip ci] 
## [1.0.1](https://github.com/VinciGit00/Scrapegraph-ai/compare/v1.0.0...v1.0.1) (2024-05-15)

### Bug Fixes

* **searchgraph:** used shallow copy to serialize obj ([096b665](096b665c01))
 2024-05-15 13:07:36 +00:00
Marco Perini
096b665c01 fix(searchgraph): used shallow copy to serialize obj 2024-05-15 15:05:47 +02:00
Marco Vinciguerra
24d56af97c
Update pyproject.toml 2024-05-15 14:10:56 +02:00
semantic-release-bot
c0b6f0260f ci(release): 1.0.0 [skip ci] 
## [1.0.0](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.11.1...v1.0.0) (2024-05-15)

### ⚠ BREAKING CHANGES

* **package manager:** move from poetry to rye

### chore

* **package manager:** move from poetry to rye ([8fc2510](8fc2510b37)), closes [#198](https://github.com/VinciGit00/Scrapegraph-ai/issues/198)

### Docs

* **main-readme:** fixed some typos ([78d1940](78d1940235))
 2024-05-15 11:45:54 +00:00
Marco Vinciguerra
672bd29b65
Merge pull request #244 from f-aguzzi/main 
chore(package manager)!: move from poetry to rye
 2024-05-15 13:43:58 +02:00
Federico Aguzzi
8fc2510b37 chore(package manager)!: move from poetry to rye 
closes #198
 2024-05-15 13:42:58 +02:00
Marco Perini
78d1940235
docs(main-readme): fixed some typos 2024-05-15 12:09:14 +02:00
semantic-release-bot
6effe25900 ci(release): 0.11.1 [skip ci] 
## [0.11.1](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.11.0...v0.11.1) (2024-05-14)

### Bug Fixes

* **docs:** requirements-dev ([b0a67ba](b0a67ba387))
 2024-05-14 16:14:46 +00:00
Marco Perini
b0a67ba387 fix(docs): requirements-dev 2024-05-14 18:13:24 +02:00
semantic-release-bot
c55a3b1162 ci(release): 0.11.0 [skip ci] 
## [0.11.0](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.10.1...v0.11.0) (2024-05-14)

### Features

* **parallel-exeuction:** add asyncio event loop dispatcher with semaphore for parallel graph instances ([627cbee](627cbeeb20))
* **webdriver-backend:** add dynamic import scripts from module and file ([db2234b](db2234bf5d))
* add gpt-4o ([52a4a3b](52a4a3b22d)), closes [#232](https://github.com/VinciGit00/Scrapegraph-ai/issues/232)
* add new prompt info ([e2350ed](e2350eda62))
* **proxy-rotation:** add parse (IP address) or search (from broker) functionality for proxy rotation ([2170131](217013181d))
* add support for deepseek-chat ([156b67b](156b67b91e)), closes [#222](https://github.com/VinciGit00/Scrapegraph-ai/issues/222)
* Add support for passing pdf path as source ([f10f3b1](f10f3b1438))
* **omni-search:** added omni search graph and updated docs ([fcb3abb](fcb3abb01d))
* added proxy rotation ([0c36a7e](0c36a7ec1f))
* **safe-web-driver:** enchanced the original `AsyncChromiumLoader` web driver with proxy protection and flexible kwargs and backend ([768719c](768719cce8))
* **gpt-4o:** image to text single node test ([90955ca](90955ca52f))
* revert fetch_node ([864aa91](864aa91326))
* **batchsize:** tested different batch sizes and systems ([a8d5e7d](a8d5e7db05))
* update info ([4ed0fb8](4ed0fb89c3))
* **omni-scraper:** working OmniScraperGraph with images ([a296927](a296927624))

### Bug Fixes

* **pytest:** add dependency for mocking testing functions ([2f4fd45](2f4fd45700))
* add json integration ([0ab31c3](0ab31c3fdb))
* Augment the information getting fetched from a webpage ([f8ce3d5](f8ce3d5916))
* bug for claude ([d0167de](d0167dee71))
* **fetch_node:** bug in handling local files ([a6e1813](a6e1813ddd))
* **chromium-loader:** ensure it subclasses langchain's base loader ([b54d984](b54d984c13))
* fixed bugs for csv and xml ([324e977](324e977b85))
* limit python version to < 3.12 ([a37fbbc](a37fbbcbcf))
* **proxy-rotation:** removed duplicated arg and passed the loader_kwarhs correctly to the node ([1e9a564](1e9a564616))
* **fetch-node:** removed isSoup from default ([0c15947](0c1594737f))
* **proxy-rotation:** removed max_shape duplicate ([5d6d996](5d6d996e8f))
* **asyncio:** replaced deepcopy with copy due to serialization problems ([dedc733](dedc733047))

### chore

* update models_tokens.py with new model configurations ([d9752b1](d9752b1619))

### Docs

* add diagram showing general structure/flow of the library ([13ae918](13ae9180ac))
* **refactor:** added proxy-rotation usage and refactor readthedocs ([e256b75](e256b758b2))
* **refactor:** changed example ([c7ec114](c7ec114274))
* **concurrent:** refactor theme and added benchmarck searchgraph ([ced2bbc](ced2bbcdc9))
* update overview diagram with more models ([b441b30](b441b30a5c))

### CI

* **release:** 0.10.0-beta.3 [skip ci] ([ad32298](ad32298e70))
* **release:** 0.10.0-beta.4 [skip ci] ([548bff9](548bff9d77))
* **release:** 0.10.0-beta.5 [skip ci] ([28c9dce](28c9dce7cb))
* **release:** 0.10.0-beta.6 [skip ci] ([460d292](460d292af2))
* **release:** 0.11.0-beta.1 [skip ci] ([63c0dd9](63c0dd9372))
* **release:** 0.11.0-beta.10 [skip ci] ([218b8ed](218b8ede8a)), closes [#232](https://github.com/VinciGit00/Scrapegraph-ai/issues/232)
* **release:** 0.11.0-beta.11 [skip ci] ([8727d03](8727d03384))
* **release:** 0.11.0-beta.2 [skip ci] ([7ae50c0](7ae50c035e))
* **release:** 0.11.0-beta.3 [skip ci] ([106fb12](106fb12531)), closes [#222](https://github.com/VinciGit00/Scrapegraph-ai/issues/222)
* **release:** 0.11.0-beta.4 [skip ci] ([4ccddda](4ccddda5eb))
* **release:** 0.11.0-beta.5 [skip ci] ([353382b](353382b4d3))
* **release:** 0.11.0-beta.6 [skip ci] ([2724d3d](2724d3dd5f))
* **release:** 0.11.0-beta.7 [skip ci] ([f0f7373](f0f73736f7))
* **release:** 0.11.0-beta.8 [skip ci] ([fa4edb4](fa4edb4703))
* **release:** 0.11.0-beta.9 [skip ci] ([d2877d8](d2877d89e5))
 2024-05-14 15:14:12 +00:00
Marco Vinciguerra
2a57940008
Merge pull request #234 from VinciGit00/pre/beta 
Pre/beta
 2024-05-14 17:12:53 +02:00
semantic-release-bot
8727d03384 ci(release): 0.11.0-beta.11 [skip ci] 
## [0.11.0-beta.11](https://github.com/VinciGit00/Scrapegraph-ai/compare/v0.11.0-beta.10...v0.11.0-beta.11) (2024-05-14)

### Features

* **omni-search:** added omni search graph and updated docs ([fcb3abb](fcb3abb01d))
* **gpt-4o:** image to text single node test ([90955ca](90955ca52f))
* **omni-scraper:** working OmniScraperGraph with images ([a296927](a296927624))

### Bug Fixes

* **fetch_node:** bug in handling local files ([a6e1813](a6e1813ddd))
 2024-05-14 15:10:47 +00:00