CodeCow/Scrapegraph-ai
1
0
Fork 0
mirror of https://github.com/VinciGit00/Scrapegraph-ai.git synced 2026-07-09 21:19:20 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,293 Commits 2 Branches 448 Tags 17 MiB
5faac9fb46
Commit Graph

2293 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Marco Vinciguerra
aaa011cc89
Merge pull request #754 from ekinsenler/cond_node_refactor 
feat: add conditional node to the smart_scraper_graph
 2024-10-16 17:53:10 +02:00
ekinsenler
eaa83edc04 update project requirement and add example 2024-10-16 15:21:23 +03:00
Marco Vinciguerra
488821abfa Update smart_scraper_openai.py 2024-10-16 14:07:01 +02:00
roryhaung
464b8b04ea test: Add smart_scraper_multi_parse_merge_first_graph test 2024-10-16 20:05:36 +08:00
roryhaung
cdb3c1100e test: Add scrape_graph test 2024-10-16 20:05:03 +08:00
roryhaung
3e3e1b2f3a feat: Implement SmartScraperMultiParseMergeFirstGraph class that scrapes a list of URLs and merge the content first and finally generates answers to a given prompt. 
(Different from the SmartScraperMultiGraph is that in this case the content is merged before to be processed by the llm.)
 2024-10-16 19:38:53 +08:00
roryhaung
612c644623 feat: implement ScrapeGraph class for only web scraping automation 2024-10-16 18:37:50 +08:00
Lorenzo Padoan
f20c60acd0
Update FUNDING.yml 2024-10-16 11:06:15 +02:00
Marco Vinciguerra
082b8af050 Merge branch 'main' of https://github.com/ScrapeGraphAI/Scrapegraph-ai
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Details
/ build (push) Has been cancelled
Details
Release / Build (push) Has been cancelled
Details
Release / Release (push) Has been cancelled
Details
2024-10-15 15:28:39 +02:00
Lorenzo Padoan
c7e7259e08
Update FUNDING.yml 2024-10-15 15:22:18 +02:00
Lorenzo Padoan
82dbb9ca92
Update FUNDING.yml 2024-10-15 15:19:12 +02:00
Lorenzo Padoan
560934dbb3
Update FUNDING.yml 2024-10-15 15:06:05 +02:00
Lorenzo Padoan
77deb858e4
Create FUNDING.yml 2024-10-15 15:03:45 +02:00
ekinsenler
038d2ef916 refactor cond node structure to fit with the new implementation 2024-10-15 15:23:05 +03:00
ekinsenler
cacd9cde00 feat: add conditional node structure to the smart_scraper_graph and implemented a structured way to check condition 2024-10-15 14:23:02 +03:00
Marco Vinciguerra
a84429114e Merge branch 'main' of https://github.com/ScrapeGraphAI/Scrapegraph-ai 2024-10-15 11:21:05 +02:00
Marco Vinciguerra
db4b7cd3d1 refactoring of the code 2024-10-15 11:21:04 +02:00
semantic-release-bot
e0fc457d1a ci(release): 1.26.6-beta.1 [skip ci] 
## [1.26.6-beta.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.26.5...v1.26.6-beta.1) (2024-10-14)

### Bug Fixes

* remove variable "max_result" not being used in the code ([e76a68a](e76a68a782))
 2024-10-14 12:14:59 +00:00
Marco Vinciguerra
39c7352cea
Merge pull request #751 from shenghongtw/pre/beta 
Since the variable "max_result" is not used in the code, it has been removed to avoid causing confusion for users. (The example code was tested, and the presence or absence of max_result does not affect the results.)
 2024-10-14 14:13:28 +02:00
Marco Vinciguerra
deb7c2d1c7
Merge pull request #750 from ScrapeGraphAI/main 2024-10-14 07:39:38 +02:00
roryhaung
e76a68a782 fix: remove variable "max_result" not being used in the code 2024-10-13 23:04:27 +08:00
semantic-release-bot
931b975d79 ci(release): 1.26.5 [skip ci] 
## [1.26.5](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.26.4...v1.26.5) (2024-10-13)

### Bug Fixes

* async invocation ([c2179ab](c2179abc60))
 2024-10-13 09:32:08 +00:00
Marco Vinciguerra
c2179abc60 fix: async invocation 2024-10-13 11:30:39 +02:00
semantic-release-bot
0e4ff09a10 ci(release): 1.26.4 [skip ci] 
## [1.26.4](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.26.3...v1.26.4) (2024-10-13)

### Bug Fixes

* csv_node ([b208ef7](b208ef792c))
 2024-10-13 08:34:05 +00:00
Marco Vinciguerra
6ba772302a Merge branch 'main' of https://github.com/ScrapeGraphAI/Scrapegraph-ai 2024-10-13 10:31:14 +02:00
Marco Vinciguerra
b208ef792c fix: csv_node 2024-10-13 10:31:12 +02:00
semantic-release-bot
c833d7969a ci(release): 1.26.3 [skip ci] 
## [1.26.3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.26.2...v1.26.3) (2024-10-13)

### Bug Fixes

* generate answer node ([431b209](431b2093be))
 2024-10-13 08:27:00 +00:00
Marco Vinciguerra
9fb0ff7a87 Merge branch 'main' of https://github.com/ScrapeGraphAI/Scrapegraph-ai 2024-10-13 10:25:39 +02:00
Marco Vinciguerra
431b2093be fix: generate answer node 2024-10-13 10:25:36 +02:00
semantic-release-bot
673241a9ca ci(release): 1.26.2 [skip ci] 
## [1.26.2](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.26.1...v1.26.2) (2024-10-13)

### Bug Fixes

* add new dipendency ([35c44e4](35c44e4d2c))
 2024-10-13 08:18:57 +00:00
Marco Vinciguerra
54b37bbda5 Merge branch 'main' of https://github.com/ScrapeGraphAI/Scrapegraph-ai 2024-10-13 10:17:05 +02:00
Marco Vinciguerra
35c44e4d2c fix: add new dipendency 2024-10-13 10:17:03 +02:00
semantic-release-bot
cd36c2a0f3 ci(release): 1.26.1 [skip ci] 
## [1.26.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.26.0...v1.26.1) (2024-10-13)

### Bug Fixes

* async tim ([7b07368](7b073686ef))
* typo ([9c62f24](9c62f24e73))
* typo ([c9d6ef5](c9d6ef5915))
 2024-10-13 08:12:23 +00:00
Marco Vinciguerra
9c62f24e73 fix: typo 2024-10-13 10:10:56 +02:00
Marco Vinciguerra
c9d6ef5915 fix: typo 2024-10-13 10:09:04 +02:00
Marco Vinciguerra
c8f821c45d Merge branch 'main' of https://github.com/ScrapeGraphAI/Scrapegraph-ai 2024-10-13 10:07:54 +02:00
Marco Vinciguerra
7b073686ef fix: async tim 2024-10-13 10:07:40 +02:00
semantic-release-bot
70a883fa32 ci(release): 1.26.0 [skip ci] 
## [1.26.0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.25.2...v1.26.0) (2024-10-13)

### Features

* add deep scraper implementation ([4b371f4](4b371f4d94))
* add google proxy support ([a986523](a986523884))
* add html_mode to smart_scraper ([bdcffd6](bdcffd6360))
* add reasoning integration ([b2822f6](b2822f620a))
* async invocation ([257f393](257f393761))
* conditional_node ([f837dc1](f837dc16ce))
* finished basic version of deep scraper ([85cb957](85cb957297))
* prompt refactoring ([5a2f6d9](5a2f6d9a77))
* refactoring fetch_node ([39a029e](39a029ed9a))
* refactoring of mdscraper ([3b7b701](3b7b701a89))
* refactoring of research web ([26f89d8](26f89d895d))
* refactoring of the conditional node ([420c71b](420c71ba2c))
* undected_chromedriver support ([80ece21](80ece2179a))
* update chromium loader ([4f816f3](4f816f3b04))

### Bug Fixes

* bugs ([026a70b](026a70bd3a))
* import error ([37b6ba0](37b6ba08ae))
* integration with html_mode ([f87ffa1](f87ffa1d8d))
* nodes prompt ([8753537](8753537ecd))
* pyproject.toml ([3b27c5e](3b27c5e88c))
* refactoring prompts ([c655642](c655642577))
* removed pdf_scraper graph and created document scraper ([a57da96](a57da96175))
* search_on_web paremter ([7f03ec1](7f03ec15de))
* typo ([e285127](e28512720c))

### Perf

* Proxy integration in googlesearch ([e828c70](e828c7010a))

### CI

* **release:** 1.22.0-beta.4 [skip ci] ([4330179](4330179cb6))
* **release:** 1.22.0-beta.5 [skip ci] ([6d8f543](6d8f5435d1))
* **release:** 1.22.0-beta.6 [skip ci] ([39f7815](39f78154a6))
* **release:** 1.26.0-beta.1 [skip ci] ([ac31d7f](ac31d7f710))
* **release:** 1.26.0-beta.10 [skip ci] ([0c7ebe2](0c7ebe28ac))
* **release:** 1.26.0-beta.11 [skip ci] ([6d8828a](6d8828aa62))
* **release:** 1.26.0-beta.12 [skip ci] ([44d10aa](44d10aa1c0))
* **release:** 1.26.0-beta.13 [skip ci] ([12f2b99](12f2b9946b))
* **release:** 1.26.0-beta.14 [skip ci] ([eb25725](eb257259f8))
* **release:** 1.26.0-beta.15 [skip ci] ([528a974](528a9746fe))
* **release:** 1.26.0-beta.16 [skip ci] ([04bd2a8](04bd2a87fb))
* **release:** 1.26.0-beta.17 [skip ci] ([f17089c](f17089c123))
* **release:** 1.26.0-beta.2 [skip ci] ([5cedeb8](5cedeb8486))
* **release:** 1.26.0-beta.3 [skip ci] ([4f65be4](4f65be44b5))
* **release:** 1.26.0-beta.4 [skip ci] ([84d7937](84d7937472))
* **release:** 1.26.0-beta.5 [skip ci] ([ea9ed1a](ea9ed1a981))
* **release:** 1.26.0-beta.6 [skip ci] ([4cd21f5](4cd21f500d))
* **release:** 1.26.0-beta.7 [skip ci] ([482f060](482f060c9a))
* **release:** 1.26.0-beta.8 [skip ci] ([38b795e](38b795e48a))
* **release:** 1.26.0-beta.9 [skip ci] ([4dc0699](4dc0699483))
 2024-10-13 07:34:53 +00:00
Marco Vinciguerra
5799124883
Merge pull request #747 from ScrapeGraphAI/pre/beta 2024-10-13 09:33:28 +02:00
semantic-release-bot
f17089c123 ci(release): 1.26.0-beta.17 [skip ci] 
## [1.26.0-beta.17](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.26.0-beta.16...v1.26.0-beta.17) (2024-10-12)

### Features

* async invocation ([257f393](257f393761))
* refactoring of mdscraper ([3b7b701](3b7b701a89))

### Bug Fixes

* bugs ([026a70b](026a70bd3a))
* search_on_web paremter ([7f03ec1](7f03ec15de))
 2024-10-12 07:59:48 +00:00
Marco Vinciguerra
e4421ffdcd Merge branch '731-fetchnode-object-has-no-attribute-update_state' into pre/beta 2024-10-12 09:58:20 +02:00
Marco Vinciguerra
026a70bd3a fix: bugs 2024-10-12 09:57:59 +02:00
Marco Vinciguerra
47645d92ce removed unused files 2024-10-12 09:41:02 +02:00
Marco Vinciguerra
5211cbe5ae
Merge pull request #744 from aziz-ullah-khan/pre/beta 2024-10-11 14:56:05 +02:00
aziz-ullah-khan
7f03ec15de Fix: search_on_web paremter 2024-10-11 16:58:37 +05:00
Marco Vinciguerra
694032243f Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegraph-ai into pre/beta 2024-10-11 12:47:01 +02:00
Marco Vinciguerra
1fb618cba4 removed unused node 2024-10-11 12:46:39 +02:00
semantic-release-bot
04bd2a87fb ci(release): 1.26.0-beta.16 [skip ci] 
## [1.26.0-beta.16](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.26.0-beta.15...v1.26.0-beta.16) (2024-10-11)

### Features

* add google proxy support ([a986523](a986523884))

### Bug Fixes

* typo ([e285127](e28512720c))

### Perf

* Proxy integration in googlesearch ([e828c70](e828c7010a))
 2024-10-11 10:14:12 +00:00
Marco Vinciguerra
e28512720c fix: typo 2024-10-11 12:12:41 +02:00
Marco Vinciguerra
630e6405c8 Merge branch 'pre/beta' of https://github.com/ScrapeGraphAI/Scrapegraph-ai into pre/beta 2024-10-11 12:08:51 +02:00