JGalego
859c5d5ba5
Refactored to include custom AWS client for bedrock; Added missing Anthropic class
2024-05-13 17:57:53 +01:00
Marco Vinciguerra
f3d44c09ae
Merge pull request #228 from JGalego/fix/bedrock-support
...
fix: Amazon Bedrock Support
2024-05-13 14:57:23 +02:00
VinciGit00
d0167dee71
fix: bug for claude
2024-05-13 13:43:42 +02:00
JGalego
39be38f292
Fixed anthropic/bedrock conflict; Removed duplicate class Claude; Updated BedrockEmbeddings class
2024-05-13 12:35:42 +01:00
Marco Perini
0c1594737f
fix(fetch-node): removed isSoup from default
2024-05-13 12:09:55 +02:00
Marco Perini
7e8acd8e6a
Merge branch 'pre/beta' into fix/fetch-node-proxybroker
2024-05-13 11:17:37 +02:00
Marco Perini
5d6d996e8f
fix(proxy-rotation): removed max_shape duplicate
2024-05-13 07:26:43 +02:00
Marco Perini
1e9a564616
fix(proxy-rotation): removed duplicated arg and passed the loader_kwarhs correctly to the node
2024-05-12 18:39:03 +02:00
VinciGit00
e2350eda62
feat: add new prompt info
2024-05-12 11:14:30 +02:00
Federico Aguzzi
156b67b91e
feat: add support for deepseek-chat
...
closes #222
2024-05-12 01:00:11 +02:00
mayurdb
dd29c16cbe
Merge branch 'deepScrape' of github.com:mayurdb/Scrapegraph-ai into deepScrape
2024-05-11 16:58:54 +05:30
mayurdb
9a67a26cd3
Update documentation
2024-05-11 16:57:22 +05:30
Mayur Bhosale
ea3b545573
Merge branch 'pre/beta' into deepScrape
2024-05-11 16:54:40 +05:30
mayurdb
8f1fbe7e10
minor changes
2024-05-11 16:52:36 +05:30
mayurdb
df271b6451
Add search link node that can find out relevant links in the webpage
2024-05-11 16:39:55 +05:30
Federico Minutoli
b54d984c13
fix(chromium-loader): ensure it subclasses langchain's base loader
2024-05-11 11:44:56 +02:00
VinciGit00
dc91719365
Update cleanup_html.py
2024-05-11 10:49:16 +02:00
Marco Vinciguerra
b752499fab
Merge pull request #217 from mayurdb/fetchLinkFix
...
Fetch links in the page while parsing html
2024-05-11 09:42:40 +02:00
mayurdb
300fd5d253
Fetch links in the page while parsing html
2024-05-11 09:46:51 +05:30
Eric Page
0683e78e78
Merge branch 'pre/beta' into fix-GenerateScraperGraph
2024-05-11 01:59:28 +02:00
Eric Page
24c3b0580c
Removed nonfunctional RAG node from ScriptCreatorGraph
2024-05-11 01:38:40 +02:00
Eric Page
aac51ba290
Removed dead code, allows GenerateScraperNode to generate scraper with
...
one chunk of context
2024-05-11 01:34:51 +02:00
Eric Page
40884747c7
Added parse_html option in parse_node
2024-05-11 00:32:01 +02:00
Federico Minutoli
fc2aa3ac1c
Merge branch 'pre/beta' of https://github.com/DiTo97/Scrapegraph-ai into fix/fetch-node-proxybroker
2024-05-10 21:20:40 +02:00
Federico Minutoli
768719cce8
feat(safe-web-driver): enchanced the original AsyncChromiumLoader web driver with proxy protection and flexible kwargs and backend
...
the original class prevents passing kwargs down to the playwright backend, making some config unfeasible, including passing a proxy server to the web driver.
the new class has backward compatibility with the original, but 1) allows any kwarg to be passed down to the web driver, 2) allows specifying the web driver backend (only playwright is supported for now) in case more (e.g., selenium) will be supported in the future and 3) automatically fetches a suitable proxy if one is not passed already
2024-05-10 21:13:38 +02:00
Federico Minutoli
217013181d
feat(proxy-rotation): add parse (IP address) or search (from broker) functionality for proxy rotation
...
the broker has been made fully configurable for anonymity level, admissible locations, scheme and max shape not to waste resources, unlike the original `free-proxy` package.
other options have been explored (e.g., `proxybroker`, `proxybroker2`) due to their built-in proxy server and rotation capabilities, but the former is no longer maintained, and the latter has issue with any python version outside of python 3.9
2024-05-10 21:09:48 +02:00
Federico Minutoli
db2234bf5d
feat(webdriver-backend): add dynamic import scripts from module and file
2024-05-10 21:06:05 +02:00
Marco Perini
864aa91326
feat: revert fetch_node
2024-05-10 15:11:54 +02:00
Marco Vinciguerra
99adc9799f
Merge branch 'pre/beta' into fetchNodeFix
2024-05-10 11:13:54 +02:00
mayurdb
f8ce3d5916
fix: Augment the information getting fetched from a webpage
2024-05-10 13:28:53 +05:30
VinciGit00
0ab31c3fdb
fix: add json integration
2024-05-09 21:07:07 +02:00
VinciGit00
324e977b85
fix: fixed bugs for csv and xml
2024-05-09 20:46:46 +02:00
Shubham Kamboj
f10f3b1438
feat: Add support for passing pdf path as source
2024-05-09 21:55:05 +05:30
VinciGit00
94156755d1
Update abstract_graph.py
2024-05-09 12:29:39 +02:00
VinciGit00
403979337c
Update abstract_graph.py
2024-05-09 10:46:31 +02:00
VinciGit00
8272d736a6
add tokenizatio for mxbai-embed-large
2024-05-08 21:50:42 +02:00
VinciGit00
4ed0fb89c3
feat: update info
2024-05-08 21:25:03 +02:00
VinciGit00
e7d39a5daf
fixed gemini embeddings
2024-05-08 21:24:17 +02:00
Marco Perini
6b71ec1d2b
fix(examples): local, mixed models and fixed SearchGraph embeddings problem
2024-05-08 15:36:26 +02:00
Marco Perini
186c0d035d
fix(examples): openai std examples
2024-05-08 14:56:44 +02:00
VinciGit00
2258fe5ac0
add new search graph examples
2024-05-06 22:31:48 +02:00
VinciGit00
5a67bca0db
Merge branch 'pre/beta' into pr/161
2024-05-06 14:50:04 +02:00
VinciGit00
ac6d2005bb
Merge branch 'pre/beta' of https://github.com/VinciGit00/Scrapegraph-ai into pre/beta
2024-05-06 14:46:25 +02:00
VinciGit00
cbd77dfdb0
removed claude
2024-05-06 14:46:24 +02:00
Marco Vinciguerra
97c3fffaf3
Merge pull request #162 from f-aguzzi/patch-1
...
feat: add new hugging_face models
2024-05-06 14:43:29 +02:00
Marco Vinciguerra
f6442ccca3
Merge pull request #157 from VinciGit00/llava_integration
2024-05-06 14:34:32 +02:00
Federico Aguzzi
d5547a450c
feat: add new hugging_face models
2024-05-06 14:28:58 +02:00
Cem Uzunoglu
2ac9e16dd9
Fixed accidental reformatting.
2024-05-06 15:09:56 +03:00
Cem Uzunoglu
e264e92e72
Added support for Claude 3 models from Anthropic
2024-05-06 14:34:35 +03:00
VinciGit00
8c7c3e3ced
Merge branch 'pre/beta' of https://github.com/VinciGit00/Scrapegraph-ai into pre/beta
2024-05-06 13:29:32 +02:00
