mirror of
https://github.com/VinciGit00/Scrapegraph-ai.git
synced 2026-06-23 21:00:30 +08:00
add examples + documentation in the readme
This commit is contained in:
parent
1adc18a551
commit
f311c3e005
@ -3,6 +3,7 @@
|
||||
Thank you for your interest in contributing to **AmazScraper**! We welcome contributions from the community to help improve and grow the project. This document outlines the guidelines and steps for contributing.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Getting Started](#getting-started)
|
||||
- [Contributing Guidelines](#contributing-guidelines)
|
||||
- [Code Style](#code-style)
|
||||
@ -11,6 +12,7 @@ Thank you for your interest in contributing to **AmazScraper**! We welcome contr
|
||||
- [License](#license)
|
||||
|
||||
## Getting Started
|
||||
|
||||
To get started with contributing, follow these steps:
|
||||
|
||||
1. Fork the repository on GitHub.
|
||||
@ -23,6 +25,7 @@ To get started with contributing, follow these steps:
|
||||
8. Submit a pull request to the main repository.
|
||||
|
||||
## Contributing Guidelines
|
||||
|
||||
Please adhere to the following guidelines when contributing to AmazScraper:
|
||||
|
||||
- Follow the code style and formatting guidelines specified in the [Code Style](#code-style) section.
|
||||
@ -31,20 +34,17 @@ Please adhere to the following guidelines when contributing to AmazScraper:
|
||||
- Be respectful and considerate towards other contributors and maintainers.
|
||||
|
||||
## Code Style
|
||||
|
||||
Please make sure to format your code accordingly before submitting a pull request.
|
||||
|
||||
### Python
|
||||
|
||||
- [Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008/)
|
||||
- [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html)
|
||||
- [The Hitchhiker's Guide to Python](https://docs.python-guide.org/writing/style/)
|
||||
|
||||
### Arduino
|
||||
- [Arduino Style Guide for Writing Content](https://docs.arduino.cc/learn/contributions/arduino-writing-style-guide)
|
||||
- [Arduino Style Guide for Creating Libraries](https://docs.arduino.cc/learn/contributions/arduino-library-style-guide)
|
||||
|
||||
### C++
|
||||
- [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html)
|
||||
|
||||
## Submitting a Pull Request
|
||||
|
||||
To submit your changes for review, please follow these steps:
|
||||
|
||||
1. Ensure that your changes are pushed to your forked repository.
|
||||
@ -56,9 +56,11 @@ To submit your changes for review, please follow these steps:
|
||||
7. Once your pull request is approved, it will be merged into the main repository.
|
||||
|
||||
## Reporting Issues
|
||||
|
||||
If you encounter any issues or have suggestions for improvements, please open an issue on the GitHub repository. Provide a clear and detailed description of the problem or suggestion, along with any relevant information or steps to reproduce the issue.
|
||||
|
||||
## License
|
||||
|
||||
AmazScraper is licensed under the **Apache License 2.0**. See the [LICENSE](LICENSE) file for more information.
|
||||
By contributing to this project, you agree to license your contributions under the same license.
|
||||
|
||||
|
||||
56
README.md
56
README.md
@ -16,28 +16,27 @@ Follow the following steps:
|
||||
1. ```bash
|
||||
git clone https://github.com/VinciGit00/AmazScraper.git
|
||||
```
|
||||
2. (Optional)
|
||||
```bash
|
||||
python -m venv venv
|
||||
source ./venv/bin/activate
|
||||
```
|
||||
4. ```bash
|
||||
2. (Optional)
|
||||
```bash
|
||||
python -m venv venv
|
||||
source ./venv/bin/activate
|
||||
```
|
||||
3. ```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
5. Go to [https://openai.com](https://openai.com/) and login
|
||||
6. Now you can access to [https://platform.openai.com/docs/overview](https://platform.openai.com/docs/overview)
|
||||
7. Create a new API key and copy it
|
||||
|
||||
4. Go to [https://openai.com](https://openai.com/) and login
|
||||
5. Now you can access to [https://platform.openai.com/docs/overview](https://platform.openai.com/docs/overview)
|
||||
6. Create a new API key and copy it
|
||||
|
||||
<img src="docs/generate_api_key/step_1.png" alt="Step 1 Screenshot" width="60%"/>
|
||||
|
||||
|
||||
<img src="docs/generate_api_key/step_2.png" alt="Step 2 Screenshot" width="60%"/>
|
||||
|
||||
|
||||
<img src="docs/generate_api_key/step_3.png" alt="Step 3 Screenshot" width="60%"/>
|
||||
|
||||
|
||||
<img src="docs/generate_api_key/step_4.png" alt="Step 4 Screenshot" width="60%"/>
|
||||
|
||||
|
||||
7. Create a .env file inside the main and paste the API key
|
||||
7. Create a .env file inside the main and paste the API key
|
||||
|
||||
```config
|
||||
API_KEY="your openai.com api key"
|
||||
@ -52,15 +51,15 @@ API_KEY="your openai.com api key"
|
||||
```bash
|
||||
python -m AmazScraper.examples.html_scraping
|
||||
```
|
||||
|
||||
|
||||
# Practical use
|
||||
|
||||
## Using AmazScraper as a library
|
||||
|
||||
```python
|
||||
from AmazScraper.classes.class_generator import Generator
|
||||
from classes.class_generator import Generator
|
||||
|
||||
from AmazScraper.utils.getter import get_function, scraper
|
||||
from utils.getter import get_function, scraper
|
||||
|
||||
values = [
|
||||
{
|
||||
@ -75,13 +74,15 @@ if __name__ == "__main__":
|
||||
generator_instance = Generator(values, 0, "gpt-3.5-turbo")
|
||||
|
||||
res = generator_instance.invocation(scraper("https://www.mockupworld.co", 4197))
|
||||
|
||||
print(res)
|
||||
```
|
||||
|
||||
### Case 2: Passing your own HTML code
|
||||
|
||||
```python
|
||||
import sys
|
||||
from AmazScraper.classes.class_generator import Generator
|
||||
from classes.class_generator import Generator
|
||||
|
||||
values = [
|
||||
{
|
||||
@ -133,11 +134,26 @@ if __name__ == "__main__":
|
||||
|
||||
generator_instance = Generator(values, 0, "gpt-3.5-turbo")
|
||||
|
||||
generator_instance.invocation(query_info)
|
||||
res = generator_instance.invocation(query_info)
|
||||
print(res)
|
||||
```
|
||||
|
||||
Note: all the model are avaiable at the following link: [https://platform.openai.com/docs/models](https://platform.openai.com/docs/models), be sure you have enabled that keys
|
||||
|
||||
Is it possible to run the examples through the command line inside the principal root:
|
||||
|
||||
For the first example:
|
||||
|
||||
```bash
|
||||
python -m examples.value_scraping
|
||||
```
|
||||
|
||||
For the second example:
|
||||
|
||||
```bash
|
||||
python -m examples.html_scraping
|
||||
```
|
||||
|
||||
# Example of output
|
||||
|
||||
Given the following input
|
||||
|
||||
@ -2,4 +2,4 @@
|
||||
from langchain_core.pydantic_v1 import BaseModel, Field
|
||||
|
||||
class _Response(BaseModel):
|
||||
title: str = Field(description='Title of the items')
|
||||
title: str = Field(description='Title of the news')
|
||||
|
||||
@ -51,4 +51,5 @@ if __name__ == "__main__":
|
||||
|
||||
generator_instance = Generator(values, 0, "gpt-3.5-turbo")
|
||||
|
||||
generator_instance.invocation(query_info)
|
||||
res = generator_instance.invocation(query_info)
|
||||
print(res)
|
||||
@ -14,4 +14,6 @@ if __name__ == "__main__":
|
||||
|
||||
generator_instance = Generator(values, 0, "gpt-3.5-turbo")
|
||||
|
||||
res = generator_instance.invocation(scraper("https://www.mockupworld.co", 4197))
|
||||
res = generator_instance.invocation(scraper("https://www.mockupworld.co", 4197))
|
||||
|
||||
print(res)
|
||||
Loading…
Reference in New Issue
Block a user