Commit Graph

24 Commits

Author SHA1 Message Date
aziz-ullah-khan
c218546a3d feat: Serper API integration for Google search 2024-11-02 15:02:08 +05:00
Umut CAN
827f7260ad This commit focuses on optimizing the utility modules in the codebase for better performance and maintainability. Key improvements include: - More efficient HTML processing with combined regex operations and optimized tag handling - Enhanced deep copy functionality with better type handling and optimized recursion - Refactored web search with improved error handling and modular helper functions The changes maintain all existing functionality while improving code quality, performance, and maintainability. Documentation and type hints have been enhanced throughout.
Optimize utils modules for better performance and maintainability

- Improve HTML cleanup and minification:
  - Combine regex operations for better performance
  - Add better error handling for HTML processing
  - Optimize tag removal and attribute filtering

- Enhance deep copy functionality:
  - Add special case handling for primitive types
  - Improve type checking and error handling
  - Optimize recursive copying for collections

- Refactor web search functionality:
  - Add input validation and error handling
  - Split search logic into separate helper functions
  - Improve proxy handling and configuration
  - Add better timeout and error management
  - Optimize URL filtering and processing

Technical improvements:
- Better type hints and documentation
- More efficient data structures
- Improved error handling and validation
- Reduced code duplication
- Better separation of concerns

No breaking changes - all existing functionality maintained
2024-10-28 22:40:32 +03:00
aziz-ullah-khan
e828c7010a perf: Proxy integration in googlesearch 2024-10-11 12:30:58 +05:00
aziz-ullah-khan
46195d65ca Update: proxy integration in googlesearch 2024-10-11 10:04:28 +05:00
Marco Vinciguerra
26f89d895d feat: refactoring of research web 2024-10-01 10:25:09 +02:00
Marco Vinciguerra
d0969feb7c removed unused imports and comments + removed dead code 2024-09-23 09:25:13 +02:00
Marco Vinciguerra
7eda6bc06b fix: issue about parser 2024-09-22 18:55:05 +02:00
Marco Vinciguerra
fe3aa28fe7 refactoring of the code
Some checks failed
/ build (push) Has been cancelled
Release / Build (push) Has been cancelled
Release / Release (push) Has been cancelled
2024-09-11 16:04:43 +02:00
Marco Vinciguerra
9355507a2d feat: refactoring of the code 2024-08-02 12:00:00 +02:00
Federico Aguzzi
7ba2f6ae0b chore: correct search engine name
Some checks failed
/ build (push) Has been cancelled
2024-07-14 22:39:34 +02:00
Marco Vinciguerra
7e5789baa3 Update research_web.py 2024-07-14 19:31:22 +02:00
Marco Vinciguerra
aa2160c108 feat: add research with bing + test function
Some checks failed
/ build (3.10) (push) Has been cancelled
2024-06-18 21:28:29 +02:00
Marco Vinciguerra
073d226723 feat: add new search engine avaiability and new tests 2024-06-18 14:35:13 +02:00
Marco Perini
5fb9115330 feat(version): python 3.12 is now supported 🚀 2024-05-26 08:49:27 +02:00
VinciGit00
aeb1acbf05 feat: refactoring search function 2024-05-03 21:06:09 +02:00
Perinim
cf038b33ea docs: update utils docstrings 2024-05-01 12:35:12 +02:00
EURAC\marperini
57a53bd6af refactor examples 2024-04-06 13:22:42 +02:00
EURAC\marperini
c2709aa50c working search graph 2024-04-06 13:01:04 +02:00
VinciGit00
b53bfefeea removed junk if 2024-04-02 12:57:27 +02:00
VinciGit00
471df48a22 add google search 2024-04-01 13:36:02 +02:00
VinciGit00
b8a3590235 add SearcInetnetNode 2024-03-28 13:14:39 +01:00
VinciGit00
fff1f9ee51 new function for searching 2024-03-28 12:41:45 +01:00
VinciGit00
672e9a5191 refactoring of research function 2024-03-27 21:29:10 +01:00
VinciGit00
97d0a6887f add reserch web function 2024-03-27 21:22:04 +01:00