mirror of
https://github.com/VinciGit00/Scrapegraph-ai.git
synced 2026-06-28 21:01:55 +08:00
Optimize utils modules for better performance and maintainability - Improve HTML cleanup and minification: - Combine regex operations for better performance - Add better error handling for HTML processing - Optimize tag removal and attribute filtering - Enhance deep copy functionality: - Add special case handling for primitive types - Improve type checking and error handling - Optimize recursive copying for collections - Refactor web search functionality: - Add input validation and error handling - Split search logic into separate helper functions - Improve proxy handling and configuration - Add better timeout and error management - Optimize URL filtering and processing Technical improvements: - Better type hints and documentation - More efficient data structures - Improved error handling and validation - Reduced code duplication - Better separation of concerns No breaking changes - all existing functionality maintained |
||
|---|---|---|
| .. | ||
| screenshot_scraping | ||
| tokenizers | ||
| __init__.py | ||
| cleanup_code.py | ||
| cleanup_html.py | ||
| code_error_analysis.py | ||
| code_error_correction.py | ||
| convert_to_md.py | ||
| copy.py | ||
| custom_callback.py | ||
| data_export.py | ||
| dict_content_compare.py | ||
| llm_callback_manager.py | ||
| logging.py | ||
| model_costs.py | ||
| output_parser.py | ||
| parse_state_keys.py | ||
| prettify_exec_info.py | ||
| proxy_rotation.py | ||
| research_web.py | ||
| save_audio_from_bytes.py | ||
| save_code_to_file.py | ||
| schema_trasform.py | ||
| split_text_into_chunks.py | ||
| sys_dynamic_import.py | ||
| tokenizer.py | ||