D4Vinci/Scrapling v0.3.9

Release v0.3.9

on GitHub

A new update with many important changes

🚀 New Stuff and quality of life changes

• Now the impersonate argument in Fetcher and FetcherSession can accept a list of browsers that the library will choose a random browser from them with each request.

from scrapling.fetchers import FetcherSession

with FetcherSession(impersonate=['chrome', 'firefox', 'safari']) as s:
  s.get('https://github.com/D4Vinci/Scrapling')

• A new argument to the clean method in TextHandler to remove html entities from the current text easily.
• Huge improvements to the documentation with more precise explanations of many parts and automatic translations of the main README.md file.

🐛 Bug Fixes

• Fixed a big issue with retrieving responses from browser-based fetchers. Now, there is intelligent content type detection that ensures response.body contains the rendered browser content only if the content is HTML; otherwise, it contains the raw content of the last request made. This allows you to download binary files and text-based files without having to find them wrapped in HTML tags, while being able to retrieve the rendered content you want from the website when fetching it.

🔨 Misc

• Updated the contributing guide to make it clearer and easier.
• Add a new workflow to enforce code quality tools (Same ones used as pre-commit hooks).

🙏 Special thanks to our Discord community for all the continuous testing and feedback

Big shoutout to our biggest Sponsors