github D4Vinci/Scrapling v0.3.8
Release v0.3.8

latest releases: v0.4.7, v0.4.6, v0.4.5...
5 months ago

A new update with many important changes

🚀 New Stuff and quality of life changes

  • For all browser-based fetchers: websites that never finish loading their requests won't crash the code now if you used network_idle with them.
  • The logic for collecting/checking for page content in browser-based fetchers has been changed to make browsers more stable on Windows systems now, as Linux/MacOS (All this difference in behaviour is because of Playwright's different implementation on Windows systems).
  • Refactored all the validation logic, which made all requests done from all browser-based fetchers faster by 8-15%
  • A New option called extra_flags has been added to DynamicFetcher and its session to allow users to add custom Chrome flags to the existing ones while launching the browser.
  • Reverted the route logic for catching responses (changed in the last version) to use the old routing version when page_action is used. This was added to collect the latest version of a page's content in case page_action changes it without making a request. (Thanks for @gembleman to pointing it in #100 and #102 )

🐛 Bug Fixes

  • Fixed a typo in load_dom in DynamicSession's async_fetch
  • Fixed an issue with Cloudflare solver that made the solver wait forever for embedded captchas that don't disappear after solving. Now it will wait for the captcha to disappear for 30 seconds, then assume it's the type that doesn't disappear (Fixes #100 )

🔨 Misc

  • Now the Docker image is automatically pushed to Dockerhub and GitHub's container registry for user convenience.
  • Added a new documentation page to show how to use Scrapeless browser with Scrapling.

🙏 Special thanks to our Discord community for all the continuous testing and feedback


Big shoutout to our biggest Sponsors

Don't miss a new Scrapling release

NewReleases is sending notifications on new releases.