What's changed
- Fixed a bug where the logging fetch logging sentence was showing in the first request only.
- The default behavior for Playwright API while browsing a page is returning the first response that fulfills the load state given to the
gotomethod["load", "domcontentloaded", "networkidle"]so if a website has a wait page like Cloudflare's one that redirects you to the real website afterward, Playwright will return the first status code which in this case would be something like 403. This update solves this issue for bothPlaywrightFetcherandStealthyFetcheras both are using Playwright API so the result depends on Playwright's default behavior no more. - Added support for proxies that use SOCKS proxies in the
Fetcherclass. - Fixed the type hint for the
wait_selector_stateargument so now it will show the accurate values you should use while auto-completing.
Note
A friendly reminder that maintaining and improving Scrapling takes a lot of time and effort which I have been happily doing for months even though it's becoming harder. So, if you like Scrapling and want it to keep improving, you can help by supporting me through the Sponsor button.