A solid update bringing the first platform spider template, a faster parser, and important fixes 🎉
🚀 New Stuff and quality of life changes
- Added
ShopifySpider, the first platform spider template! Extract every product from any Shopify-powered store through its JSON API without touching the website's HTML. Subclass it, set the store's domain, and you are done (Check the docs)from scrapling.spiders import ShopifySpider class MyStore(ShopifySpider): target_website = "example.com" result = MyStore().start()
- Added
--executable-pathto the CLI browser commands. Bothscrapling extract fetchandscrapling extract stealthy-fetchnow accept a custom Chromium-compatible browser executable, and fall back to theSCRAPLING_EXECUTABLE_PATHenvironment variable when the option isn't passed, bringing full parity with the MCP server (Solves #371)scrapling extract fetch "https://example.com" page.html --executable-path "/path/to/chromium"
🤖 Quality of life changes
- Made
find_by_textandfind_by_regexup to ~2x faster whenfirst_matchis enabled (the default) by wrapping elements lazily so the search stops at the first match, by @yetval in #370 - Updated the benchmarks with the new numbers against the latest versions of all libraries.
- Updated contribution rules
🐛 Bug Fixes
- Fixed the MCP server's fetch tools crashing on pages containing control characters with the error
All strings must be XML compatible, by @yetval in #368 (Fixes #366)
🙏 Special thanks to the community for all the continuous testing and feedback
Big shoutout to our Platinum Sponsors: