crwlrsoft/crawler v1.10.0

on GitHub

Added

• URL refiners: UrlRefiner::withScheme(), UrlRefiner::withHost(), UrlRefiner::withPort(), UrlRefiner::withoutPort(), UrlRefiner::withPath(), UrlRefiner::withQuery(), UrlRefiner::withoutQuery(), UrlRefiner::withFragment() and UrlRefiner::withoutFragment().
• New paginator stop rules PaginatorStopRules::contains() and PaginatorStopRules::notContains().
• Static method UserAgent::mozilla5CompatibleBrowser() to get a UserAgent instance with the user agent string Mozilla/5.0 (compatible) and also the new method withMozilla5CompatibleUserAgent in the AnonymousHttpCrawlerBuilder that you can use like this: HttpCrawler::make()->withMozilla5CompatibleUserAgent().