github germondai/trawl v1.9.1

5 hours ago

Highlights

  • Keep browser pages open: create a session with keepPage: true, evaluate async JavaScript through /sessions/:id/eval and read its cookie jar through /sessions/:id/cookies, including HttpOnly cookies.
  • Scrape through sessions: POST /sessions/:id/scrape accepts the same options as /scrape with sessionId.
  • Fixes for retained pages: eval can access site-defined globals, outbound URL checks remain active between operations, and Cloudflare capture is reused across navigations.
  • Web/docs deployment fix: keep the required sources in the shared Docker build context, with a shorter README and updated setup and upgrade guides.

Major changes

  • ca0a156 adds retained pages and native eval/cookies endpoints from @everpcpc (#230, #236).
  • 0097c4e fixes main-context eval, persistent outbound URL checks and repeated Cloudflare capture (#237).
  • df8717b adds /sessions/:id/scrape with shared validation, response handling and metrics; conflicting session IDs return 400 (#238).
  • 9f65075 fixes the shared web/docs build context, and 73c1a1f refreshes the README and deployment documentation (#234).

Thanks to @everpcpc for the contribution.

Full changelog: v1.9.0...v1.9.1

Don't miss a new trawl release

NewReleases is sending notifications on new releases.