github germondai/trawl v1.3.0

5 hours ago

Highlights

  • Akamai Bot Manager bypass - detects and resolves behavioral and sec-cpt interstitials across scraper tiers and the general HTTP/HTTPS proxy
  • Self-healing browser pool - automatically reclaims stalled checkouts and recovers from hung Firefox close, launch, and restart operations
  • Authenticated upstream proxies - correctly passes proxy credentials to Camoufox in Tier 3 and Tier 4
  • Complete Tier 1 responses - fixes text and HTML responses being silently truncated to the 4 KiB challenge-detection preview
  • Reliable health reporting - /health now returns HTTP 503 when no genuinely usable browser capacity remains
  • Strict API validation - malformed /v1 and /scrape requests return clear HTTP 400 responses before tier execution
  • Improved Docker deployments - forwards upstream proxy configuration, preserves architecture-specific Camoufox metadata, and provides safer browser-pool defaults
  • Updated runtime stack - upgrades packages to their latest mutually compatible versions, including Camoufox JS 0.12.0
  • Refreshed documentation and website - expands proxy, residential routing, integrations, architecture, deployment, and challenge-coverage information

Major changes

  • 121f57b finalize the v1.3.0 release notes
  • 1f38465 return complete Tier 1 responses instead of the 4 KiB detection preview (PR #47). Thanks @nandyalu
  • 3a3226e update packages to v1.3.0 and align dependencies to compatible versions
  • 7456e75 preserve architecture-specific Camoufox metadata and tune browser-pool defaults
  • 2fc1031 update API, proxy, architecture, deployment, and integration documentation
  • 357767c refresh the landing page, comparisons, proxy showcase, and SEO metadata
  • d899a50 document residential and upstream proxy configuration
  • f036541 support authenticated upstream proxies in Camoufox browser tiers
  • 3a4d749 add browser-pool self-healing, bounded restarts, stalled-checkout recovery, and reliable health reporting (PR #37). Thanks @funkypenguin
  • 98e61d3 add Akamai Bot Manager challenge detection and bypass (PR #33). Thanks @edasque
  • 52e9700 validate /v1 and /scrape request bodies before execution (PR #42). Thanks @potatosips
  • f397815 pass upstream proxy configuration through Docker Compose deployments

Don't miss a new trawl release

NewReleases is sending notifications on new releases.