Pipepito/acestream-scraper v2.1

2.1

on GitHub

v2.1.0 adds tools for teaching the scraper new sources, comparing stream observations and understanding container storage. It also fixes ARM64 engine playback, improves WARP startup and DNS recovery, strengthens source-fetch protection and keeps older environment configurations working with clearer upgrade guidance.

Extraction builder and reusable recipes

Create your own extraction rules for websites, text lists and JSON/API endpoints without changing the scraper's code.

• HTML field selection: load or paste a sample, choose a repeating channel record, then select its name, AceStream ID and optional metadata.
• Regex builder: select a record and its fields in the raw source to generate editable patterns. Names and IDs stay paired within each record.
• JSON mapping: choose the records array and map fields from the endpoint's response, including nested keys and array indices.
• Preview before saving: review extracted channels, invalid records and duplicates. Tests do not import or remove channels, and extraction errors preserve existing channel data.
• Recipe catalogue and sharing: start from versioned example templates, then import, export or copy recipes between users and installations. The initial catalogue contains synthetic examples to adapt to your sources.

The installed extraction builder can fetch a source URL, test it with the scraper's extractor and save the recipe to that source. Sources without a recipe continue using automatic extraction.

The public helper works with pasted or uploaded samples and never connects to your installation. Build a recipe there, then import it into your scraper and test the actual URL.

Source samples support up to 32 MiB. Visual rendering has a separate 2 MiB budget and a markup-density limit to keep heavy pages responsive; larger sources can use raw selection, manually entered JSON paths or installed extraction tests. Original scripts and external assets are disabled in the visual preview, so it may differ from the live website.

Extraction builder docs · Contributing source formats

One user tools hub

The user tools page now brings together Docker setup, the Extraction builder and Docs. Build a Docker command, prepare a source recipe and find the relevant documentation from the same place.

Advanced stream statistics

Acestream Channels now shows the latest peer count, P2P download/upload speeds and sample age alongside the separately measured media bitrate, on desktop and phone layouts.

Manual, bulk and scheduled status checks collect the observations through the configured checking engine. Missing measurements stay unknown; a measured zero remains zero. Skipped checks and engine outages retain the previous sample with its original timestamp.

These are short observations, useful for comparing sources. Online status still requires verified audio/video packets. Historical charts, automatic playlist thresholds and looping-content detection are not included in this release.

Stream statistics docs · #122

Storage information in Overview

The new Overview → Storage panel shows:

• Mounted container directories and configured data directories.
• Space allocated to files in each directory.
• Filesystem capacity and available free space.
• Read-only mounts, incomplete scans and unavailable measurements.

Scans are bounded and cached so large directories do not block the app. Directories on the same filesystem share the reported free space. The report needs no Docker socket access; it shows container paths, while original host paths and Docker volume names remain available through your container manager or Compose configuration.

Storage docs

ARM64 playback and WARP reliability

• Fixed mod_detected playback failures: the bundled ARM engine now uses its distribution's correct startup entry point. Previously, health/version checks could pass while streams were refused. The pinned 3.2.17-fix engine and persistent device identity are retained; no downgrade is required.
• Reliable dual-engine DNS: playback and checking engines share the Android DNS listener without removing each other's socket. The surviving engine takes over when the listener's owner stops.
• WARP startup readiness: with ENABLE_WARP=true and WARP_ENABLE_NAT=true, startup waits for the tunnel to report Connected before launching the engines. A successful CLI response alone no longer counts as a connected tunnel.
• DNS follows connection changes: ARM engines refresh their DNS servers when WARP connects or disconnects, retaining the last valid settings during an incomplete configuration rewrite.

Verified on Apple Silicon: fresh ARM64 containers resolved and played all three tested IDs with WARP connected, and the dedicated checking engine verified media for all three through the normal status-check API. Tuner and Acexy streams ran for at least two minutes, DNS reconnects and a full container restart passed, and the app displayed 1080p video after restart.

The same IDs failed with failed to load content on the tested direct route and worked after WARP was reconnected to the same engine. This was a network-path dependency, not a reason to extend playback timeouts. WARP remains opt-in and needs its documented container capabilities and TUN device. Results do not guarantee every source or ARM board; ARMv7 remains experimental.

ARM64 test evidence · Docker and WARP setup

Documentation publishing

• The wiki is updated by the manual release job, after successful, non-dry-run latest promotion.
• Develop validates documentation without publishing the wiki or Docker Hub description.
• GitHub Pages continues serving the main branch's docs folder.
• Updated Docker Hub overview text is kept in docs/dockerhub/ for manual copying to Docker Hub.

Reliability and maintenance

• Safer source connections: HTTP/IPFS, ZeroNet, nested M3U, custom recipe and EPG fetches connect to validated DNS addresses and check redirect destinations. HTTPS preserves hostname and certificate validation. The synchronous source client can try another validated address when a connection fails. (#149)
• Bounded extraction work: preview uploads have size, concurrency and time limits before JSON parsing; extraction runs in disposable workers with resource limits.
• EPG maintenance: source management, XMLTV processing, channel matching and export responsibilities are separated behind the existing service interface. (#153)
• Expanded regression coverage: includes source connection security, recipe pairing and limits, statistics, storage reporting, and database upgrade/backup restoration.

Upgrading to v2.1.0

Keep your existing persistent volume mappings when updating the container. Existing SQLite databases are backed up before pending schema upgrades, and the new recipe/statistics fields are added automatically.

Legacy environment names continue working in v2.1.0. Startup logs and Overview show deprecation warnings with the preferred names. A nonempty canonical value takes precedence when both are set; otherwise the legacy value remains effective. This also applies to .env files.

Rename them when convenient, keeping their values unchanged. ENABLE_LEGACY_ENV_ALIASES is no longer needed and does not disable compatibility. Removal of the aliases is deferred beyond this release.

For installations using environment HTTP proxies: source fetches do not inherit HTTP_PROXY or HTTPS_PROXY, so proxy-only source access may need a routing adjustment. Source-fetch protection does not bypass container network routing, including WARP. Existing private-source defaults and configured gateway exemptions remain supported.

Configuration docs · Install and upgrade

Issues closed by this release

• Closes #122 — collect and display the latest stream peer and transfer statistics through manual and scheduled checks.
• Closes #149 — validate and pin scrape/EPG destinations, with redirect protection and documented private-source settings.
• Closes #153 — split the oversized EPG and channel-management modules while preserving their interfaces.
• Closes #159 — complete the security hardening epic: #148 and #150 are already closed, and this release completes #149.

#155 and its maintenance epic #160 remain open: legacy environment alias removal was deferred. Historical stream charts, automatic playlist thresholds and looping-content detection remain follow-up scope beyond #122's statistics collection request.

What's Changed

• Refresh project guides and skip unrelated CI for documentation changes by @Pipepito in #197
• Fix Jenkins sandbox failure in change selection by @Pipepito in #199
• Develop changes by @Pipepito in #198
• feat: add extraction tools, stream statistics and storage reporting by @Pipepito in #200
• Keep docs publication aligned with releases and document v2.1 by @Pipepito in #202
• Fix ARM engine mod_detected startup and shared DNS ownership by @Pipepito in #203
• v2.1.0 by @Pipepito in #201

Full Changelog: v2.0...v2.1