pacnpal/compressatorium v4.0.1

Release v4.0.1 - CSO/ZSO/DAX

on GitHub

4.0.1 (2026-06-02)

PSP / PS2 support: CSO / ZSO / DAX via maxcso (#127)

New

• A fifth tool: CSO. Compress PSP/PS2 .iso disc images to every format maxcso writes, and back. Five modes: cso_compress (ISO → CSO v1, the universally-supported default), cso2_compress (ISO → CSO v2, better block alignment for recent emulators), zso_compress (ISO → ZSO, lz4, faster to decode), dax_compress (ISO → DAX, legacy PSP format), and cso_decompress (CSO/ZSO/DAX → ISO). PPSSPP and PCSX2 read these formats directly, so no separate decompress step is needed to play. Available in the Web UI and REST API, with live progress, single + batch verify, and file-info. No keys required.
• Compression-effort presets. The compress modes expose a Fast / Default / Max effort preset (mapped to maxcso's --fast, default trials, and extra --use-zopfli/--use-libdeflate / --use-lz4brute trials), reusing the same compression picker the other tools use and remembered per tool between sessions. CSO ships a strong default (the Max preset, smallest output).
• Reset to default. Every compression-capable tool (CHD, Dolphin, Switch, CSO) gains a "Reset to default" button under its compression picker that restores that tool's codec/layout/level/effort to its default and confirms with a toast (disabled when already at the default).
• Lossless and verifiable. The compress/decompress round trip reproduces the original .iso byte-for-byte. Verify runs maxcso's --crc over the compressed container, and delete-on-verify is offered for the compress modes. CSO/ZSO/DAX sources can also be converted from inside ZIP/7z/RAR archives.

Internal

• maxcso is built from source in a dedicated Dockerfile builder stage, pinned to a tagged release for reproducible images (build deps liblz4/libuv/libdeflate/zlib; the runtime stage ships their shared libs, including Debian trixie's time64 libuv1t64). The binary lands at /usr/local/bin/maxcso, exposed via the new MAXCSO_PATH env var.
• The tool slots into the registry as cso with zero job-pipeline edits — job_manager / convert / files / info all dispatch through the registry. New app/services/maxcso.py service + app/services/tools/maxcso.py plugin, a CsoInfo model and cso_* FileEntry fields, the five ConversionMode entries (CSO_COMPRESS / CSO2_COMPRESS / ZSO_COMPRESS / DAX_COMPRESS / CSO_DECOMPRESS), and an info.py _VERIFY_CONFIG entry behind the shared verify-route factory. CSO outputs are picked up automatically by the registry-driven library scan / DAT-match (scannable_extensions()) and the tool-neutral priority/timeout policy (COMPRESSATORIUM_MAXCSO_*).
• Tests: new tests/test_cso_service.py (convert / verify / cancel / output-path mapping) and tests/test_cso_routes.py (info + single/batch SSE verify), plus extended registry, dispatch-routing, mode-parity, outputs-parity, and archive-conversion suites. docs/ADDING_PLATFORMS_AND_TOOLS.md was corrected for the globals it had omitted — the _VERIFY_CONFIG step, the fileIcon.js / FileRow.svelte / conversion.svelte.js / HelpView.svelte frontend touch-points, and the inaccurate conftest "binary stub" note.

Library scan + DAT matching now cover every format (not just CHD)

New

• The background library scan discovers all tool outputs, not just .chd. Discovery is registry-driven (every registered tool's output / verify extensions), so Dolphin (.rvz/.wia/.gcz), 3DS (.z3ds/.zcci/.zcia), Switch (.nsz/.xcz), and plain .iso/.bin libraries are now visited. A new scan phase (Phase 3) primes the DAT-match cache for every discovered file, so non-CHD libraries get cached match results just like CHDs. Discovery resolves symlinks (so a file is matched/cached under the same path the on-demand match endpoints use), drops symlinks whose target escapes the configured volumes, and matches on the real file suffix (a .ciso is not admitted just because it ends with .iso). When no DATs are imported, Phase 3 is skipped and the job progress advances cleanly instead of appearing stuck.
• Compressed Dolphin images match Redump DATs. Matching now uses each tool's embedded/derivable content hashes via a per-tool hook: chdman reports the CHD header/data SHA1, and Dolphin reports the reconstructed disc SHA1 from dolphin-tool verify — the hash MAME Redump records for GameCube/Wii discs — so .rvz/.wia/.gcz outputs match without the container bytes being identical. Formats with no embedded hash (3DS/Switch/plain .iso/.bin) fall back to file-level SHA1 as before.
• extractcd .bin data tracks are now scannable. CHDMAN's extractcd writes a .cue plus a .bin data track; Redump DATs index the .bin bytes, so the scan now discovers and matches the .bin (previously only the tiny .cue was visited).
• Heavy Dolphin hashing is bounded and interruptible. The dolphin-tool verify disc-hash pass runs under the match workload lane (so MAX_MATCH_CONCURRENCY caps concurrent full-disc reconstructions), honors the MATCH_MAX_FILE_SIZE cap before doing the work, and is cancelled promptly when you cancel the scan or DAT-match job instead of running to completion.

Fixed

• Stale / failed hashes no longer poison the match cache. A transient hash failure (e.g. dolphin-tool verify crashing, a stat error, or a stale CHD whose metadata refresh failed) is recorded as a non-cacheable error rather than a false "unmatched", and a forced rescan clears any existing match row whose recompute is non-cacheable so /dat/matches/lookup doesn't keep showing an outdated result. A stale CHD still suppresses its outdated embedded disc hash but falls back to a file-level SHA1 of the current file.

Internal

• ToolPlugin gains embedded_hashes(path, *, cancel_event=None) (default empty) and an embedded_hash_is_exhaustive flag. EmbeddedHashUnavailable signals a non-cacheable miss; for exhaustive tools (Dolphin, where the recompressed container can never be DAT-indexed) a content-hash miss is definitive and skips the file-level fallback, while non-exhaustive tools (CHD) still fall back. ToolRegistry gains output_extensions() / scannable_extensions(); DATStore gains delete_match(). The DAT lookup + result-dict building is centralized in one _lookup_sha1_match() helper shared by the embedded-hash and file-level paths.
• The shared SubprocessRunner gains a one-shot, cancel/timeout-aware run_capture() (PID-tracked, TERM→KILL on abort, honoring the tool nice/ioprio policy) that backs the Dolphin disc-hash extraction. Cancellation threads from the scan / DAT-match jobs through embedded_hashes → disc_hashes → run_capture. media_type, disc-ID embedding, and the chd_metadata_store cache remain CHD-only. The plugin contract additions are documented in docs/DESIGN_tool_plugin_architecture.md.

Tool-neutral process priority and timeout settings (#132)

New

• The process-priority and timeout knobs are no longer chdman-specific. The nice level, I/O priority, and info/verify timeouts govern every conversion tool (chdman, Dolphin, 3DS, Switch), not just chdman, so they're now exposed under tool-neutral names: COMPRESSATORIUM_TOOL_NICE, COMPRESSATORIUM_TOOL_IOPRIO_CLASS, COMPRESSATORIUM_TOOL_IOPRIO_LEVEL, COMPRESSATORIUM_TOOL_INFO_TIMEOUT, and COMPRESSATORIUM_TOOL_VERIFY_TIMEOUT. The old CHD_CHDMAN_* / CHD_INFO_TIMEOUT / CHD_VERIFY_TIMEOUT names keep working as aliases, so existing setups are unaffected.
• Per-tool overrides. You can give a single tool a different priority or timeout with COMPRESSATORIUM_<TOOL>_* (e.g. COMPRESSATORIUM_DOLPHIN_TOOL_NICE=15, COMPRESSATORIUM_NSZ_VERIFY_TIMEOUT=300), falling back to the shared default when unset. <TOOL> is CHDMAN, DOLPHIN_TOOL, NSZ, Z3DS, or MAXCSO; the info-timeout override applies only to the tools whose info runs a subprocess (chdman, Dolphin).
• Switch/3DS verification now honors the verify timeout. Long or hung nsz/z3ds verify runs are bounded by COMPRESSATORIUM_TOOL_VERIFY_TIMEOUT (or the per-tool override), matching chdman and Dolphin; previously these ran unbounded.

Internal

• The priority/timeout policy is resolved once in the shared SubprocessRunner (nice_value / ioprio_prefix / nice_prefix / apply_nice / info_timeout / verify_timeout) instead of each service re-reading a chdman-named setting. The Docker image no longer bakes the priority defaults as ENV (which would have shadowed the lower-precedence legacy aliases); the 10/2/6 defaults come from app config.

Delete-on-verify no longer leaks verification records (#130)

Fixed

• Non-CHD verify-class sources are cleared from the verification store on delete-on-verify. When a conversion deleted its source after verifying, job_manager only cleared the persistent verification record if the source ended in .chd. But verification_store is tool-wide — it tracks verify-class outputs for every registered tool — so deleting a non-CHD verify-class source (.rvz/.wia/.gcz/.z3ds/.zcci/.zcia/.nsz/.xcz) left a stale record pointing at a file that no longer exists. The clear is now gated on ext in registry.verify_extensions() (tool-wide), mirroring the already-generalized file-browser delete/rename paths, while the CHD-specific chd_metadata_store clear stays gated on .chd.

Internal

• tests/test_mode_parity_fixes.py gained a regression test deleting a .rvz source on verify (asserting the tool-wide store is cleared but chd_metadata_store is not), and the existing z3ds test now asserts a non-verify-class .3ds source leaves both stores untouched.