What's Changed
New feature: One-click MAME Redump DAT sync
- Added MAME Redump DAT sync service (
dat_sync.py) — one-click sync of DAT files from the MAMERedump GitHub repository - New sync/cancel API endpoints (
/dat/sync,/dat/sync/status,/dat/sync/cancel) with streaming progress - Frontend UI for triggering sync, viewing progress, and cancelling in-progress syncs
- Configurable via
MAMEREDUMP_GITHUB_TOKENenvironment variable for authenticated API access - Race-condition safe 409 detection + streaming size cap in downloads
- Deferred DAT deletion — only removes old DATs after all imports succeed
- Rollback new DATs on partial failure +
complete_with_errorsstatus - Per-file 100MB size guard on raw downloads
- Validates HTTPS scheme before
urlopento preventfile://access (CWE-73)
NKit2 removal
- Completely removed NKit2 service (
app/services/nkit2.py) and vendor directory - Removed NKit2-related conversion routes and model references
- Cleaned up Dockerfile (removed NKit2 binary setup)
Reliability & data integrity
- Crash-safe DAT store writes — atomic file writes + normalize error field to empty string
- Bulk DAT store writes with
process_queueexception logging - Extracted
_import_dat_core()to eliminateimport_datduplication - Consolidated background task tracking into
app.state.background_tasks - Guard
_ensure_initwith dedicated lock to prevent init race condition - Derive
onImported(hasDats)from actual loaded DATs after sync completes - URL-encode
refin_raw_url()for safe GitHub API calls
Security fixes
- Fixed Generic Object Injection Sink in
api.js— use destructuring on parsed JSON instead ofinoperator to prevent prototype pollution - Removed PAT from raw downloads
- Pin labeler action to commit SHA
CI & repo automation
- Added GitHub Actions labeler workflow with PR auto-labeling config (
.github/labeler.yml) - Added stale issue/PR bot (
.github/workflows/stale.yml)
Tests
- Added
tests/test_dat_sync.py— comprehensive test suite for the DAT sync service - Added
tests/test_dat_routes.py— route-level tests for DAT endpoints
Full Changelog: v3.5.3...v3.6.0