github rommapp/romm 5.3.0-alpha.1

pre-release4 hours ago

Warning

Structure B is gone.
We no longer detect the {platform}/roms/ layout, and an instance laid out that way refuses to start rather than scanning an empty library and marking every ROM missing. To keep a Structure B library, declare it, and note that it takes two templates:

filesystem:
  structure:
    default: "{platform}/roms/{game}"
    firmware: "{platform}/bios"

filesystem.roms_folder and filesystem.firmware_folder are gone too. Each named one path segment that a template now contains, and an instance that still sets either refuses to start, printing the equivalent template to paste:

filesystem:
  structure:
    default: "<your roms_folder>/{platform}/{game}"
    firmware: "<your firmware_folder>/{platform}"

See Custom library structure and the filesystem.structure schema.

Warning

Emulator streaming config is now one entry per container, not one per emulator.
A container's host, protocol and label become the defaults for every platform it serves, and a platform block only overrides what differs. Existing streaming setups need their config.yml rewritten before the feature will come back up.

See Emulator Streaming for the new schema.

Note

Environment variables

Emulator streaming

variable default description
STREAMING_LAUNCH_TIMEOUT 600 Seconds to wait for a streamed emulator to launch
STREAMING_STATE_HISTORY_LIMIT 50 Save states kept per ROM, emulator and user

Physical games

variable default description
UPC_LOOKUP_ENABLED true Barcode lookups when adding a physical game
UPC_LOOKUP_API_KEY - Key for the UPC lookup service
UPC_LOOKUP_URL https://api.upcitemdb.com/prod/trial/lookup UPC lookup endpoint

Recommendations

variable default description
ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS true Nightly rebuild of the recommendation index
SCHEDULED_BUILD_RECOMMENDATIONS_CRON 30 5 * * * When that rebuild runs

Browser players

variable default description
DISABLE_JSDOS false Disable the js-dos player for everyone
DISABLE_PICO8 false Disable the PICO-8 player for everyone

Metadata providers

variable default description
DEMOZOO_API_ENABLED false Enable the Demozoo metadata provider
POUET_API_ENABLED false Enable the Pouët metadata provider
CSDB_API_ENABLED false Enable the CSDb metadata provider
STEAM_API_ENABLED false Enable the Steam storefront metadata provider
HASHEOUS_API_URL https://hasheous.org/api/v1 Use a self-hosted Hasheous instance

General

variable default description
SCAN_WORKERS 4 Parallel scan workers, was 1
WEB_SERVER_CONCURRENCY 4 API server workers, was 1
DB_POOL_RECYCLE_SECONDS 300 Recycle pooled DB connections after this many seconds

Highlights

Custom library structure

Instead of guessing between two folder structures, describe your exact library layout in config.yml. {platform} is the platform folder, the final {game} is where a game starts, and anything else in {braces} is just a folder you use to organize things. You can override the layout per platform, and give a platform a list of layouts when its games sit at more than one depth. #3571

ROM identity is now content-hashed, so you can move files around and keep their saves, favorites, play history and collections. And if you came from Batocera, RetroBat or ES-DE, their platform folder names are recognized out of the box, so most libraries need no mapping at all. #4345

filesystem:
  structure:
    default: "roms/{platform}/{game}"
    firmware: "bios/{platform}"
    ps3: "roms/{platform}/{category}/{game}"
    snes: "roms/{platform}/{region}/{game}"
    nes:
      - "roms/{platform}/{game}"
      - "roms/{platform}/{category}/{game}"

Docs: Custom library structure, Folder name aliases

Emulator streaming V2

A massive update to emulator streaming. You configure one entry per container instead of one per emulator, a platform can pool several containers and grab whichever one is free, and platforms that do both browser and streamed play now get separate Play and Stream buttons. #4314

Saves and save states made while streaming are pulled back into your library with history and thumbnails. Multi-disc games get a disc-swap control, and whichever disc was mounted when you saved is the one remounted when you load. PS2 and GameCube can sync a whole memory card instead of individual save files, with a memory card library to create, rename, version, share and download them. Admins get a desktop session for setting up an emulator with no game running, plus a fleet view to force-release sessions across every container.

Docs: Emulator Streaming

Physical games

Own a game on a cartridge or disc, but have no ROM for it? You can now add it anyway, by name or by scanning its barcode with your camera, and it gets matched to metadata like any other game. #3848

Docs: Physical Games

Walkthroughs

Walkthroughs are now documents on a game, sitting next to its manuals. Upload your own, or paste a GameFAQs URL and have it fetched directly (as plain text). It'll even remember where you left off and show a progress bar for each user. #3816

Docs: Walkthroughs

Title IDs read from the binary

Scans now pull a game's native Title ID straight out of its binary on PSX, PS2, PS3, PSP, PS Vita, Switch, 3DS, Wii, Wii U, GameCube, Dreamcast, Xbox and Xbox 360. It identifies games on the platforms we don't hash, so a renamed or moved file keeps its saves and collections, and it tells the system where a game writes its saves, so device sync knows what to look for. #3925

Docs: Title ids read from the binary

Jukebox

A soundtrack player for your whole library that follows you as a mini player anywhere in the app. Browse by album, platform, artist or genre, or just put a playlist on: Free Radio picks a randomized, album-balanced hour, and there's a Decade Mix, recently added, and your personal favorites. #4197

Docs: Jukebox

JS-DOS and PICO-8 in the browser

Windows 3.x and 9x games now run on js-dos and its DOSBox-X backend. Note that js-dos saves stay in your browser and are not synced back to the server. #4061

PICO-8 cartridges (.p8 and .p8.png) are also now playable, on FAKE-08, with keyboard + mouse, gamepad, or touch input. #4400

Docs: js-dos, PICO-8

Four new metadata sources

Steam is now a metadata source for the win, linux and mac platforms, no API key required! #4241

Demozoo, Pouët and CSDb cover demoscene productions on platforms we already support. Tag a file (demozoo-108), or paste an id or a production URL into the ROM editor. #4221

scan:
  priority:
    metadata:
      - steam # PC platforms only

Docs: Metadata Providers

EmulatorJS cores and multi-disc games

A multi-disc game is handed to EmulatorJS all at once now, so you can swap discs from the emulator's own menu instead of going back and reloading. Set emulatorjs.disable_batch_bootup to boot only the disc you launched, if a core mishandles the batch. #3988

You can also preselect a core per platform with emulatorjs.default_cores. That sets the starting point, and anyone who has already picked a core on their device keeps their choice. #4452

Docs: Cores, Multi-disc games

Library-aware recommendations

Recommendations are now built from your own library, blending metadata overlap, the list pulled from IGDB, and what you actually play. You'll find them under Similar games on a game's page, and in a new Recommended for you row on the home screen. Enable it by setting ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS=true in your env vars. #4174

Docs: Recommendations

Automatic save sync

A save reaches the server when you save and quit, which means a closed tab, a browser crash or a laptop going to sleep leaves it on the device. Turn on emulatorjs.auto_save_sync and the player uploads the save whenever the emulator writes one instead, so there's nothing to lose by closing the tab mid-game. It's off by default because a game that saves often uploads often, and each upload is the whole save file. #4378

Docs: Automatic save sync

Sort and filter by game length

You can now sort and filter the gallery by how long a game takes, straight from the gallery rather than having to build a collection for it. It's based on the HowLongToBeat main story time, so it needs HLTB_API_ENABLED and a scan that matched the game. #4392

Docs: Game length

Minor changes

  • feat(v2): link Server Stats platform rows to their platform by @sdornan in #4252
  • feat(gallery): pick a game's default version by region priority by @sdornan in #4260
  • feat: split company metadata into publishers and developers by @gantoine in #3898
  • feat(saves): scope save discovery to a set of ROM IDs by @gantoine in #4304
  • feat(hasheous): make the Hasheous API URL configurable by @closer in #4300
  • feat(scan): refresh files scan and uploads into rom folders by @zurdi15 in #4277
  • feat(player): confirm before leaving a running game by @gantoine in #4308
  • feat(v2): make game header tags clickable by @gantoine in #4372
  • feat(v2): zoom the game cover, and dismiss the lightbox from anywhere on the backdrop by @gantoine in #4373
  • feat(home): add a game anniversary widget by @Spinnich in #4396
  • feat(firmware): let missing firmware be found, cleaned up, and kept out of the player by @Spinnich in #4077

Fixes

  • fix(v2): Fix missing v2 auth notifications by @Florian-Cullmann in #4246
  • fix(v2): remember the core a game was launched with by @gantoine in #4251
  • fix(frontend): keep collection ROM counts in step with the gallery by @sdornan in #4261
  • fix(hltb): keep a pinned HowLongToBeat match through a rescan by @sdornan in #4262
  • fix(roms): Fix N+1 and entity sort in GET /api/roms by @tmgast in #4264
  • fix(auth): send PKCE on the OIDC authorization request by @jgarcesres in #4267
  • fix: Preserve nested arcade ROM filenames by @mvanhorn in #4271
  • fix: map Neo Geo MVS/AES to the RetroAchievements Arcade console by @gantoine in #4282
  • fix: restore HowLongToBeat lookups and repair endpoint discovery by @sdornan in #4286
  • fix(scan): let update metadata scans refresh artwork, and record uploads explicitly by @sdornan in #4160
  • fix(screenscraper): map mega-duck-slash-cougar-boy to ss id 90 by @gantoine in #4307
  • fix(platforms): add doom as a platform with its standalone provider mappings by @gantoine in #4309
  • fix(roms): stop a racing upload from undoing a folder conversion by @tmgast in #4316
  • fix(manual): only ask where a manual goes when the answer is open by @tmgast in #4315
  • fix(logs): stop a multi-line record painting over the rows after it by @sdornan in #4303
  • fix: allow INI files in library scans by @mvanhorn in #4328
  • fix(frontend): Add crossorigin attribute to manifest link so CloudFlare Tunnels does not break site.webmanifest by @Macleykun in #4332
  • fix(player): Show a clear message instead of the generic error for threaded cores over non-secure origins by @TRusselo in #4313
  • fix(v2): read the date picker's value in UTC, not local time by @sdornan in #4322
  • fix(backend): Rate limit the unauthenticated metadata heartbeat endpoint by @gantoine in #4327
  • fix(download): make cached zips readable by nginx and serialize their builds by @tmgast in #4329
  • fix(scan): Follow-ups to the sigil title ID branch by @gantoine in #4333
  • fix(export): write gamelist media where ES-DE and Batocera frontends find it by @sdornan in #4344
  • fix(retroachievements): don't match ROMs against games with no achievement set by @Spinnich in #4352
  • fix(scan): let a complete rescan clear the ids it rematched by @Spinnich in #4353
  • fix(collections): honor "Show autogenerated collections" on the v2 collections page by @Spinnich in #4355
  • fix(hashing): read zips zipfile can't decode through 7zz by @sdornan in #4357
  • fix(db): make virtual_collections and generated columns collation-agnostic by @gantoine in #4374
  • fix(v2): don't reload the page on reconnect while a game is running by @gantoine in #4369
  • fix(db): make migrations collation-agnostic on MariaDB by @gantoine in #4371
  • fix(export): share the ES-DE media folders with the Pegasus export by @sdornan in #4370
  • fix(auth): require platforms.write and enforce platform visibility on the export endpoints by @gantoine in #4376
  • fix(export): merge gamelist.xml and metadata.pegasus.txt instead of overwriting by @sdornan in #4368
  • fix(screenscraper): let the daily-quota breaker recover by @Spinnich in #4360
  • fix(server): default to 4 API workers and fix the WEB_SERVER_CONCURRENCY guidance by @Spinnich in #4385
  • fix(gallery): join roms_metadata before the grouped dedup window by @gantoine in #4390
  • fix(docker): bump sigil past the zstd asm link failure by @sdornan in #4404
  • fix(firmware): 404 the content endpoints when the file is gone by @gantoine in #4405
  • fix(v2): read the webp flag the heartbeat actually sends by @gantoine in #4406
  • fix(auth): sanitize the username an OIDC provider hands us by @gantoine in #4407
  • fix(ra): return achievements in RetroAchievements' display order by @gantoine in #4408
  • fix(v2): keep a click in selection mode from navigating away by @gantoine in #4409
  • fix(db): recycle pooled connections before the server drops them by @gantoine in #4411
  • fix(v2): keep the player from booting a rom's media instead of the game by @gantoine in #4412
  • fix(ss): let region_mode decide the title, not just the artwork by @gantoine in #4413
  • fix(scan): tell the clients when SCAN_TIMEOUT killed the scan by @gantoine in #4414
  • fix(roms): keep unplayed games when sorting by a rom_user column by @zurdi15 in #4424
  • fix(player): keep RomM chrome off the emulator on mobile by @zurdi15 in #4425
  • fix(tasks): match a running task by its registry key by @sdornan in #4430
  • fix(tasks): refill the Switch TitleDB store a schema bump drops by @sdornan in #4432
  • fix(frontend): 404 unmatched URLs in v2 instead of "not migrated yet" by @sdornan in #4455
  • fix(frontend): stop ESLint and Prettier fighting over formatting by @sdornan in #4456
  • fix(v2): announce the hash chip's disclosure state and action by @zurdi15 in #4451
  • fix(v2): give the gallery a real Select all in both layouts by @zurdi15 in #4453
  • fix(roms): sort unset per-user and metadata keys last on every engine by @zurdi15 in #4454

Note

API changes

Change Description
⚠️ POST /export/gamelist-xml, POST /export/pegasus Now require the PLATFORMS/WRITE grant instead of ROMS/READ, and enforce platform visibility.
⚠️ GET /setup/library Replaces the detected_structure discriminator (struct_a/struct_b/null) with library_ready: bool plus library_structure: str.
⚠️ ConfigResponse Splits DEFAULT_EXCLUDED_DIRS into DEFAULT_EXCLUDED_PLATFORM_DIRS and DEFAULT_EXCLUDED_MULTI_FILE_DIRS.
ConfigResponse Adds EJS_DEFAULT_CORES and EJS_ENABLE_AUTO_SAVE_SYNC.
HeartbeatResponse Adds DEMOZOO_API_ENABLED, POUET_API_ENABLED, CSDB_API_ENABLED and STEAM_API_ENABLED; TITLE_ID_EXTRACTION_ENABLED; DISABLE_JSDOS and DISABLE_PICO8.
DetailedRomSchema Adds publishers and developers alongside the existing companies, plus is_physical, upc, has_file_on_disk, title_id, save_target and the demozoo/pouet/csdb/steam id and metadata fields.
PUT /roms/{id}/identity Rebind a ROM's identity after the files behind it moved.
POST /roms/physical Create a ROM with no file on disk.
GET /roms/{id}/similar Library-aware similar games for one ROM.
GET /recommendations The requester's recommended games.
GET /music/games, /game-genres, /platforms, /stats Jukebox browsing facets over the soundtrack library.

Walkthrough endpoints

Method Path Description
POST /roms/{id}/walkthroughs/files Upload a walkthrough document
POST /roms/{id}/walkthroughs/gamefaqs Import one from a GameFAQs URL, as plain text
DELETE /roms/{id}/walkthroughs/files/{file_id} Delete a walkthrough
GET /roms/{rom_id}/files/{file_id}/progress The requester's reading progress for a document
PUT /roms/{rom_id}/files/{file_id}/progress Record reading progress

Memory card endpoints

Method Path Description
GET /memory-cards The requester's memory cards
GET /memory-cards/shared Cards other users have shared
GET /memory-cards/{id}, /{id}/content One card, and its image
GET /memory-cards/{id}/versions, /versions/{id}/content Version history, and one version's image
POST /memory-cards Create a card, or upload your own
POST /memory-cards/{id}/versions Snapshot the current state as a version
POST /memory-cards/delete Delete cards in bulk
PUT /memory-cards/{id}, /{id}/visibility Rename a card, or share it

Streaming endpoints

Method Path Description
GET /streaming/containers The configured containers and what they serve
GET /streaming/sessions/joinable Sessions open to a second player
GET /streaming/sessions/{platform}/status One session's live state
POST /streaming/desktop Admin desktop session with no game running
POST /streaming/sessions/{platform}/join Join someone else's session
POST /streaming/sessions/{platform}/heartbeat Keep a claim alive
POST /streaming/sessions/{platform}/swap-disc Mount another disc of a multi-disc game
POST /streaming/sessions/{platform}/state-frame Attach a thumbnail to a captured state

Other changes

  • chore(agents): add a review-polish skill from recurring PR corrections by @gantoine in #4250
  • chore(deps): bump starlette from 1.0.1 to 1.6.0 by @sdornan in #4254
  • refactor(v2): give the metadata provider taxonomy one home by @sdornan in #4263
  • chore(deps-dev): Bump brace-expansion from 2.0.3 to 2.1.4 in /frontend by @dependabot[bot] in #4266
  • chore(deps): Bump asyncssh from 2.23.0 to 2.23.1 by @dependabot[bot] in #4292
  • refactor(v2): extract a shared player shell by @gantoine in #4301
  • refactor(assets): make rom_ids the only ROM scope on saves and states by @gantoine in #4305
  • docs(skills): merge pre-pr-verification into review-polish by @gantoine in #4311
  • chore(deps): Bump tornado from 6.5.7 to 6.5.8 by @dependabot[bot] in #4324
  • chore(deps-dev): Bump postcss-selector-parser from 7.1.1 to 7.1.5 in /frontend by @dependabot[bot] in #4325
  • chore(deps-dev): Bump browserslist from 4.28.1 to 4.28.8 in /frontend by @dependabot[bot] in #4326
  • chore(deps-dev): Bump fast-uri from 3.1.5 to 3.1.7 in /frontend by @dependabot[bot] in #4330
  • chore(deps-dev): Bump @humanfs/node from 0.16.7 to 0.16.8 in /frontend by @dependabot[bot] in #4331
  • chore: drop the vcrpy exclude-newer override by @sdornan in #4334
  • refactor(tasks): schedule with RQ instead of rq-scheduler by @sdornan in #4274
  • docs(agents): never run the full backend test suite locally by @gantoine in #4336
  • perf(scan): give scans their own queue and worker by @sdornan in #4275
  • ci: Bump GitHub Actions to Node 24 runtimes by @sdornan in #4341
  • chore(deps): Bump cryptography from 49.0.0 to 50.0.0 by @dependabot[bot] in #4342
  • perf(scan): coalesce scan progress reports, and give the rom refetch one home by @gantoine in #4340
  • chore(dev): move the Authentik stack into docker-compose.oidc.yml by @gantoine in #4367
  • chore(python): move the backend to Python 3.14 by @sdornan in #4359
  • perf(tests): stop the suite waiting on connections it never wanted by @gantoine in #4375
  • perf(backend): add missing gallery indexes and reconcile model/schema index drift by @sdornan in #4384
  • perf(scan): read files in 256 KiB chunks instead of 8 KiB by @Spinnich in #4387
  • perf(scan): default to 4 scan workers instead of 1 by @Spinnich in #4386
  • chore(tasks): Quiet RQ per-tick scheduler heartbeat log line by @gantoine in #4393
  • chore(deps-dev): bump vitest from 4.1.8 to 4.1.11 in /frontend by @dependabot[bot] in #4399
  • chore(deps-dev): bump js-yaml from 4.3.1 to 4.3.2 in /frontend by @dependabot[bot] in #4401
  • security: strip control characters from names reaching the mod_zip manifest by @gantoine in #4402
  • perf(gallery): materialize the ids sibling_roms matches on by @Spinnich in #4394
  • chore(agents): Add a pr-ready skill chaining the four PR review passes by @sdornan in #4415
  • perf(metadata): stop storing the metadata dumps twice over in the cache by @gantoine in #4416
  • perf(metadata): compress the cache-only metadata dump records by @sdornan in #4426
  • docs: add a one-click deploys section to the README by @gantoine in #4427
  • chore(deps): upgrade rq to 2.12 by @sdornan in #4428
  • ci: align and manage the Docker service image pins by @sdornan in #4433
  • docs: tighten the README copy by @gantoine in #4438
  • ci: build each arch on a native runner and cache layers by @sdornan in #4437
  • chore(deps): update astral-sh/setup-uv action to v10.1.0 by @renovate[bot] in #4440
  • chore(deps): update postgres docker tag to v18 by @renovate[bot] in #4444
  • chore(deps): update github actions (major) by @renovate[bot] in #4445
  • ci: pin the amd64 image build to ubuntu-24.04 by @sdornan in #4458
  • ci: run trunk check on master pushes by @sdornan in #4457
  • chore(deps): update ghcr.io/goauthentik/server docker tag to v2024.12.5 by @renovate[bot] in #4441
  • docs(examples): drop the gc override from the config example by @sdornan in #4461

New Contributors

Full Changelog: 5.2.0...5.3.0-alpha.1

Don't miss a new romm release

NewReleases is sending notifications on new releases.