a1ex4/ownfoil 2.5.0

on GitHub

Dedicated Ownfoil menu in Sphaira with automatic shop discovery on the local network,
Titles are identified from the files themselves instead of only from titledb, a details page for every game, a page to review and manage library files, duplicate files handling, self-hosted artwork downloading and serving.

Sphaira's Ownfoil support

Ownfoil was started to be a Tinfoil shop backend, and with it the development was shaped around its limitations: while it was adapted for the "shop" ecosystems it was supporting, a self-hosted library backend should be able provide a better eShop like experience, with better performance. Legacy clients browse the shop as a list of files, using their own title metadata which can differ from what Ownfoil actually identified.

The GraphQL API introduced in 2.4.0 was groundwork to support this change: as title metadatas, files, and apps can be queried efficiently, anyone can create a homebrew client to browse and install content served by Ownfoil. In this version it has been completed to support a new client integration, Sphaira: since v1.0.8 there is a built-in Ownfoil menu, under Menu → Menus → Ownfoil, which is the recommended client from now on, as it fully makes use of the backend features. It looks like this:

It features:

• automatic server discovery on the network, no typing IPs and ports
• server config carry both local and remote address, http/https protocols supported
• browse library by New games, Updates, DLC, All games, Search, paginated and with sorting
• all artwork served by Ownfoil server, the Switch never connects to N servers
• support nsp, nsz, xci, xcz
• browse and install library by content, not by files: select content to be installed (which version, which DLCs) and Sphaira will only download and install what's choosen, however it is bundled (multiple nsp files, multi content nsp/xci files)

In Sphaira, select Discover local servers and pick your shop, no address to type on the console. Discovery is configured in the new Services admin page, and the Setup page now leads with Sphaira and tells you whether discovery is enabled. With Docker, discovery requires the container to use the host network, and on Windows the firewall must allow Python on private networks, see the Install doc.

Saves management as well as backuping installed content directly to your Ownfoil library is on the roadmap for a later version. A big thanks to the Sphaira maintainers for making this possible.

Metadata extracted from files - homebrew and ports support

Until now Ownfoil fully relied on blawar/titledb to get up to date titles metadata: it is an upstream dependency, and the periodic updates can stop because of eShop backend changes, leaving many of your titles without name, publisher, icon... It is also the case for homebrew, ports, some physical-only releases that cannot be scraped from the eShop.

Ownfoil now reads the metadata directly from the files: the title name, publisher, icon and display version are extracted from each content, deterministically from the title's latest update you own. Extracted values are merged with TitleDB in the locally managed metadata database introduced in 2.4.0, so a game missing from TitleDB is now identified, organized, renamed, displayed and served with its own metadata, like any other.

Self-hosted artwork

Icons, banners and screenshots were served as direct links to Nintendo's servers, your browser would contact them every time they displayed your shop. With the new Download artwork locally setting, enabled by default, Ownfoil downloads the artwork of every game in your library and its DLC, stores it on your server, and serves it to the Web UI and clients. Your shop no longer relies on external sources, and the console never connects to Nintendo's servers to browse it. That way shop browsing from a Nintendo Switch client would be fully self-hosted, not relying nor connecting to Nintendo's Server.

In the same spirit, the Web UI's front-end libraries are now served by Ownfoil instead of being loaded from CDNs, so the Web UI works fully offline.

Title details page

Clicking a game in the library now opens its own page to display the title's metadatas: banner, icon, description, publisher and release date, a screenshots viewer. Details of its content, base game, updates and DLC, with their owned/missing status are also presented. This is the first iteration, read only, and the base the editing features will be built on.

Files page

The verification results, the unidentified files, the duplicates were only visible as counts in the Stats page, with no way of knowing which files they were. There is now a Files page in the admin section, dedicated to reviewing and managing your library files. As it is the first iteration of an admin UI for files management, you can share your feedback in the discussion opened for this.

Tiles at the top count the files in each state, so you see at a glance whether anything needs attention: all files, unidentified, verification issues, duplicates, outdated updates, and files still pending processing. Files can be searched by title name, Title ID or file name, and filtered by content type, format, verification status, identification method, compression, organization and library.

The Duplicates and Outdated updates lists compare the copy that is kept against the ones that would be removed, and why it stays. Files can be deleted manually from this page; like any other operation it runs as a background task, and a file currently being processed is never deleted. The settings page also shows, beside the cleanup and verification settings, how many files each one would act on right now, with a link to the matching list.

Duplicate files handling

A library built over time usually ends up with the same content in several files: an nsp and its compressed nsz, an xci bundling a base game and an update you also have separately, a re-download in another folder. A new Delete duplicate files setting keeps the best copy of each content and deletes the others.

All copies of a content are ranked, in order: an intact file over a corrupt or modified one, a single-content file over a multi-content one (or the opposite with the new Prefer multi-content files setting), then compressed, then organized, and finally the oldest. The same ranking decides which file is served when a client downloads a content. Deletion is deliberately conservative: a content's last copy is never deleted, a multi-content file is only deleted once every content it holds has a better copy elsewhere, and nothing is judged while one of the copies is still being processed, has only been identified from its filename, or is missing from disk.

Shop name

The shop name can also now be customized in the Shop settings: it replaces Ownfoil in the navigation bar, the browser tab and the app name when the Web UI is added to a home screen, and it is given to clients.

Issues fixed

• #350 Duplicate handling - implemented as the Delete duplicate files setting, with a preference for multi-content files.
• #365 [FR] Game details popup and file verification panel in the library - implemented as the title details page and the Files page.
• #352 Results of verification - the Files page lists every file by verification status.
• #168 Physical only + homebrew titles missing - metadata is extracted from files when missing from TitleDB.
• #158 Mark files which cannot be identified, and fix them in the UI - unidentified files are listed in the Files page, and titles missing from TitleDB now get their metadata from the files.
• #136 Custom Shop name - new Shop name setting.