new-usemame/Calibre-Web-NextGen v4.0.59

v4.0.59 — OPDS responses no longer drop metadata fields (#184 #185)

on GitHub

OPDS responses sometimes came back with empty fields — a book in the same feed could be missing its <summary>, <author>, <dcterms:language>, or <category> while sibling books on the same page rendered correctly. Identical refresh of the same feed against an unchanged library gave different results each time. Fixed in this release; see #184 and #185 for the original reports — thanks @droM4X.

docker pull ghcr.io/new-usemame/calibre-web-nextgen:v4.0.59
docker pull ghcr.io/new-usemame/calibre-web-nextgen:latest

Fix (user-facing)

• OPDS feeds no longer drop metadata fields intermittently (closes #184, #185). Books in /opds/new, /opds/discover, /opds/author/<n>, /opds/category/<n>, /opds/series/<n>, /opds/publisher/<n>, /opds/shelf/<n>, /opds/search/<q> etc. now always carry the relationship fields they have data for — author, tag, language, comments — instead of dropping a subset per request. Same underlying fix improves Kobo sync (book metadata fields were intermittently empty), KOReader cwasync (books occasionally not matching by ISBN because the identifiers collection was empty), and the web UI's grid / detail page. Thanks @droM4X for both reports — they surfaced the same bug from different OPDS endpoints.

What was wrong

Internally, the database layer was configured to populate Books relationships via SQLAlchemy's lazy='subquery' post-load (introduced upstream PR #1279, fork PR #40 by @dbraendle, v4.0.16) — which re-runs the parent query as an inner subselect to fetch related rows. Under the OPDS request shape (two parent queries per request: random panel + main feed) with the long-lived shared database session, parent-query compile state could leak into the post-load subselect and cause it to miss a subset of parent IDs. Those books then handed off to the template with empty relationship collections.

lazy='selectin' is the modern SQLAlchemy 2.0 recommendation for collection eager loads. It emits one extra WHERE parent_id IN (?, ?, ...) per relationship, atomic, depending only on the parent primary keys — no parent-SQL coupling, no compile-state leak. Still eager, so the DetachedInstanceError case that PR #40 originally addressed remains covered.

Full SQL trace + reasoning in notes/184-185-DESIGN.md.

Verification

Reproduced before the fix on the v4.0.58 GHCR image against a 21-book test library (30 sequential /opds/new requests):

iter=1  sum=1 auth=4 cat=21   ← Crime+Punishment dropped (id 27)
iter=2  sum=1 auth=6 cat=31   ← full
iter=8  sum=0 auth=6 cat=15   ← Oz comments dropped (id 26)
iter=16 sum=1 auth=4 cat=12   ← Time Machine dropped
iter=27 sum=0 auth=4 cat=12   ← Oz + Modest Proposal partial

After the fix (same image rebuilt with the patch, same library):

• 100 sequential /opds/new requests → every response identical: sum=1 auth=6 cat=31
• 50 parallel via xargs -P 16 → all 50 returned the same hash
• Regression test in tests/integration/test_books_relationship_loading.py source-pins all nine Books relationships to lazy='selectin' and behaviorally runs the sequential + parallel stress against the in-memory test library so a future revert is caught at unit-test time

Upstream note

Upstream Calibre-Web-Automated (crocodilestick/Calibre-Web-Automated) is not affected by this exact bug because PR #1279 — which introduced lazy='subquery' — is still open and has never been merged. Anyone applying that PR will inherit the issue; a comment on that PR pointing reviewers at lazy='selectin' is queued for posting on the next outreach pass.

If you're stuck on a CWA or stock-Calibre-Web build that's serving empty OPDS metadata, the fork image above is a drop-in replacement.