Three correctness fixes to the v3.4.x restore path — the actual fix v3.4.0 needed. v3.4.0 added server-side backup/restore around the working assumption that databases can grow into the hundreds of GB; v3.4.1 tightened the UX. Neither actually worked end-to-end at the scale they were designed for. v3.4.2 fixes the real bug: the restore-extract path was loading entire zip entries into memory, OOM-killing the process on multi-GB databases. The architecture promise of v3.4.0 ("streaming everywhere, no whole-zip-in-memory") was technically true for the zip overall but false for individual entries — and the database is by definition the entry that matters most.
Fix #1 — streaming zip extraction in the restore path. The previous restore body called zf.read("aerodrome.db"), which Python's zipfile module implements as "decompress the entire entry into a single bytes object." For a 5 GB uncompressed database (which is what a year-plus retention regime produces), that's a 5 GB Python allocation in the FastAPI process. The kernel OOM-killer triggers, systemd auto-restarts the service, the client browser sees a connection-reset NetworkError, and the user is left with no data restored AND a confusing "Restore failed: NetworkError when attempting to fetch resource" message that doesn't suggest the actual cause. v3.4.2 replaces zf.read() with shutil.copyfileobj() over zf.open() — the zip entry is streamed to disk in 1 MB chunks, memory use is bounded at the chunk size regardless of entry size. Same fix-shape as the v3.4.0 upload streaming, just applied to the read side. Includes a bytes_written == uncompressed_size verification at the end so partial writes don't pass silently — if the extract stops short, the error surfaces loud with a pointer at the .pre-restore snapshot for recovery.
Fix #2 — pre-flight disk-space check against UNCOMPRESSED entry size, not the zip's compressed size. v3.4.0's disk-space check used the zip's compressed file size, which understates the actual disk requirement by the compression ratio (typically 2-3x for SQLite data with lots of repetitive ICAOs and callsigns). A 2 GB zip can extract to 5+ GB on disk; the old check would say "looks fine" and then run out of disk midway. v3.4.2's check uses zip_info.file_size (the uncompressed entry size from the zip's central directory) for the database entry specifically, plus the current DB size (the .pre-restore snapshot still on disk during the swap) plus a 256 MB safety margin. Refuses with HTTP 507 + clear message BEFORE any destructive work if insufficient.
Fix #3 — deferred service restart via FastAPI BackgroundTasks. Even with the streaming fix, the v3.4.x restore endpoints had a second bug: the post-restore service restart was triggered inline via subprocess.run("systemctl restart --no-block aerodrome") immediately before returning the success JSON response. The --no-block flag means systemctl returns to Python instantly, but systemd has already scheduled the restart and will SIGTERM the uvicorn process within milliseconds. The Python return statement assembles the response dict; uvicorn converts to bytes; the kernel writes to the TCP socket buffer; SIGTERM lands; the process dies; the kernel send buffer may or may not flush to the client depending on TCP_LINGER settings. In practice the client sees a connection-reset NetworkError mid-response — exactly the same symptom as Fix #1 above, but with a completely different root cause. v3.4.2 routes the restart through FastAPI's BackgroundTasks, which runs the callback ONLY after the response has been fully sent to the client. A 750ms sleep before the actual _do_restart() call gives the kernel TCP buffer more headroom to drain on slow links. With this change, restore responses get cleanly delivered to the browser before the service goes down, and the page reload (handled by the existing 5-second setTimeout) lands on a freshly-restarted Aerodrome with the restored state visible.
Fix #4 (defense in depth) — UI tolerance for the race in case any restart path still cuts the response. Even with the BackgroundTasks fix, there are edge cases where a response can be cut off (the user closes their laptop lid mid-response, a flaky wifi, a reverse proxy timing out). The new _handleRestoreFetchError(originalError, btnEl) helper takes over from the catch block: when a fetch fails mid-restore, it polls /api/status for up to 30 seconds. If the service comes back during that window, the helper concludes the restore probably succeeded (the server-side work usually finishes before the response is dropped), shows a success modal that explains what happened, and reloads the page. If the service stays unreachable for the full 30 seconds, the helper shows an error modal with specific diagnostic next steps (check .pre-restore snapshots, run systemctl/journalctl commands). This applies to both the browser-upload restore (fullRestore) and the server-side restore (restoreFromServerSideBackup).
Combined effect. With v3.4.2, the restore flow that surfaced this whole investigation — restoring a 5.46 GB uncompressed database from a 2.18 GB compressed backup zip — works correctly end-to-end: streaming extract keeps memory bounded, disk-space pre-check refuses if there's no room before doing any destructive work, deferred restart lets the response reach the client, and UI race-tolerance handles any remaining edge case where the response still gets cut. The user sees a normal "Restore complete — service restarting" modal followed by a reload onto the restored state.
Honest disclosure about v3.4.0/v3.4.1. Both prior versions claimed to support large-backup restore in their CHANGELOGs and UI hint text. Neither actually did — restores above ~2 GB would silently OOM-kill the process. The architecture was correct; the implementation had a critical gap in the extract path that the design conversations didn't catch and that the test data (small synthetic backups under the v3.4.0 streaming refactor) didn't exercise. The retrospective: "stream the upload" and "stream the export" both correctly streamed; "extract from the zip" was overlooked and continued to use the unsafe zf.read() pattern. Anyone running v3.4.0 or v3.4.1 should treat the server-side restore flow as untested at scale until they upgrade to v3.4.2. The export side (creating backups) was never broken — those work at any scale. Only the restore side had the OOM gap.
Why this is a patch (3.4.2) rather than a feature. No new endpoints, no schema changes, no UI surface changes the user encounters in normal operation. The fix is internal: replace one zipfile method with another (a four-line diff at the core), restructure two endpoint signatures to accept BackgroundTasks, add a UI helper for race tolerance. The user-visible improvement is "the feature works now."
Scope:
server.py— replacezf.read("aerodrome.db")with streamedshutil.copyfileobj(zf.open(...), out, length=1MB)plus uncompressed-size disk-space pre-check, partial-write verification, and a clear error path (~70 lines net). Add_do_restart_after_response(delay)helper that wraps_do_restart()with a brief sleep, intended for use with BackgroundTasks (~20 lines). Addbackground_tasks: BackgroundTasks = Noneparameter to_restore_from_local_zip,backup_import, andrestore_from_pathendpoints — passed through to defer the restart correctly.from fastapi import ... BackgroundTasksadded at module top.templates/config.html— new_handleRestoreFetchError(originalError, btnEl)helper that polls/api/statusfor up to 30s when a restore fetch fails mid-response, distinguishes "service came back → probably worked" from "service still down → real failure" (~50 lines). BothfullRestore(browser-upload restore) andrestoreFromServerSideBackup(server-side restore) updated to call the helper from their catch blocks instead of immediately showing a generic error modal.- No schema changes, no config-key changes, no sudoers changes, no API endpoint additions or removals. 88 endpoints unchanged.
Test contract for v3.4.2:
- Small-DB restore works (regression check) — KB-sized synthetic backup → restore → state matches.
- Large-DB restore works (the actual fix) — multi-GB backup → restore → memory stays bounded during extract → success response reaches the client cleanly → service restarts → restored state visible after page reload.
- Disk-space pre-check fires honestly — synthetic test where free disk is less than uncompressed entry size → HTTP 507 with clear message → no destructive work done.
- Race tolerance works — manually kill the service mid-response (e.g. SIGTERM the uvicorn worker right after the restore endpoint returns) → UI catches the network error → polls /api/status → reports service back → reloads cleanly.
Operational note: users on v3.4.0 or v3.4.1 should upgrade to v3.4.2 before relying on restore for any database larger than ~1 GB. Backups created on v3.4.x are bit-identical to v3.4.2 backups (same format, same logic), so they're forward-compatible — no need to re-create existing backups.