github tonikelope/megabasterd v8.33

53 minutes ago

Multi-slot download corruption fixed (#749, #748, #746, #740, #672)

This release fixes a long-standing intermittent CBC-MAC failure on multi-slot downloads with more than one slot. Single-slot (mono) downloads and multi-slot configured at exactly 1 slot were never affected; the corruption only appeared when multiple worker threads opened parallel HTTPS connections to the same MEGA storage host.

The pattern predates 8.23. It only became visible in 8.24, when the file integrity check (CBC-MAC verification) was switched on by default — earlier versions were silently producing the same broken files but never told the user.

What changed

ChunkDownloader

  • Connection: close on every chunk request. Each worker now uses a fresh TCP/TLS socket and explicitly opts out of Java's KeepAliveCache reuse. With multiple workers in parallel, the cache could hand the same idle socket to two workers in sequence with a partially-drained response in between — the next request would then read its response through a socket that still had leftover bytes in the JDK or TLS layer. Isolating sockets per slot eliminates that cross-talk.
  • flush() + getFD().sync() before closing the .tmp file, but only when the chunk was fully read. Guarantees bytes are durable on disk before the rename, so the writer thread cannot decrypt a stale-cache view.
  • Producer-side size guards: tmp_chunk_file.length() must equal the expected chunk size before the rename, and the renamed chunk_file must still match afterwards. Bad files are evicted and the chunk id is requeued.

ChunkWriterManager

  • Snapshot the chunk file's length once per iteration and use the same value for both the cipher loop and the running byte counter. The previous double-read pattern could drift the CTR-IV offset if the two reads disagreed.
  • Validate the on-disk chunk against the size MEGA's protocol mandates for that chunk id. On mismatch the file is evicted, partial progress is rolled back, the chunk id is requeued, and all workers are notified to pick it up promptly.
  • If a bad chunk cannot be deleted, abort the whole download rather than carry on with a corrupted byte stream.

Footprint

Memory is unchanged from 8.22 (~16 KiB per slot, streaming to disk). The Connection: close change costs one extra TLS handshake per chunk — negligible against 20 MiB multi-slot chunks.

Compatibility

JDK 11+. No DB migration. Settings are preserved.

Don't miss a new megabasterd release

NewReleases is sending notifications on new releases.