TL;DR
-
Fixes #776 — "Checking previous downloads/uploads, please wait..." stayed forever on the status bar. A regression introduced in 8.53 (the
THREAD_POOLcap added for #773) effectively froze every background task submitted to the global pool after the first one, including the startup resume of previous downloads and uploads, the SmartProxy refresh, the single-instance watchdog, and the MEGAcrypter reverse server. The status label that says "Checking previous..." is set on the EDT but cleared from the resume worker — and the resume worker never got to run, so the label stayed forever. Visible on a clean install with an empty queue too. No previous downloads or accounts were actually being checked — the runnable that does that work was just stuck in the queue. -
Cause: classic
ThreadPoolExecutor(0, N, LinkedBlockingQueue)antipattern.ThreadPoolExecutor.execute()only spawns a recovery worker after queuing whenworkerCountOf == 0. The first task on the pool (thewhile (!_exit)memory monitor added in 8.52 for #773) grabbed the one worker forever and every later submit enqueued behind it without ever waking another worker. Documented in the JDK source for years; same trap that bites everyone who reaches forThreadPoolExecutorinstead ofnewCachedThreadPool. -
Fix is a 5-line, byte-exact replacement of the pool config.
corePoolSize == maxPoolSize == 64,allowCoreThreadTimeOut(true), same unboundedLinkedBlockingQueue. The 64-thread RSS cap from #773 is preserved, idle workers still die after the 60 s keepAlive, and nothing else changes. -
Withdrawn binaries. 8.53 and 8.54 are defective and have been removed from the releases page. Their original changelogs are preserved at the bottom of these notes (minus the broken
THREAD_POOLsection of 8.53) so the #773 mono-buffer work and the #774 post-finish hooks are not lost from the changelog.
What was wrong
8.53 wired the global pool as:
new ThreadPoolExecutor(0, 64, 60s, new LinkedBlockingQueue<>(), factory)ThreadPoolExecutor.execute() does this:
- If
workerCountOf(c) < corePoolSize, spawn a new core worker. WithcorePoolSize=0this branch never fires. - Otherwise, offer the task to the queue. An unbounded
LinkedBlockingQueuealways accepts, so this always succeeds. - After enqueuing, if
workerCountOf(recheck) == 0, spawn a non-core recovery worker. This is the only way a worker ever gets created in this configuration — and it requiresworkerCountto already be zero.
The startup sequence runs three long-lived submits very early:
- The memory monitor (
while (!_exit) Thread.sleep(2000)). resumeDownloads().resumeUploads().
First submit: queue is empty, no workers exist, the recheck-recovery branch sees workerCount == 0, creates worker #1, worker #1 picks up the memory monitor runnable and loops forever inside while (!_exit). Worker #1 never returns to the pool.
Second submit: workerCount == 1, queue accepts the task, recheck-recovery doesn't fire because workerCount != 0. Task sits in the queue indefinitely.
Third submit: same. Plus every later submit for the entire lifetime of the process (SmartProxy refresh, watchdog accept loop, MEGAcrypter reverse, etc.).
Result for #776: the "Checking previous downloads, please wait..." label is set on the EDT inside resumeDownloads() and is cleared inside the worker's THREAD_POOL.execute(() -> { ... }) body. Since the body never runs, the label stays. Same story for "Checking previous uploads..." set inside resumeUploads().
What changes
MainPanel.THREAD_POOL is now built as:
ThreadPoolExecutor tpe = new ThreadPoolExecutor(
GLOBAL_THREAD_POOL_MAX, GLOBAL_THREAD_POOL_MAX,
60L, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(),
_megabasterdDaemonThreadFactory());
tpe.allowCoreThreadTimeOut(true);That is the canonical "grow up to N on demand, shrink back when idle" pool: new submits create workers up to the cap, idle workers die after the 60 s keepAlive, the unbounded queue still buffers anything past 64 concurrent tasks so nothing is ever rejected. Identical RSS bound as 8.53 once idle because allowCoreThreadTimeOut(true) lets all 64 slots release back to zero workers when the pool is quiet — so the #773 memory-mitigation goal is preserved end-to-end.
Audit of every other commit between 8.52 and 8.54
Done before shipping this fix to make sure nothing else regressed in the same window:
- 4651ad4 (
#773mono chunk buffer reuse). Pure local change insideChunkDownloaderMono: lifts the per-chunknew byte[chunk_size]to a worker-level scratch buffer that grows on demand. No behavioural change outside that method. Clean. - e396b14 / 6a9c0ee / 1f876f5 (
#774post-finish hooks + SettingsDialog rows + i18n keys). NewonAllTransferencesFinished()hook inTransferenceManager, plus two new config keys inSettingsDialog. The hook only fires whenfinish > 0 && pre+prov+wait+run == 0 && _isOKFinishedInQueue(), which is impossible to satisfy at startup with an empty queue. Cannot have produced the #776 symptom. Clean.
So the entire #776 symptom traces back to one commit (92a9dbe) and is fixed by reverting its pool-config pattern to one that actually creates workers.
Other
- Version bumped to 8.55 in
pom.xmlandMainPanel.VERSION. - No changes to transfer-path logic, the resume queue persistence, the account-login flow, the MEGA error popup machinery, the SmartProxy machinery, the #774 post-finish hooks, or any of the i18n bundles.
- For users who were on 8.53 or 8.54 and noticed pending downloads/uploads not resuming on startup, they will resume normally on the first launch of 8.55. The queue persistence on disk (the
downloads_queue/uploads_queueSQLite rows) was always being read correctly; the problem was that the runnable doing the reading never got scheduled.
Closes #776.
Preserved changelog from withdrawn v8.53 and v8.54
The 8.53 and 8.54 binaries have been removed from the releases page because the broken THREAD_POOL cap shipped in both. Everything below is the original changelog text from those releases, minus the broken Cap global THREAD_POOL at 64 workers section of 8.53 (which is exactly what this 8.55 release withdraws). The mono-buffer optimisation, the JVM-RAM click-to-release UI, and the #774 post-finish hooks all shipped intact in 8.55 too, so this section exists purely so the changelog history of those features is not lost when their original release entries are deleted.
Originally shipped in v8.54 (withdrawn)
#774 — Independent post-finish hooks for downloads and uploads
-
What changed.
MainPanelgrows two new pairs of fields modelled on the existing 509 command:_run_command_dl_finish+_run_command_dl_finish_path(withLAST_DL_FINISH_COMMAND_TIMESTAMP) and the upload-side equivalents. Two new public methods,MainPanel.run_dl_finish_command()andMainPanel.run_ul_finish_command(), each short-circuit when their feature flag is off and otherwise spawn a non-blockingProcessBuilder.inheritIO().start()exactly the same way the legacy 509 path has been doing for years. The actual argv-resolution body (single-arg if the configured value is a real file, whitespace-token split otherwise) lives in a single shared_spawnExternalProcess(...)helper used by all three call sites so the three commands stay byte-for-byte identical in execution semantics. -
Where the trigger lives.
TransferenceManagergains a virtual hookonAllTransferencesFinished()(default no-op) and a new latch_finish_command_firedseparate from the existing_all_finished/_tray_icon_finishlatches._genStatus()fires the hook exactly once per "non-empty → empty with at least one OK finish" transition, regardless of main-window visibility (the existing tray-notification block still requires!isVisible()and is left untouched). The latch resets together with_all_finishedwhenever a new transference enters the pipeline, so each fresh batch retriggers the command on completion.DownloadManagerandUploadManageroverride the hook to call their respectiveMainPanel.run_*_finish_command(). -
Why a separate latch and timestamp. Sharing
LAST_EXTERNAL_COMMAND_TIMESTAMP(or even a single new finish timestamp) with both queues would mean a download finish landing inside the cooldown window of an upload finish silently skips its command, which is the opposite of what #774 asks for. Each command gets its own cooldown so all three (509, downloads-finish, uploads-finish) can fire on the same tick if the timing works out. The separate_finish_command_firedlatch (one perTransferenceManagersubclass, since each side has its own instance) keeps the natural "queue must transition from non-empty to empty" gate intact without coupling to the visibility-aware tray notification. -
Why gate on
_isOKFinishedInQueue(). A queue that emptied because the user cancelled everything, or because every transfer failed with an error, almost certainly does not have files for the post-processing script to act on. Silently no-firing on all-error / all-cancelled is the safer default than running the script against an empty result set. If you want the script to fire regardless, leaving at least one successful transfer in the finished queue before clearing it satisfies the gate.
Settings dialog (Advanced tab)
- Two new checkbox-and-textbox rows appear directly below the existing "Execute this command when MEGA download limit is reached:" row:
- Execute this command when ALL downloads finish:
- Execute this command when ALL uploads finish:
- Each row has its own Test button that spawns the configured command immediately, identical to the 509 row's Test. The textbox starts disabled and is enabled by toggling the checkbox above.
- Persistence: four new SQLite settings rows (
run_command_dl_finish,run_command_dl_finish_path,run_command_ul_finish,run_command_ul_finish_path) loaded in the dialog's constructor and saved by the Save handler alongside the existingrun_command/run_command_pathkeys. No migration is needed; missing rows default to "feature disabled".
i18n
- New keys
execute_this_command_when_all_downloads_finishandexecute_this_command_when_all_uploads_finishadded to all 8 bundles. EN carries the canonical English literal, ES has the Spanish translation, and IT/DE/HU/TR/VI/ZH carry the English placeholder pending translator pickup via the #397 contributor thread.LabelTranslatorSingleton's inverse-index lookup resolves the literal inSettingsDialogto the active locale's key at component-walk time, so the new checkboxes render localised wherever the bundle has a real translation and fall through to English otherwise.
Originally shipped in v8.53 (withdrawn)
#773 — Reuse the 1 MiB in-memory buffer in ChunkDownloaderMono
- What changed. The mono (single-slot) chunk loop used to do
new byte[(int) chunk_size]inside the chunk loop, throwing away ~1 MiB ofbyte[]per chunk. The buffer is now hoisted to a worker-level local and grown on demand; after the warmup ramp (128 KiB, 256 KiB, ..., 896 KiB, then 1 MiB steady state) the array stops growing and every subsequent chunk reuses it. - Why it matters. A stable mono download processes ~60 chunks/min, so the old code translated to ~60 MB/min of pure allocation pressure per active mono download. G1 recycles young-gen fast enough that the live set stayed small, but the constant churn kept the heap commit elevated and made auto-shrink slow. The new path is exactly one allocation per worker for the entire transfer.
- Atomic-commit semantics preserved. The reason mono buffers in memory at all is the comment in
ChunkDownloaderMono.java:256-266: writing directly to the sequential output stream while reading would let a mid-chunk network failure shift every subsequent chunk and corrupt the file. The cipher stream still fills the buffer fully beforeoutput_stream.write()runs, and theMAX_CHUNK_BUFFER_BYTESguard above the allocation still applies. Only the allocation strategy changed; the safety invariant did not.
Rolled in from the unreleased 8.52 development cut
- #773 — click-to-release on the JVM-RAM status label + auto-GC hint at 70 %. The
FORCE_GARBAGE_COLLECTION_MAX_MEMORY_PERCENT = 0.7constant had been declared inMainPanelfor a long time but never read. The 2 s memory-monitor loop now requests a GC when live usage crosses 70 % of-Xmx, throttled to once per 60 s so it cannot pin a core during heavy chunk decryption. Left-clicking theJVM-RAM used: ...label fires the same hint on demand; hover for a tooltip explaining the behaviour and suggesting-Xmxfor a hard cap. The G1 collector that ships as default since Java 9 honoursSystem.gc()and, after a couple of cycles, returns free regions to the OS — which is the user-visible reduction in Task Manager that #773 was about.
Recommended launch flag for users on systems with lots of RAM
Java's default -Xmx is roughly 1/4 of physical RAM on Java 9+. On a 16 GB box the JVM happily commits up to ~4 GB even if the live working set is small. If you want a hard ceiling, launch with:
java -Xmx512m -jar MegaBasterd_8.55.jar
512 MB is plenty for everyday use; bump to -Xmx1g only if you see OutOfMemoryError in the DEBUG LOG tab under heavy parallel downloads.