tonikelope/megabasterd v8.37

MegaBasterd 8.37

on GitHub

TL;DR

• Fixes #736: every upload was being placed two folders deep on MEGA — Season 26/Video.mp4 ended up as MEGA/Season 26/Season 26/Video.mp4 (and with an empty upload name it produced an extra unnamed folder). The redundant inner createDir is gone; uploads now land directly under the user-chosen folder.
• Fixes silent loss of "import previous-version settings": when MegaBasterd detected an older .megabasterd<X.Y> data dir on boot and the user clicked Yes, the import was being silently discarded — the new (empty) megabasterd.db was still open and its WAL/-shm sidecars were left in place, so SQLite could replay the empty-DB WAL on top of the imported DB after restart. On Windows the copy could also fail outright with a swallowed IOException. The boot now shuts down the SQLite connection, wipes the orphan WAL/-shm before copying, and surfaces copy failures via an error dialog instead of pretending nothing happened.
• Fixes #737: adding/editing a MEGA account that uses the local #alias suffix (bob@mail.com#bob_main) failed intermittently with the "errors with some accounts" popup. check2FA had been sending the raw aliased email to MEGA's mfag endpoint since 5.81 (2019); MEGA returned -9 ENOENT and the save aborted. login()/fastLogin() already stripped the suffix — check2FA did not. Now it does.
• Hotfix for uploads broken since 8.25: every MEGA-account upload has silently failed since the v8.25 release (regression introduced by adding &v=3 to API calls; MEGA's v3 protocol returns async response shapes MegaBasterd's createDir was not built for). If you uploaded in any 8.25 – 8.34 build and it sat "preparing" forever, that's why. Fixed.
• Fixes #751: 509 retry storm, no auto-resume after VPN / IP change, slow shutdown.
• New SmartProxy resilience: 10 audit findings closed (NPE on exhausted pool, port-parse crash, race in blockProxy, transient URL failure wiping the list, static-proxy silently disabling SmartProxy, mono-slot path with no fallback, per-worker 509 state, ...).
• VPN-aware 509 recovery: workers detect public-IP change during quota backoff and resume in seconds instead of waiting out the 10 min progress watchdog.
• New DEBUG LOG tab (right of Uploads): live console of every log line on a dark monospace background. Save/Copy/Clear buttons. Replaces the old "Save debug to file" setting.
• New friendly MEGA-error popups: context-aware (link vs account), Copy-to-clipboard for GitHub issues, gated to FATAL codes only (no popup spam on transient errors).
• New Edit → Quota recovery & SmartProxy (509) dialog: tune the recovery knobs + test the live SmartProxy list with one click.

New in 8.37

Upload folder duplication (#736)

Reported by @tommyyangdev: uploading Season 26/Video.mp4 was producing MEGA/Season 26/Season 26/Video.mp4. With an empty upload-name field the result was even weirder — an unnamed wrapper folder.

MainPanelView._new_upload_dialog was creating the upload root twice. In 7.81 (commit 5eb7603) the inner folder was a sentinel literally named "MEGABASTERD" so a public share landed the recipient on a tidy single entry. In 8.14 (commit 52d5ae6, "Default upload folder name timestamp remove") that literal was replaced with root_name, which silently turned every upload into <root_name>/<root_name>/<files>.

The sentinel folder serves no purpose for non-shared uploads and confusingly duplicates the chosen name for shared ones. Drop the inner createDir entirely and point file_paths straight at parent_node. The per-file subpath loop further down still handles real subfolders from the local tree (e.g. Season 26/Episodes/), so multi-folder uploads keep working.

Old-version settings import silently dropped

On first launch of a new version, MegaBasterd detects any older .megabasterd<X.Y> data dir, moves them all into .megabasterd_old_backups/, and asks "An older version (X.Y) of MegaBasterd has been detected. Do you want to import all current settings and transfers from the previous version?". Clicking Yes was supposed to copy the previous megabasterd.db over the new one and restart. In practice the import was being silently lost. Three problems stacked:

1. By the time _check_old_version runs, MainPanel.run() has already opened the new (empty) megabasterd.db via dozens of DBTools.selectSettingValue calls. A megabasterd.db-wal and megabasterd.db-shm sidecar exist next to it.
2. The import only did Files.copy(REPLACE_EXISTING) on the .db file. The empty-DB WAL/-shm sidecars were left in place, so on next boot SQLite could replay them on top of the imported DB and silently discard the imported state.
3. On Windows the open SQLite connection can fail the REPLACE_EXISTING copy itself with an IOException. The surrounding catch logged it and moved on without telling the user. Worse, the .old_version_check marker was created at the top of the method, so the prompt never came back — the user saw "click Yes → nothing happens", forever.

Fix:

• SqliteSingleton.shutdown() before the copy so the new-DB connection is closed (file unlocked on Windows, WAL checkpointed/emptied).
• Explicitly delete the new dir's megabasterd.db-wal and megabasterd.db-shm before copying.
• Wrap the copy in its own try/catch and surface failures via a user-visible ERROR_MESSAGE JOptionPane instead of a silent log line.

Highlights (carried over from 8.36)

Uploads regression hotfix

MegaAPI._apiStdParams() no longer appends &v=3. The &ak= half (which fixed MEGA's 402 throttling) is kept. createDir falls back to the legacy sync [{node}] response shape that MegaBasterd has always understood. Diagnosed via the new DEBUG LOG tab — the raw body [[\"!Q|ama", []]]made the v=3 protocol shape obvious (echoed request token + empty action-packet array, with the actual node arriving asynchronously via thesc` channel which MegaBasterd doesn't consume).

createDir is also more defensive now: on a Jackson MismatchedInputException it logs the truncated raw body and attempts a one-level unwrap before giving up. If MEGA changes shape again we will see what they sent on the first failed upload instead of a generic stack trace.

Issue #751 (download flow)

• 509 retry storm fixed (ChunkDownloader.java:525): HTTP 509 was excluded from the exp-backoff path, so workers tight-looped on MEGA's quota responses with zero sleep. Now 509 backs off like every other transient error; the sleep is chunked into 1s slices that respect main_panel.isExit() for prompt shutdown.
• Auto-resume after VPN / IP change (ChunkDownloader.sleepWithIpAwareBackoff): worker snapshots its public IP on the first 509 of a burst, re-checks every 30s during backoff, breaks out the moment the IP changes (you activated a VPN). Goes from "wait 10 min for the watchdog" to "resume in <60s". Configurable via Settings → Quota recovery dialog.
• Quota-specific watchdog (Download.java:719): when any worker reports it is stuck on 509 specifically, the progress watchdog flips from the generic 600s timeout to the configurable 180s quota_stall_timeout. The _auto_retry_on_error flag is also set so the 3s auto-restart kicks in instead of leaving the row in FATAL ERROR.
• Slow shutdown fixed (MainPanel._byebye): the drain loop now persists the queue ONCE up front, calls RESET_CURRENT_CHUNK() on each in-flight worker (closes the HTTP socket so blocking reads return immediately instead of waiting out the 60s HTTP_READ_TIMEOUT), drains at 500ms ticks instead of 1s, and gives up after a 30s hard cap. New WarningExitMessage shows live worker/transfer counts, a JProgressBar of "workers drained", a queue-saved-to-DB indicator, and an elapsed / force-exit countdown.

SmartProxy hardening (10 fixes)

• getProxy() null check in ChunkDownloader and StreamChunkDownloader (was NPE-ing the worker when the pool was exhausted — exactly when SmartProxy is supposed to save you).
• Defensive port parse: Integer.parseInt(proxy_info[1]) is now wrapped + range-checked + auto-bans malformed entries instead of killing the worker.
• blockProxy() mutation is now atomic via ConcurrentHashMap.computeIfPresent.
• refreshProxyList() preserves the previous list on HTTP error / IOException / empty body / zero usable entries (was wiping the list silently on any blip). Connect+read timeouts (15s/30s) added so a hung URL no longer holds the SmartMegaProxyManager monitor.
• Custom-list + URL-list both configured now logs a WARNING (was silently ignoring the URL).
• Static HTTP proxy + SmartProxy both enabled now logs a WARNING at startup (ChunkDownloader.java:200 silently disables SmartProxy in that case; users could not tell).
• Multiple-@ proxy entries are now rejected with a WARNING instead of stored half-parsed.
• 509 state moved from per-worker to per-Download: when worker A hits 509, ALL workers of that download flip to SmartProxy mode in lockstep. Previously 7/8 slots kept hammering MEGA direct after slot 1 had already switched.
• ChunkDownloaderMono (streaming / single-slot path) gains the same SmartProxy fallback that the multi-slot path has had. KissVideoStreamServer streams now get the same 509 resilience.
• SMART_PROXY_RECHECK_509_TIME (was hardcoded 1h) is now configurable via DB setting smart_proxy_509_recheck_window (60s – 24h, default 1h).

MEGA error popup system

New MegaErrorMessages class maps every MEGA API error code (-1 … -26) to short name (EBLOCKED, EOVERQUOTA, ESID, ...), plain-English meaning and a suggested action. Context-aware: a -9 on an account ("email not found / credentials no longer match") gets a different message than -9 on a link ("file deleted by owner"). A "Copy to clipboard" button copies a markdown-formatted version ready to paste into a GitHub issue (version, code, source, identity, meaning, action).

Gated to FATAL_API_ERROR_CODES (-2, -4, -8, -14, -15, -16, -17, 22, 23, 24). Transient codes (-3 EAGAIN, -5 EFAILED, -18 ETEMPUNAVAIL, …) are handled by the retry/backoff machinery and dropped to FINE log instead of a modal dialog.

Hooked in: FileGrabberDialog quota check, Download.getMegaFileMetadata fatal path, Upload.initUploadFile fatal path, MainPanelView._new_upload_dialog on createDir failure.

DEBUG LOG tab

New tab right of Uploads. Dark console-style (RGB 30,30,30 / 220,220,220), monospaced, captures every java.util.logging record via a JUL Handler. Drained via a 300ms javax.swing.Timer (single append per tick, max 500 lines/tick) so high log volume does not flood the EDT. The previous attempt to also tee System.out / System.err was reverted because it created a feedback loop with ConsoleHandler during startup.

Toolbar: "Save to file..." (UTF-8 dump with timestamp filename), "Copy all" (clipboard), "Clear" (with YES/NO confirmation).

The historical "Save debug info to file" setting in Settings → Advanced is hidden — the live tab replaces it. Existing debug_file=yes rows in the settings DB are ignored.

Pre-release audit fixes

A pre-merge code review caught three real concurrency / resource issues on previously-untested paths and patched all three:

• DebugLogBus.enqueue() poll-then-add was not atomic; concurrent publishers could drift past QUEUE_MAX by N (N = concurrent threads). Now under a small synchronized block.
• MainPanel.getCachedPublicIp() held a synchronized monitor across a call that can block ~20s worst case. Switched to ReentrantLock.tryLock() — concurrent callers get the cached value instead of queuing behind the fetch. Per-source timeouts in MiscTools.getMyPublicIP reduced from 5s to 2s.
• MegaErrorMessages.showPopup() "Copied!" revert Timer is now stored + stopped on dialog close (was leaking the dialog tree until the 1.8s timer fired).

Settings additions

New DB-backed settings, exposed in Edit → Quota recovery & SmartProxy (509):

The same dialog has a live "Current public IP" indicator with Refresh, and a "Test SmartProxy list" button that TCP-probes every entry in the live pool (3s connect timeout) and reports the result.

Notes

• This release is API-compatible with 8.25 – 8.36 settings DBs. Existing debug_file=yes rows are ignored harmlessly.
• The MEGA v=3 experiment is intentionally OFF until a real sc action-packet pump is implemented. Adding &v=3 back without that pump will break uploads again — do not do it.
• Closes #736, #737, #751.