github derekshreds/Snacks v2.2.2
Snacks v2.2.2

4 hours ago

Snacks v2.2.2

Automated Video Library Encoder

Patch release fixing cluster node state machine race conditions that caused uploads and downloads to collide on the same worker node, and several related state-tracking bugs across the dispatch, heartbeat, and recovery code paths.


Bug Fixes

Node State Machine Overhaul

  • New Uploading and Downloading node states -- the NodeStatus enum now includes Uploading and Downloading in addition to Online, Busy, Offline, Unreachable, and Paused. These states are set by the master when a file transfer begins and are owned exclusively by the transfer codepath -- the heartbeat loop cannot override them.
  • Heartbeat no longer clobbers active transfers -- previously the heartbeat would see a node as "idle" (no currentJobId reported) during a master-side upload or download and reset ActiveWorkItemId to null. The next dispatch cycle would then assign a new job to that node, causing both the upload and download to run simultaneously in a loop. The heartbeat now skips nodes in Uploading or Downloading state entirely.
  • Pause no longer overrides transfer state -- the isPaused heartbeat check was evaluated before the transfer state guard, allowing a pause during an active upload/download to clobber the node status. The transfer guard now takes priority.

Upload Lifecycle

  • _activeUploads key leak fixed -- when a work item ID was swapped to a reused database ID mid-upload, the original key was never removed from _activeUploads and the finally block removed the wrong key. Upload tracking now correctly migrates the key on ID swap.
  • Duplicate upload guard cleans up -- if _activeUploads.TryAdd rejects a duplicate, the node is now reset to Online and the work item is requeued instead of being silently dropped.
  • Node timeout during upload requeues the work item -- previously, a node timeout cancelled the upload CTS, but DispatchToNodeAsync treated timeout-triggered cancellation identically to user cancellation and did not requeue. The work item was silently lost. Timeout cancellations are now detected via node.Status == Unreachable and properly requeued.

Download Lifecycle

  • Node no longer released during download retries -- RetryDownloadAsync and DownloadOutputAsync schedule delayed retries (5s and 60s respectively), but the finally block previously released the node immediately. During the delay, dispatch could assign new work to the same node. The finally block now only releases the node when the job is fully complete (removed from _remoteJobs).
  • Early return in HandleRemoteCompletionAsync no longer leaves node stuck -- if the job was concurrently cancelled and removed from _remoteJobs, the method returned before the try/finally block, leaving the node permanently stuck in Downloading state. The early return now detects and releases stuck nodes.
  • Max download retries releases the node -- when download retries are exhausted, the node's ActiveWorkItemId and status are now properly cleared before requeuing.

Recovery Path

  • Recovery downloads set Downloading state -- RecoverRemoteJobsAsync calls HandleRemoteCompletionAsync for recovered jobs but never set the node to Downloading first, leaving the download unprotected from heartbeat interference. Recovery now sets the node state before initiating the download.
  • Recovery upload failures release the node -- all error paths in the recovery upload (duplicate guard, verification failure, exceptions) now reset the node to Online instead of leaving it stuck in Uploading.

Frontend

  • Worker node panel shows new states -- the cluster worker node UI now displays "Uploading" and "Downloading" status with the correct enum values and colors.

Files Changed

Modified Files

  • Snacks/Models/ClusterNode.cs -- added Uploading and Downloading to NodeStatus enum
  • Snacks/Services/ClusterService.cs -- heartbeat state priority, upload/download lifecycle fixes, recovery path cleanup, node timeout requeue
  • Snacks/wwwroot/js/transcoding.js -- updated NodeStatus enum mapping for new states
  • Snacks/Controllers/HomeController.cs -- version bump
  • Snacks/Services/ClusterDiscoveryService.cs -- version bump
  • electron-app/package.json -- version bump
  • README.md -- version bump

Full documentation: README.md

Don't miss a new Snacks release

NewReleases is sending notifications on new releases.