michaelfeil/infinity v3.0.1-beta.2

Infinity v3.0.1-beta.2

on GitHub

Adds configurable service-tier scheduling across Infinity v3 inference APIs (#834).

• Set "service_tier": "priority" in the request body to give prepared inputs a virtual queue age bonus.
• Configure the bonus with --priority-queue-bonus-ms (default: 1000 ms; 0 uses FIFO).
• Scheduling uses two FIFO queues with constant-time head selection. Downloads, tokenization, admission limits, and already-dispatched batches are unaffected.
• Cargo, OpenAPI, and publishing versions are aligned to 3.0.1-beta.2 (#836).

Published images:

• ghcr.io/michaelfeil/infinity_v3:cpu-3.0.1-beta.2
• ghcr.io/michaelfeil/infinity_v3:nvidia-3.0.1-beta.2

Validation: the scheduling PR passed CPU CI and inference smoke tests. Both release images passed startup, version reporting, and CPU inference checks before publication. GPU scheduling measurements were performed on the feature commit using the previous release's native backend; they are not validation of the final beta.2 NVIDIA image.