KTransformers v0.7.0: Expanded MoE Fine-Tuning and Deployment
KTransformers v0.7.0 brings broader hardware compatibility and new fine-tuning capabilities for large MoE models. This release focuses on five major improvements.
Highlights
1. Full AVX512 Support for LoRA Fine-Tuning
KTransformers LoRA fine-tuning now fully supports compatible AVX512 x86 CPUs without requiring AMX.
The runtime automatically selects the appropriate CPU implementation, allowing MoE expert training to run on a broader range of large-memory servers.
kt_config:
kt_backend: autoauto is the recommended setting for both AMX and AVX512 platforms.
Related change: #2141
2. VLM Fine-Tuning Support
KTransformers now supports MoE Vision-Language Model fine-tuning, including the Qwen VLM MoE architecture.
Users can fine-tune multimodal MoE models through the existing KTransformers SFT workflow while retaining the CPU–GPU heterogeneous execution model.
Related change: #2156
3. Native FP8 LoRA Support
KTransformers now supports native FP8 LoRA fine-tuning for DeepSeek-V3.1.
The original block-wise E4M3 routed-expert weights and scales are loaded directly from the checkpoint, avoiding the need to materialize a complete BF16 copy before training.
In the tested DeepSeek-V3.1 configuration, host-memory requirements decreased from approximately 1.4 TB with BF16 expansion to about 800 GB.
Related change: #2141
4. DeepSeek V4 Docker Quick Deployment
This release adds and expands the DeepSeek V4 Flash Docker deployment guide.
Users can start a DeepSeek V4 environment through a standardized Docker workflow without manually configuring the full build and runtime stack.
5. CPU Activation Reuse
Advanced users can now retain CPU expert activations during activation checkpoint recomputation, reducing repeated CPU computation and improving training throughput.
Enable it in the training YAML when sufficient host memory is available:
kt_cpu_activation: retainLeaving this option unset preserves the default recomputation behavior. Activation Reuse trades additional CPU memory for reduced recomputation.
Related change: #2141
Release Validation
The release regression covered:
- Qwen3.5-397B-A17B BF16 LoRA on two GPUs
- DeepSeek-V3.1 native FP8 LoRA on four GPUs
- Finite training loss
- Non-zero LoRA parameter updates
- Distributed FSDP2 execution
- FP8 loading, TP staging, checkpoint, and adapter lifecycle tests
Installation
pip install "ktransformers[sft]==0.7.0"Matching SFT integration packages:
transformers-kt==5.6.0.post2accelerate-kt==1.14.0.post2
What's Changed
Fine-Tuning
- Activation Reuse and native block-FP8 routed-expert LoRA SFT by @yyj6666667
- Qwen VLM MoE fine-tuning support by @Illumination111
Deployment and Documentation
Contributors
New Contributors
- @Illumination111 made their first contribution in #2156.
Full Changelog: v0.6.4...v0.7.0