Details
spec : add DFlash2 support (local convolution + candidate selector) (#27342) (#27816)
-
spec : add DFlash2 support (local convolution + candidate selector) (#27342)
-
support DFlash2
-
Add p_min in DFlash2
Assisted-by: Claude Opus 5
- Revert unnecessary changes
Assisted-by: Claude Opus 5
- Revert draft sampling in rejection sampling
Assisted-by: Claude Opus 5
- Refactor code structure
Assisted-by: Claude Opus 5
- Delete embedding scaling
Assisted-by: Claude Opus 5
- Gate output transforms on DFlash2
Assisted-by: Claude Opus 5
- Optimize Dflash 2 cost
Assisted-by: Claude Opus 5
- Avoid using atoi
Assisted-by: Claude Opus 5
- Modify comments
Assisted-by: Claude Opus 5
- Move llama_model_dflash_selector_top_k to llama-ext.h
Assisted-by: Claude Opus 5
- Formatting
Assisted-by: Claude Opus 5
- Apply patch to fix the mrope bug
Assisted-by: Claude Opus 5
- fix ci
Assisted-by: Claude Opus 5
- Fix graph number calculation
Assisted-by: Claude Opus 5
- rename hid and unary
Assisted-by: Claude Opus 5
Co-authored-by: Jian Chen jianchen0311@gmail.com
Co-authored-by: Xuan-Son Nguyen son@huggingface.co
- revert top-k.cu changes
Co-authored-by: Zihan Zhang tiancaizhangdaxian@sjtu.edu.cn
Co-authored-by: Jian Chen jianchen0311@gmail.com
Website:
Attestations:
macOS/iOS:
- macOS Apple Silicon (arm64)
- macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED
- macOS Intel (x64)
- iOS XCFramework
Linux:
- Ubuntu x64 (CPU)
- Ubuntu arm64 (CPU)
- Ubuntu s390x (CPU)
- Ubuntu x64 (Vulkan)
- Ubuntu arm64 (Vulkan)
- Ubuntu x64 (ROCm 7.14)
- Ubuntu x64 (OpenVINO)
- Ubuntu x64 (SYCL FP32)
- Ubuntu x64 (SYCL FP16)
Android:
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows arm64 (OpenCL Adreno)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.3 DLLs
- Windows arm64 (CUDA 13) (preview) - CUDA 13.4 DLLs
- Windows x64 (Vulkan)
- Windows x64 (OpenVINO)
- Windows x64 (SYCL)
- Windows x64 (ROCm 7.14)
openEuler:
- DISABLED
- openEuler x86 (310p)
- openEuler x86 (910b, ACL Graph)
- openEuler aarch64 (310p)
- openEuler aarch64 (910b, ACL Graph)
UI: