Details
CUDA: use mmvq for mul-mat-id for small batch sizes (#18958)
-
CUDA: use mmvq for mul-mat-id for small batch sizes
-
add mmvq too
-
Fix perf issue on ampere. Use mmvf mm-id only for non-nvidia GPUs
-
templatize multi_token_path
macOS/iOS:
Linux:
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.1 DLLs
- Windows x64 (Vulkan)
- Windows x64 (SYCL)
- Windows x64 (HIP)
openEuler: