Details
model: Add support for Tiny Aya Models (#19611)
-
changes for tiny aya
-
changes to hash
-
changes to vocab
-
fix some tokenizer regex edge cases
-
update comment
-
add some comments for regex
-
Apply suggestion from @ngxson
Co-authored-by: Xuan-Son Nguyen thichthat@gmail.com
macOS/iOS:
Linux:
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.1 DLLs
- Windows x64 (Vulkan)
- Windows x64 (SYCL)
- Windows x64 (HIP)
openEuler: