ggml-org/llama.cpp b12031
on GitHub

latest release: b12032

pre-releaseone hour ago

Details

opencl: add initial iq3 support (#31338)

opencl: add iq3_s AOS mv

Assisted-by: Codex

opencl: add iq3_s SOA mv and tiled mm

Assisted-by: Codex

opencl: add iq3_s noshuffle gemm/gemv for adreno

Co-authored-by: Shaofei Qi shaoqi@qti.qualcomm.com
Assisted-by: Codex

opencl: add iq3_s dp4a for adreno

Co-authored-by: Shaofei Qi shaoqi@qti.qualcomm.com
Assisted-by: Codex

opencl: optimize iq3_s adreno decode and narrow batches

Co-authored-by: Shaofei Qi shaoqi@qti.qualcomm.com
Assisted-by: Codex

opencl: add iq3_xxs AOS mv

Assisted-by: Codex

opencl: add iq3_xxs SOA mv and tiled mm

Assisted-by: Codex

opencl: add iq3_xxs noshuffle gemm/gemv for adreno

Co-authored-by: Shaofei Qi shaoqi@qti.qualcomm.com
Assisted-by: Codex

opencl: add iq3_xxs dp4a for adreno

Co-authored-by: Shaofei Qi shaoqi@qti.qualcomm.com
Assisted-by: Codex

opencl: optimize iq3_xxs adreno decode and narrow batches

Co-authored-by: Shaofei Qi shaoqi@qti.qualcomm.com
Assisted-by: Codex

Co-authored-by: Li He lih@qti.qualcomm.com

Website:

https://llama.app

Attestations:

https://github.com/ggml-org/llama.cpp/attestations/62283042

macOS/iOS:

macOS Apple Silicon (arm64)
macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED
macOS Intel (x64)
iOS XCFramework

Linux:

Android:

Windows:

openEuler:

DISABLED
openEuler x86 (310p)
openEuler x86 (910b, ACL Graph)
openEuler aarch64 (310p)
openEuler aarch64 (910b, ACL Graph)

UI:

UI

Check out latest releases or
releases around ggml-org/llama.cpp b12031

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.

Get notifications