3.21.0 (2026-09-12)

Features

• LlamaRankingContext: add an option to chunk large documents (#645) (a5532a9) (documentation: RankingOptions["onOverflow"])
• LlamaModel: lazyMode support (#645) (a5532a9) (documentation: LlamaModelOptions["lazyMode"])
• get model and context memory usage (#645) (a5532a9) (documentation: LlamaModel["memoryUsage"], LlamaContext["memoryUsage"])
• native jinja implementation fallback (#645) (a5532a9)

Bug Fixes

• reading gguf files that have no tensors (#645) (a5532a9)
• reading network gguf file retry conditions (#645) (a5532a9)
• add missing supported quant file name (#645) (a5532a9)
• check arch features directly from llama.cpp (#645) (a5532a9)
• align the Vulkan device reading more with llama.cpp (#645) (a5532a9)
• Metal 4 tensor API issues on M5 mac machines (#645) (a5532a9)

Shipped with llama.cpp release v0.4.0

To use the latest llama.cpp release available, run npx -n node-llama-cpp source download --release latest. (learn more)