flashinfer-ai/flashinfer v0.1.5

on GitHub

0.1.5 (2024-08-13)

Bugfix

• Fix PagedPrefill python api and some typos (#441) (3fff008)
• fix prefill kernels' lse result for empty kv-cache (#440) (6ac28f4)

Features

• decouple float and int workspace buffer (#442) (a7ee566)

Performance Improvements

• faster fp8->fp16 dequantization for pre sm_90 arch (#439) (c93f647)

Acknowledgement

We thank contributions and feedbacks from the community: @comaniac, @hnyls2002, @jianfei-wangg, @Yard1.