flashinfer-ai/flashinfer v0.0.3

on GitHub

0.0.3 (2024-03-08)

Features

• adding sm_scale field for all attention APIs (#145) (85d4018)
• enable head_dim=256 for attention kernels (#132) (0372acc)
• pytorch api of fp8 kv-cache (#156) (66ee066)
• support ALiBi (#146) (383518b)

Misc

• add stream argument in BeginForwardFunction of TVMWrapper (#164) (fabfcb5)

Bug Fixes

• bugfix to pr 135 (#136) (3d55c71)
• fix bugs introduced in #132 (#135) (9b7b0b9)
• fix FindThrust.cmake (#161) (30fa584)

Performance Improvements

• multiple q by sm_scale in decode kernels (#144) (660c559)