flashinfer-ai/flashinfer v0.0.7

on GitHub

0.0.7 (2024-06-28)

Breaking Changes

• batch_decode_with_padded_kv_cache was removed, we encourage user to use BatchDecodeWithPagedKVCacheWrapper instead. (#343)

Bugfix

• fix the forward_return_lse function in BatchPrefillWithRaggedKVCache class (#337)
• fix the scheduler behavior of large page size (#333)

Features

• customize logits_soft_cap value (#339) (a2498f5)

Performance Improvements

• change minimal kv_chunk_size back to 128 (#329) (f237f5f)
• more options for kv tile size (#336) (bf2a6c7)