github Dao-AILab/flash-attention fa4-v4.0.0.beta40

pre-releaseone hour ago

What's Changed

  • [CuTe] Clamp aux_tensors indices instead of wrapping them modulo seqlen by @drisspg in #3150
  • [FA3] Check kv_batch_idx length in mha_fwd by @bhargavchintam in #3153
  • Fix FA3 out-of-bounds reads with empty KV-cache page tables by @Super262 in #3161
  • [CuTe] Return early for a paged KV cache whose page table has no column by @18853477039 in #3167
  • [CuTe] Reject a paged KV cache whose key count reaches 2**31 by @18853477039 in #3168
  • [FA2] Keep the softmax_lse, softmax_d and attention-probs offsets in 64 bits by @yunweili3 in #3169
  • [FA2] Write the LSE of empty query blocks at the unpadded offset in the split-KV kernel by @yunweili3 in #3170
  • [CuTe] Clamp huge local window sides before they overflow int32 by @18853477039 in #3173
  • [CuTe] Fall back to the in-memory JIT cache when the persistent cache dir is not writable by @hivaze in #3085
  • [CI] Restore and streamline FA4 smoke coverage by @bvolpato in #3079
  • [Tests] Preserve compiled kernels on OOM retry by @bvolpato in #3082
  • [FA3] Zero the K/V rows past seqlen_k before the GEMMs read them by @shreygupta2809 in #3174
  • [CuTe, SM100] Add non-causal native MXFP8 PV forward by @drisspg in #3020
  • [FA3][Bugfix] Compute the backward's fp32 accumulator sizes, strides and block offsets in 64 bits by @arxgy in #3152

New Contributors

Full Changelog: fa4-v4.0.0.beta39...fa4-v4.0.0.beta40

Don't miss a new flash-attention release

NewReleases is sending notifications on new releases.