facebookresearch/xformers v0.0.11

on GitHub

[0.0.11] - 2022-05-30

Fixed

• Fix some torchscriptability [#246]
• Fix FourierMix being compatible with AMP [#258]
• Better asserts on QKV dimensions [#264]
• Better perfs for FusedMLP and FusedLinearLayer [#283]
• Deepnorm init missing self-attention [#284]

Added

• Simplicial Embeddings [#259]
• Mem efficient attention, FW pass [#267]
• MHA benchmark
• MLP benchmark
• Move all triton kernels to triton v2 [#272]
• Mem efficient attention, BW pass [#281]
• Metaformer support [#294]