Current status

As of September 2023 we have paused active development of TorchText because our focus has shifted away from building out this library offering. We will continue to release new versions but do not anticipate any new feature development as we figure out future investments in this space.

Bug Fixes

• Update links to multi30k dataset since original servers are down (#2194)
• Use filelock to block on concurrent model downloads (#2166)

New Features

• Add support for __contains__ for Vectors class (#2144)
• Add generation utility support to T5Bundle (#2146)
• Add option to ignore UTF-8 decoding error to scripted tokenizer (#2134)
• Add shift-right method to T5 model (#2131)
• Add XLMR and RoBERTa transforms as factory functions (#2102)
• Make sure to include padding mask in generation (#2096)
• (Prototype) Add top-p and top-k sampling (#2137)