Highlights

In this release, we add a new model architecture along with pre-trained weights, increase flexibility in our tokenizers, and improve the overall stability of the library.

• Added T5 & Flan-T5 model architecture with pre-trained weights
• Added DistilRoBERTa
• Added tutorial showing T5 in action
• Added prototype GenerationUtils

Models

Torchtext expanded its models to include both T5, Flan-T5 and DistilRoBERTa along with the corresponding pre-trained model weights. These additions represent both the smallest and largest models available in Torchtext to date as well as the first encoder/decoder model with T5. As usual, all models are Torchscriptable.

Utils

Since TorchText now has encoder/decoder models available, we prototyped a GenerationUtils for generic decoding capabilities for encoder/decoder or decoder only models.

Improvements

Features

• Add DistilRoBERTa to OSS (#1998)
• Beginning of GenerationUtils (#2011)
• Add Flan-T5 architecture (#2027)
• Optimize T5 for sequence generation (#2054)
• Add bundles for FLAN-T5 (#2061)
• Promote T5 and variants (#2064)
• Fixup generation utils for prototype release (#2065)

CI (Migrate from CircleCI to Github Actions)

• Remove CUDA binary builds (#1994)
• Remove Linux and MacOS unit tests from CircleCI (#1993)
• Validate binaries for nightly/release testing (#2010)
• Rename variable to avoid conflict with PIP system variable PIP_PREFIX (#2015, #2016)
• Refactor validation using MATRIX vars (#2021)
• Migrate validation workflows to test-infra (#2022)
• 3.11 Windows Wheels Support in CircleCI (#2053)
• Adding RC triggers for all build jobs (#2057)
• Add windows 3.11 conda (#2063)
• Channel=test for build matrix generation (#2066)
• Turn off CirclCI 3.11 unit tests (#2078)
• Fix validation workflow for test channel (#2071)
• Modify integration test workflow to use PyTorch generic CI job (#2051)

Bug Fixes

• Change read_from_tar call to load_from_tar (#1997)
• Update Multi30k test dataset hash (#2003)
• Fix device setting for T5 Model (#2007)
• Fix overwite typo (#2006)
• Fix linting error (#2019)
• Fix memory leak with C++ RegEx operator (#2024)
• Fix CodeQL workflow failure (#2046)
• Fix UTF8 decoding error in GPT2BPETokenizer decode method (#2092)

Examples

• Update T5 tutorial for 2.0 release (#2080)

Documentation

• Added min version req + readme instructions for torchdata (#2048)
• Update README w/ 3.11 (#2062)

Testing

• Replaced tabs w/ spaces to fix CodeMod (#1999)
• Add GPU testing for RoBERTa models (#2025)
• Add TorchData version to smoke tests (#2034)
• Update integration-test.yml (#2038)
• Update CUDA version on GPU test (#2040)
• Add prototype GPU tests for T5 (#2055)
• Install portalocker for testing (#2056)
• Test newly uploaded Flan-T5 weights (#2074)

Dependencies

• Add TorchData as a hard dependency (#1985)

Others

• Drop support for Python 3.7 (#2037)
• Add logo (#2050)
• Version Bumps and Update channels (#2067)