pypi tokenizers 0.23.3
Release v0.23.3 (python only)

5 days ago

Release v0.23.3

Python package release, other packages are unchanged

🚨 Fixs a breaking change introduced in v0.23.1

What's changed

Patch for the python package with three fixes:

  • Loosen the huggingface_hub requirements to allow huggingface_hub v2 #2486 (comment) - Unblocks transformers installs
  • Fix type hints not resolving for pyright: #2456
  • Fix a breaking change from 0.23.1 regarding truncation and overflowing: #2091

Full Changelog: v0.23.2...v0.23.3

Don't miss a new tokenizers release

NewReleases is sending notifications on new releases.