pypi tokenizers 0.21.2
v0.21.2

latest releases: 1.0.0rc5, 1.0.0rc4, 1.0.0rc3...
17 months ago

What's Changed

This release if focused around some performance optimization, enabling broader python no gil support, and fixing some onig issues!

New Contributors

Full Changelog: v0.21.1...v0.21.2rc0

Don't miss a new tokenizers release

NewReleases is sending notifications on new releases.