huggingface/tokenizers v0.13.4.rc3

on GitHub

Mostly checking the new release scripts actually work.

What's Changed

• pyo3: update to 0.19 by @mikelui in #1322
• Add expect() for disabling truncation by @boyleconnor in #1316
• Re-using scritpts from safetensors. by @Narsil in #1328

New Contributors

• @mikelui made their first contribution in #1322

Full Changelog: v0.13.4.rc2...v0.13.4.rc3