Major changes
- [Python] Supported wheels and builds for Python 3.13 and 3.14(rc0) #1134, #1127, #1121, #1111, #1110, #1104, #1103, #1099, #1091
- [Python] Added an experimental support for free-threading. #1134, #1127, #1110 https://github.com/google/sentencepiece/tree/master/python#free-threading-support
- [Python] Updated the supported Python version to 3.9 or later.
New features
- [ALL]: Added new build mode to prevent the precompiled normalization rules being embedded in *.so and *.a. (
-DSPM_DISABLE_EMBEDDED_DATA=ON). This reduces the runtime size by approximately 1-2 MB. This mode is enabled to build python wheels. The rules are loaded as the data package.
Bug fixes & minor changes
- [ALL]: Security fix to address a heap overflow issue that could occur when using a model containing an invalid precompiled normalization model.
- [Python]: Deprecates the wheel package for Linux i686.
- [Python]: Supported wheel for Windows Arm64. #1114
- [Python]: Fixed the crash issue on batch decoding #1051
- [ALL]: Updated the Unicode normalization rule with the latest ICU/Unicode rules.
- [ALL]: Unused code and build mode cleanup.