- Fixed Norwegian stopwords [@valeriansaliou, #239].
- Code has been formatted according to
clippy
recommendations. This does not change the way Sonic behaves [@pleshevskiy, #233]. - Added support for Chinese word segmentation in tokenizer (note that as this adds quite some size overhead to the final binary size, the feature
tokenizer-chinese
can be disabled when building Sonic) [@vincascm, #209].