Changed
- Added compatibility with Solr 10.1
Fixed
- Byte offsets of external OCR files are now correct across source-pointer region
boundaries. A region that ends inside a multi-byte character used to lose or
duplicate text and shift every following offset; the truncated character is now
indexed asU+FFFD. Offsets after supplementary characters no longer drift, and
a small read that splits a surrogate pair no longer fails indexing.
Reindex fields that useExternalUtf8ContentFilterfor the corrected
offsets to take effect, existing indexes keep the offsets they were built with.
As a nice side effect, this brings a 4-10% performance boost for indexing,
depending on the OCR.