Added
- pdf: Warning when a tall scanned page was read at reduced resolution and source-image strip recovery did not run or did not improve it. (#2502)
- ocr:
XBERG_TESSDATA_SYSTEM_PATHS=0(orfalse) stops tessdata resolution from searching the system Tesseract directories.
Changed
- cli: The wrapped Linux musl tarball uses mimalloc with immediate purging and process-wide transparent huge pages disabled by default, including for native libraries and inherited by child processes. The Docker CLI image and static binstall binaries keep their existing allocator. (#3171)
SetMIMALLOC_PURGE_DELAYorMIMALLOC_ALLOW_THPbefore launching to override either setting. - docker: The CLI image uses stable Alpine packages and runs as UID/GID 10001. (#3168)
- Breaking (ocr): All-page PDF OCR failures retain the typed backend source and use Rust
XbergError::Ocr. The raw FFI code changes from 1008 (Plugin) to 1002 (OCR). Python callers should catchOcrErrorwhere they previously caughtPluginError, except for a custom Python OCR backend, whose own exception is raised instead (below). REST and batch retain their previous plugin classification. (#3175) - Breaking (python): An exception raised by any custom Python OCR backend callback reaches the caller as the original exception object, including
initialize()inregister_ocr_backend,shutdown()inunregister_ocr_backendandclear_ocr_backends, and theAttributeErrorfor a missingprocess_image*method. Extraction previously raisedOtherError, orOcrErrorwhen every PDF page failed, and the registry functions raisedRuntimeError.except xberg.XbergErrorno longer catches these failures unless the backend raises anXbergErrorsubclass. Inextract_batcha failed image OCR callback is reported asocr(1002) instead ofother(1017); an all-page PDF failure keepsplugin(1008). A callback raisingxberg.CancelledErroris an ordinary page failure, not a typed cancellation: PDF OCR continues with the remaining pages. Other bindings are unchanged. (#3175) - docx: Parsing and document assembly run on Xberg’s shared native Tokio blocking pool and check for cancellation between parsing stages. (#3171)
- excel: Single and batch workbook reads use the shared native blocking pool. Timeouts abort queued reads while in-progress Calamine reads finish before releasing their resources. (#3171)
Security
- rendering: HTML, Markdown and Djot output drop links and image sources whose scheme is not
http,https,mailto,telorftp(relative, scheme-relative and fragment links are kept, as aredata:image/image sources), so ajavascript:orvbscript:link in a source document no longer becomes clickable script; the link text is kept. - install: The install script verifies CLI release archives against SHA-256 digests from GitHub release metadata before extraction. (#3168)
- ocr: Runtime Tesseract language-pack downloads are verified against pinned SHA-256 digests and use bounded timeouts. (#3168)
Fixed
- chunking:
ChunkSizing::Tokenizerwith a registered tokenizer backend whosecount_tokensincludes special tokens keeps every chunk withinmax_characterstokens; chunks were over by up to the special-token count. (#3291) - pdf: Native extraction keeps two separately outlined ruled tables with a blank gap between them apart and in page order in Plain and Markdown, unless they share both width and column count. (#3286)
- ocr: A process no longer crashes at exit when a page abandoned by the extraction time limit releases its Tesseract engine after
exit()has begun. Recognitions still in flight at exit are stopped and get up to 2 seconds to return. (#3290) - pdf: CCITT-compressed scans with
/Rows 0use the image height, so their pages render and can be read by OCR. (#3253) - ocr: PaddleOCR-VL and DeepSeek OCR reject images over their decoding budget before model loading.
DeepSeek also budgets local crop resizing, so images accepted before may exceed the limit. (#3168) - ocr: TrOCR checks decode, resize and tensor buffers before allocating pixels. Previously accepted images can exceed the
configured budget. The standalone API exposes typed image errors and accepts decoded rasters throughprocess_decoded_image.
The dimensions helper reads the header without validating pixels. (#3168) - ocr: GLM-OCR honors the configured image budget in both modes. Its standalone byte APIs use a 100 MiB decode and RGB-conversion budget. GLM decode failures now return
CandleOcrError::Imagein place ofInferenceFailed. UseGlmOcrEngine::process_rgb_image_with_taskfor decoded RGB input. (#3168) - pdf: Native extraction handles scanner PDFs that add spaces or tabs between
streamand the line ending, preserving the declared stream bytes. (#3252) - pdf: Native structured extraction keeps verified footnotes as
Footnotecontent and omits them from Plain and Markdown bodies unlessinclude_footnotesis set. (#2331) - runtime: Dynamically loaded ONNX Runtime libraries are discovered without changing the process environment. Sceptre OCR reports the native loader error if a discovered library cannot be loaded. (#3172)
- pdf: Explicit zero page margins preserve the same OCR text recovery as omitted margins. (#3172)
- ocr: In Plain output, scanned PDFs rejoin witnessed words split across eligible lines and retain lexical hyphens; Markdown already joined line-end hyphens. (#3170)
- pdf: Structured warnings stay with the document that produced them, including when fonts are shared across documents. (#3172)
- pdf: A running head whose page number is set off by spaces no longer makes a two-column page interleave its columns.
- pdf: Tick labels under a row of plot panels are no longer emitted as a Markdown table.
- rtf: No space is inserted where an inline group closes, so
12{\i 34}56extracts as123456. (#3176) - ocr: Typed backend cancellation stops PDF OCR before further page batches or fallback and returns
XbergError::Cancelled. (#3175) - pdf: Bold column labels under a single merged table title are kept as headers in HTML and Markdown. (#2332)
- pdf: Layout-assisted TATR recognizes source dividers and preserves source-supported row and column spans. (#2332)
- ocr: Default sparse-text OCR keeps a certified standalone table with one unreadable value as a table. (#2503)
- ocr: Tesseract PSM 6 preserves verified table row breaks in unified HTML for single-page PDF scans. (#2503)
- pdf: Recover sparse missing glyphs with a 120 DPI trial on eligible Tesseract pages in the default OCR pipeline. (Closes #3238)