github xberg-io/xberg v1.4.2

3 hours ago

Added

  • pdf: Warning when a tall scanned page was read at reduced resolution and source-image strip recovery did not run or did not improve it. (#2502)
  • ocr: XBERG_TESSDATA_SYSTEM_PATHS=0 (or false) stops tessdata resolution from searching the system Tesseract directories.

Changed

  • cli: The wrapped Linux musl tarball uses mimalloc with immediate purging and process-wide transparent huge pages disabled by default, including for native libraries and inherited by child processes. The Docker CLI image and static binstall binaries keep their existing allocator. (#3171)
    Set MIMALLOC_PURGE_DELAY or MIMALLOC_ALLOW_THP before launching to override either setting.
  • docker: The CLI image uses stable Alpine packages and runs as UID/GID 10001. (#3168)
  • Breaking (ocr): All-page PDF OCR failures retain the typed backend source and use Rust XbergError::Ocr. The raw FFI code changes from 1008 (Plugin) to 1002 (OCR). Python callers should catch OcrError where they previously caught PluginError, except for a custom Python OCR backend, whose own exception is raised instead (below). REST and batch retain their previous plugin classification. (#3175)
  • Breaking (python): An exception raised by any custom Python OCR backend callback reaches the caller as the original exception object, including initialize() in register_ocr_backend, shutdown() in unregister_ocr_backend and clear_ocr_backends, and the AttributeError for a missing process_image* method. Extraction previously raised OtherError, or OcrError when every PDF page failed, and the registry functions raised RuntimeError. except xberg.XbergError no longer catches these failures unless the backend raises an XbergError subclass. In extract_batch a failed image OCR callback is reported as ocr (1002) instead of other (1017); an all-page PDF failure keeps plugin (1008). A callback raising xberg.CancelledError is an ordinary page failure, not a typed cancellation: PDF OCR continues with the remaining pages. Other bindings are unchanged. (#3175)
  • docx: Parsing and document assembly run on Xberg’s shared native Tokio blocking pool and check for cancellation between parsing stages. (#3171)
  • excel: Single and batch workbook reads use the shared native blocking pool. Timeouts abort queued reads while in-progress Calamine reads finish before releasing their resources. (#3171)

Security

  • rendering: HTML, Markdown and Djot output drop links and image sources whose scheme is not http, https, mailto, tel or ftp (relative, scheme-relative and fragment links are kept, as are data:image/ image sources), so a javascript: or vbscript: link in a source document no longer becomes clickable script; the link text is kept.
  • install: The install script verifies CLI release archives against SHA-256 digests from GitHub release metadata before extraction. (#3168)
  • ocr: Runtime Tesseract language-pack downloads are verified against pinned SHA-256 digests and use bounded timeouts. (#3168)

Fixed

  • chunking: ChunkSizing::Tokenizer with a registered tokenizer backend whose count_tokens includes special tokens keeps every chunk within max_characters tokens; chunks were over by up to the special-token count. (#3291)
  • pdf: Native extraction keeps two separately outlined ruled tables with a blank gap between them apart and in page order in Plain and Markdown, unless they share both width and column count. (#3286)
  • ocr: A process no longer crashes at exit when a page abandoned by the extraction time limit releases its Tesseract engine after exit() has begun. Recognitions still in flight at exit are stopped and get up to 2 seconds to return. (#3290)
  • pdf: CCITT-compressed scans with /Rows 0 use the image height, so their pages render and can be read by OCR. (#3253)
  • ocr: PaddleOCR-VL and DeepSeek OCR reject images over their decoding budget before model loading.
    DeepSeek also budgets local crop resizing, so images accepted before may exceed the limit. (#3168)
  • ocr: TrOCR checks decode, resize and tensor buffers before allocating pixels. Previously accepted images can exceed the
    configured budget. The standalone API exposes typed image errors and accepts decoded rasters through process_decoded_image.
    The dimensions helper reads the header without validating pixels. (#3168)
  • ocr: GLM-OCR honors the configured image budget in both modes. Its standalone byte APIs use a 100 MiB decode and RGB-conversion budget. GLM decode failures now return CandleOcrError::Image in place of InferenceFailed. Use GlmOcrEngine::process_rgb_image_with_task for decoded RGB input. (#3168)
  • pdf: Native extraction handles scanner PDFs that add spaces or tabs between stream and the line ending, preserving the declared stream bytes. (#3252)
  • pdf: Native structured extraction keeps verified footnotes as Footnote content and omits them from Plain and Markdown bodies unless include_footnotes is set. (#2331)
  • runtime: Dynamically loaded ONNX Runtime libraries are discovered without changing the process environment. Sceptre OCR reports the native loader error if a discovered library cannot be loaded. (#3172)
  • pdf: Explicit zero page margins preserve the same OCR text recovery as omitted margins. (#3172)
  • ocr: In Plain output, scanned PDFs rejoin witnessed words split across eligible lines and retain lexical hyphens; Markdown already joined line-end hyphens. (#3170)
  • pdf: Structured warnings stay with the document that produced them, including when fonts are shared across documents. (#3172)
  • pdf: A running head whose page number is set off by spaces no longer makes a two-column page interleave its columns.
  • pdf: Tick labels under a row of plot panels are no longer emitted as a Markdown table.
  • rtf: No space is inserted where an inline group closes, so 12{\i 34}56 extracts as 123456. (#3176)
  • ocr: Typed backend cancellation stops PDF OCR before further page batches or fallback and returns XbergError::Cancelled. (#3175)
  • pdf: Bold column labels under a single merged table title are kept as headers in HTML and Markdown. (#2332)
  • pdf: Layout-assisted TATR recognizes source dividers and preserves source-supported row and column spans. (#2332)
  • ocr: Default sparse-text OCR keeps a certified standalone table with one unreadable value as a table. (#2503)
  • ocr: Tesseract PSM 6 preserves verified table row breaks in unified HTML for single-page PDF scans. (#2503)
  • pdf: Recover sparse missing glyphs with a 120 DPI trial on eligible Tesseract pages in the default OCR pipeline. (Closes #3238)

Don't miss a new xberg release

NewReleases is sending notifications on new releases.