yfedoseev/pdf_oxide v0.3.74 on GitHub

Fixed

Displayed-math tokens fused into one word — dx/dt = extracted as =dt, and whole equations could collapse into a single token (#830, #836) — the word-gap merge's backtrack check (gap ≤ font_size × 0.15) had no lower bound, so a run backtracking far behind the previous word's origin (a fraction bar returning to typeset the denominator, a relation sign closing an equation) still satisfied "a large negative gap ≤ a small positive threshold" and merged; because the merge is incremental, a chain of such backtracks could collapse an entire displayed equation — and in the worst corpus case, the start of the following sentence — into one word. The fix landed in two stages: the composed-text emitter (extract_text/to_markdown/to_html) was guarded first, then the identical guard (real baseline offset, origin-or-left backtrack, multi-em overlap, gated off for RTL) was applied to extract_words_inner's post-clustering merge so word geometry is correct too. Because table detection consumes word geometry, the detector was hardened in the same change so the corrected words no longer fabricate phantom tables out of ordinary wrapped captions.
Subscript index numbers extracted as decimals — P₁,₀ became P1.0 (#816) — the decimal-merge rule joins two adjacent pure-digit runs with a . (a heuristic for split-box dollar amounts where the whole part and cents print in separate fixed-width boxes, 123456 + 72 → 123456.72). Its upper gap bound was too permissive: real split-box amounts sit ~0.8–1.0× the font size apart, but subscript index digits are a smaller font spaced ~1.5–1.7× apart, so the old 2.0× ceiling let the rule invent decimals the document never contained. The ceiling is tightened to 1.3× the font size, separating genuine integer/cents boxes from widely spaced subscripts.
Born-digital pages were classified as Scanned and routed to pages_needing_ocr — 13.7% of a 6,269-page corpus (#840) — OCR-ing a page that already carries good native text replaces it with worse output, so a wrong Scanned verdict is actively harmful. The dominant cause: gather_page_signals and a second text_quality_gate call site both built their word-fragmentation input by joining raw content-stream spans with a forced space after each one. Math typesetting draws each atom — a parenthesis, an operator, a subscript — as its own span, so (∞) became three one-character "words"; on a dense LaTeX page this inflated the fragmented-word ratio and collapsed average word length until the quality gate mistook it for a scan and overrode an otherwise-correct TextLayer verdict. Both call sites now build their word list from extract_words — the same glyph/span clustering extract_text relies on, including the new math-backtrack guard — instead of one token per span.
extract_words split single string literals into fragments (module → m|odu|le) via phantom glyph gaps (#811) — TJ-offset space spans (ISO 32000-1 §9.4.4) were created with one char but an empty char_widths, and the span merge kept the widths in lockstep by tail-append + tail-resize. Whenever a width-less span contributed chars anywhere but the tail, every subsequent width shifted one slot, so per-glyph decomposition paired each glyph's accurate x-origin with its neighbor's nominal width — phantom ~0.3 em intra-word gaps that the word-gap clusterer split on. Space spans now carry their advance from creation, and the merge normalizes every contribution at its own position — inserted separators get the real geometric gap they stand in for.
PathContent geometry ignored stroke_width, so stroke-width-encoded table rules extracted as 1×0 pt specks (#812) — print-era generators draw a table's vertical rule as a ~1 pt segment stroked as wide as the table is tall (430 w … 0 0 m .998 0 l S). The geometric bbox of that path bears no resemblance to the rendered bar, so is_table_primitive() and the line-based table detector missed the whole grid and its text extracted column-major. stroke_width is now CTM-scaled at extraction (§8.4.3.2 — the line width transforms like all other geometry), the new PathContent::rendered_bbox() exposes the stroke-inflated extents (exact perpendicular + cap inflation for straight segments, conservative half-width outset otherwise), and line classification, clustering, and the per-row/column separator checks all judge rendered extents. The geometric bbox is unchanged for every other consumer. Also exposed as pdf_oxide_path_get_rendered_bbox (C FFI) and rendered_bbox in the Python/WASM path dicts, and threaded through the go, ruby, php, swift, csharp, dart, elixir, zig, julia, r, objc, cpp, and node bindings.
90°-rotated pages extracted in portrait order and words carried no rotation metadata (#813) — landscape tables typeset on portrait pages (text-matrix rotation, no /Rotate key) came out as interleaved word salad: the reading-order pipeline re-sorted spans with portrait-frame comparators, the plain-text assembler grouped lines in the portrait frame, and rotation_degrees was dropped at both TextSpan::to_chars and word assembly. A dominant-rotation vote (half-or-more of the page's non-whitespace spans sharing one quadrant rotation, mirroring the tategaki vote) now orders the whole page in its rotated reading frame — coordinates are restored afterwards, so callers keep true page space — and minority rotated runs (margin stamps, figure labels) are ordered upright per rotation group and appended after the horizontal flow, matching the span path's existing firewall. Runs sharing a ±90° rotation no longer span-merge across rotated lines. rotation_degrees now flows span → char → Word and is exposed on Word (Rust/serde), PyWord, the WASM word JSON, and pdf_oxide_word_get_rotation (C FFI), and surfaced on the word type of the go, ruby, php, swift, csharp, dart, elixir, zig, julia, r, objc, cpp, and node bindings, plus the JVM TextWord (Java, inherited by the Kotlin/Scala/Clojure wrappers).
Scanned Hebrew/Arabic OCR text layers extracted reversed — every word both letter- and word-order-reversed (#826) — scanned RTL PDFs whose invisible OCR text layer emits one TJ array per recognized word (the standard OCR-sandwich shape, e.g. Tesseract-style producers) had two compounding bugs in the Tj/TJ buffer-flush path. flush_tj_buffer (the default WordBoundaryMode::Tiebreaker path) never received the confidence-gated geometric direction detector, so it still used the old accumulated_width > 0.0 heuristic — true for nearly every non-empty RTL buffer — and reversed unconditionally instead of detecting direction; all three flush sites now route through one shared decision point (bidi::apply_rtl_verdict). And because already-logical invisible-OCR text and genuinely visual-order text have identical geometric signatures, text render mode is now threaded through so invisible runs (Tr 3/7) skip the geometric heuristics entirely and trust extraction order as-is.
FluentPageBuilder::rich_paragraph drew consecutive TextRuns flush together — TextRun::bold("Text Run 1") + TextRun::normal("Text Run 2") extracted as Text Run 1Text Run 2 (#837) — each run word-wraps and emits its own text, then advances cursor_x by exactly the emitted width, with nothing separating one run's end from the next's start, so a run boundary falling mid-line drew the next run against the previous one. Consecutive runs on the same line are now separated.
Stacked two-line column/table-header cells fused into one token — Comparison over rate extracted as Comparisonrate (#847) — when the structure-tree (tagged-content) assembler linearizes a header cell drawn as two stacked rows, the rows arrive as consecutive spans that horizontally overlap (negative gap) at a baseline drop sitting just under the same-line threshold, so the assembler treats them as one line and defers to the space decision — which, seeing a negative gap, returned no space and glued them. A negative gap combined with a genuine baseline shift is two stacked tokens, never intra-word kerning (which shares a baseline), so a separator is now inserted. Scoped to the tagged/structure-tree path so main-flow inputs (e.g. LaTeX math fraction stacks, already handled by dedicated line-break branches) stay byte-identical; a 419-PDF sweep confirmed the change is isolated to tagged tables/forms with only glyph-preserving spacing gains.
Per-glyph advance drifted behind the true rendered position on kerned/justified text, manufacturing phantom inter-glyph gaps (#847) — a sub-threshold TJ positioning number (ISO 32000-1 §9.4.4) advanced the text matrix but was dropped from the run's stored per-glyph advance (char_widths/accumulated width), so on a line drawn as one continuous buffer the many small post-space kerning offsets accumulated into a multi-point undershoot: the reconstructed glyph positions fell behind where the glyphs actually render. Poppler/PDFium/pymupdf all agree on the true position because they fold the offset into the advance; pdf_oxide was the sole outlier (−2.3 pt over one measured line, concentrated at word gaps). The stored advance now folds the exact §9.4.4 displacement — −Tj/1000 × Tfs × Th — into the run, so per-glyph geometry equals the text-matrix position by construction (closing ~72% of the drift on the worst case; the residual is the /Widths-vs-substitute-font-metric difference, a separate axis). This is a generic positioning fix, not a heuristic — it is the same advance the renderer uses — and it is what lets the narrow-word-gap rescue below operate on true gaps instead of phantom ones (a phantom ~0.15 em gap is what previously over-split matched → match ed, forcing this rescue to be held back). A companion guard tightens the cross-font single-letter glue ceiling from 0.25 em to 0.12 em: 0.25 em is a full word space, so a word followed by a single-letter variable set in a different font run (roman solution → math-italic U) was wrongly glued into solutionU; drop-caps and small-caps initials — the glue's real target — sit tight against their word at ~0 em, so 0.12 em keeps them while releasing genuine word→variable boundaries (poppler and PDFium keep the space).
Condensed headings and tracked runs typeset with no space glyph fused adjacent words — conformance test plans → conformancetestplans (#847) — a bold heading or a running header whose word separation is pure Td/TJ positioning (no 0x20 glyph) opens inter-word gaps of only ~0.18 em, below the intra-word kerning guard (0.75× the space-glyph advance), so the words glued. Because it now runs on the accurate per-glyph advance above, the gap distribution reflects the real render rather than the old undershoot. A fixed magnitude can't separate a 0.18 em word gap from ~0.15 em kerning — but within one line the intra-word glyph gaps cluster near zero while the inter-word gaps form a distinct larger cluster. A per-line multi-level bimodal split of the gap distribution now pins the word boundary regardless of absolute magnitude — splitting at every gap level above the intra-word cluster, so a condensed running footer (© ISO 2021 – All rights reserved) recovers its ~0.10 em word gaps too, matching the advance-aware extractors (pdfminer, poppler, Adobe Acrobat) that pymupdf/pdfplumber miss. It only ever adds a space, only when the suppression came from the geometric kerning guard (a new SpaceSource::IntraWordKerning marker) — never the semantic no-space rules, so complex-script text (Devanagari, Bengali, …), CJK, ligatures, and RTL are untouched. Two guards keep it off dense math, whose sub/superscript gaps are the same ~0.10 em magnitude: it never fires across a super/subscript baseline shift, nor when another glyph's ink occupies the gap (a subscript drawn between a variable and the next symbol, λᵢr → keeps λᵢr, never λ i r). 419-PDF sweep: glyph-preserving spacing gains on headings/footers/condensed runs, zero fusions, zero over-segmentation of math or complex scripts. (A word boundary whose two glyphs overlap — negative advance, e.g. the rights reserved seam — carries no geometric signal and is recovered by no extractor, Adobe included.)

Security

Bumped crossbeam-epoch to 0.9.20 (RUSTSEC-2026-0204) (#827).

Internal

Pinned the Go toolchain to 1.26.5 in CI (GO-2026-5856) (#834).
Consolidated the July 2026 Dependabot cargo + github-actions updates (#835).
Added tests that remove_footers preserves body content (#800).
Fixed the broken --all-features test commands in the PR template and dev guide (#838).
Bumped office_oxide to 0.1.6.

Contributors

Community fixes merged this release:

@tobocop2 — reported and submitted the fixes for the fragmented-word, stroke-encoded table-rule, and rotated-page bugs (#811, #812, #813 → #814), the subscript-decimal bug (#816 → #817), and the displayed-math relation-sign fusion (#830 → #831); also reported the word-layer math fusion (#836) and the born-digital misclassification (#840). A standout contribution across the whole release.
@ultrasaurus (Sarah Allen) — contributed the remove_footers content-preservation tests (#800).

Issues reported by:

@tobocop2 — #811, #812, #813, #816, #830, #836, #840
@RubberDuckShobe — #837 (rich_paragraph run spacing)
@palmoni5 — #826 (Hebrew OCR-sandwich reversal)
@Goldziher (Na'aman Hirschfeld) — #847 (word fusion on positioned runs)

Thank you all — reporters and fixers alike.

Installation

Rust (crates.io)

cargo add pdf_oxide

Python (PyPI)

pip install pdf_oxide

JavaScript/WASM (npm)

npm install pdf-oxide-wasm

CLI (Homebrew)

brew install yfedoseev/tap/pdf-oxide

CLI (Scoop — Windows)

scoop bucket add pdf-oxide https://github.com/yfedoseev/scoop-pdf-oxide
scoop install pdf-oxide

CLI (Shell installer)

curl -fsSL https://raw.githubusercontent.com/yfedoseev/pdf_oxide/main/install.sh | sh

CLI (cargo-binstall)

cargo binstall pdf_oxide_cli

MCP Server (for AI assistants)

cargo install pdf_oxide_mcp

Pre-built Binaries
Download archives for Linux, macOS, and Windows from the assets below. Each archive includes both pdf-oxide (CLI) and pdf-oxide-mcp (MCP server).

Platform Support

Platform	Architecture	Archive
Linux	x86_64 (glibc)	`pdf_oxide-linux-x86_64-*.tar.gz`
Linux	x86_64 (musl)	`pdf_oxide-linux-x86_64-musl-*.tar.gz`
Linux	ARM64	`pdf_oxide-linux-aarch64-*.tar.gz`
macOS	x86_64 (Intel)	`pdf_oxide-macos-x86_64-*.tar.gz`
macOS	ARM64 (Apple Silicon)	`pdf_oxide-macos-aarch64-*.tar.gz`
Windows	x86_64	`pdf_oxide-windows-x86_64-*.zip`

Changelog

See CHANGELOG.md for full details.