Romelium/mpatch v1.7.0

on GitHub

Overview

1. Format-Agnostic LLM Ingestion & Wildcards: Native parsing for Aider search/replace blocks with automatic path inference and multi-line wildcard ellipsis (...) matching that safely bridges omitted code gaps.
2. Transaction-Grade Atomicity: All-or-nothing filesystem guarantees via -a / --atomic across CLI, Rust API, and Python bindings, staging edits in-memory and rolling back cleanly on any failure.
3. Three-Way Merging & Sub-Line Diagnostics: Built-in 3-way line merges with Diff3 conflict markers (merge_three_way), sub-line word-level diff visualization (format_inline_diff), and fuzzy file path suggestions (suggest_close_file_paths) for missing targets.
4. Unix & Stdin Composability: First-class standard input piping (git diff | mpatch - ./src), implicit piping, and automated fallback to the current directory (.).
5. $O(N + M)$ Scalability & Performance: Single-pass inverted index anchor discovery, Histogram-based sequence comparisons, incumbent-seeded upper-bound diff pruning, zero-allocation window length search, and zero-allocation precomputations eliminating quadratic bottlenecks on files with >10,000 lines.
6. Anti-Hallucination & Scope Protection: Strict boundary validation including orphan addition guards, definition-aware head/tail context preservation, primary identifier and definition consistency across C-family, JVM, scripting, Lisp, and SQL syntax, and topological anchor relaxation for unnumbered hunks.

Added

Parser & Formats

• Aider Search/Replace Blocks: Added native support for Aider search/replace blocks (<<<<<<< SEARCH, =======, >>>>>>> REPLACE along with ORIGINAL and UPDATED syntax variants) via mpatch::parse_aider, mpatch::parse_aider_from_lines, and PatchFormat::Aider. Fences are matched by indentation, so indented fences inside replacement content (such as nested code examples) do not end a block early.
• Automatic Format Detection: Updated detect_patch, parse_auto, and parse_diffs to recognize and parse Aider search/replace blocks alongside unified diffs, markdown code fences, and conflict markers.
• Deduplicated Patch Merging: Added mpatch::merge_patches and Patch::merge to consolidate multiple patch blocks targeting the same file path into single Patch objects with combined hunks in linear $O(N)$ time.
• Heuristic Target File Path Detection: Implemented extract_file_path_from_line, is_plausible_file_path, and normalize_candidate_path to infer target file paths for Aider search/replace blocks from preceding markdown text, headings, code comments (// filepath:), backticks, and search fence headers. A path named in a heading stays in scope across long explanations, illustrative non-patch code blocks, and consecutive patch blocks until a #/## heading without a path or a thematic break (---, ***) starts a new section. Conflict marker blocks still default to patch_target.
• File Path Plausibility Rules: is_plausible_file_path accepts quoted or otherwise delimited paths, paths containing spaces or Unicode, route templates ({id}), extensions such as .c++, extensionless build files (BUILD, WORKSPACE, configure, VERSION), and paths up to 4,096 bytes, while rejecting URLs, code syntax, and conversational sentences.
• Wildcard & Ellipsis Matching: Added support for wildcard ellipsis lines (..., …, // ... existing code ..., <!-- ... -->, # ... rest of function ..., Fortran ! ..., Batch :: ..., etc.) across single and multi-segment hunks via is_ellipsis_line. Multi-line code gaps between anchors are accurately preserved while strictly guarding against runaway gaps or syntax false positives. Segments may end on low-entropy anchors such as }, end, or pass; a wildcard hunk is rejected only when its leading segment or every segment is low-entropy.

Core API

• Patch Methods: Added Patch::apply_to_file and Patch::apply_to_content as method forms of apply_patch_to_file and apply_patch_to_content.
• Strict Batch Application: Added try_apply_patches_to_dir, a non-atomic strict variant of apply_patches_to_dir that returns StrictBatchApplyError::Failed if any patch has a hard error or any hunk fails.
• Candidate Inspection: Added DefaultHunkFinder::find_candidate_locations to list every candidate location for a hunk in order of preference, and Hunk::required_match_span to compute the minimum window span that can contain all of a hunk's edits.
• Comment Line Detection: Added mpatch::is_comment_line to classify pure comment, docstring, and banner lines across C-family, scripting, markup, Lisp, SQL, Fortran, and Batch syntaxes, while treating pointer dereferences, preprocessor directives, attributes, pre-decrements, and CSS selectors as code.
• Zero-Allocation Window Length Iterator: Added mpatch::WindowLengthIter and mpatch::window_lengths implementing Iterator, ExactSizeIterator, and FusedIterator to generate candidate window lengths radiating outward from nominal length without heap allocations.

Transactional Atomicity

• Core Atomic APIs: Added atomic filesystem application via apply_patches_to_dir_atomic, try_apply_patches_to_dir_atomic, apply_patch_to_file_atomic, try_apply_patch_to_file_atomic, and Patch::apply_to_file_atomic. File operations are staged in-memory and committed to disk if and only if all hunks across all patches apply cleanly, with automatic rollback of written files upon commit failure.
• Batch Outcome Inspection: Added BatchResult::all_applied_cleanly(), BatchResult::has_failures(), and StrictBatchApplyError for granular programmatic inspection of batch patch outcomes.
• CLI Atomic Flag: Introduced the -a / --atomic (aliased as --all-or-nothing) command-line flag to ensure that changes are only written to disk if every patch and hunk succeeds.

Three-Way Merging & Diagnostics

• 3-Way Line Merge: Added mpatch::merge_three_way backed by similar::TextMerge to perform 3-way line merges between an ancestor (base), local code (ours), and incoming changes (theirs), with configurable Diff3 conflict marker labels.
• Inline Word-Level Diffs: Added mpatch::format_inline_diff powered by similar's inline feature to format sub-line word additions and deletions with colored terminal highlights.
• CLI Near-Miss Diagnostics: Failed hunk diagnostics in the CLI now render sub-line near-miss comparisons with highlighted word differences when encountering FuzzyMatchBelowThreshold errors.
• CLI Path Suggestions: Added mpatch::suggest_close_file_paths powered by similar::get_close_matches to suggest close matching file paths in the target directory when a patch specifies a missing target (TargetNotFound).

CLI & Unix Composability

• Standard Input (Stdin) Piping: Added first-class support for piping patches via - (e.g., cat patch.diff | mpatch - ./src, git diff | mpatch -R - ./src, curl -s https://example.com/fix.patch | mpatch - ./src), including implicit piping and automatic fallback to the current directory (.) when the target directory argument is omitted.

Python Bindings (mpatch)

• Aider Parsing: Exposed mpatch.parse_aider and updated detect_patch and type stubs (mpatch.pyi) for Aider search/replace blocks.
• Atomic Operations: Added the atomic: bool = False keyword parameter to apply_directory, apply_patches_to_dir, apply_patch_to_file, and Patch.apply_to_file, alongside dedicated apply_patch_to_file_atomic, apply_patches_to_dir_atomic, and Patch.apply_to_file_atomic functions.
• Batch Outcome Inspection: Added the BatchResult.all_applied_cleanly and BatchResult.has_failures properties, and made BatchResult path lookups (result[path], path in result) accept either / or \ separators.
• Deduplicated Patch Merging: Exposed mpatch.merge_patches and Patch.merge.
• Merge & Inline Functions: Exposed mpatch.merge_three_way and mpatch.format_inline_diff with comprehensive type annotations (mpatch.pyi).
• API Expansion: Exposed mpatch.parse_single_patch, mpatch.suggest_close_file_paths, mpatch.find_hunk_location, Hunk.find_location, Hunk.required_match_span, and mpatch.ensure_path_is_safe.
• Path & Wildcard Inspection Utilities: Exposed mpatch.is_ellipsis_line, mpatch.is_plausible_file_path, and mpatch.extract_file_path_from_line.
• Near-Miss Diagnostics: Added HunkFailure.location_start and HunkFailure.location_length to inspect rejected fuzzy near-miss match coordinates.

Tooling & Release Automation

• Release Automation: Added scripts/release.py featuring interactive pre-flight toolchain diagnostics, automated testing across Rust and Python, lockfile synchronization, atomic git commit/tagging, and optional Crates.io publishing.
• Benchmark Suites: Added Criterion benchmarks covering Aider format detection and parsing, interleaved patch merging, a 200-function 3-way merge, 100-line inline diff rendering, Patience diff patch generation, and radiating window length iteration (benches/mpatch_bench.rs).

Changed

• Diff Engine & Dependencies: Upgraded similar from 2.7.0 to 3.2.0 with the inline feature enabled.
• Error Handling & Styling Dependencies: Upgraded thiserror from 1.0.69 to 2.0.21 and colored from 2.2.0 to 3.1.1.
• Patch Generation Algorithm: Switched Patch::from_texts and internal search/replace hunk alignment to similar::Algorithm::Patience for cleaner, semantically aligned hunk boundaries across refactored functions and moved blocks.
• Same-File Patch Consolidation: parse_diffs, and therefore parse_auto and the CLI on Markdown input, now merges patches that target the same file across separate code blocks into a single Patch with combined hunks. Previously each code block produced its own Patch.
• Fuzzy Scoring Limits: Hunks longer than 80 lines are now scored on line-level similarity only, and shorter hunks are scored at the word level only when their line-level ratio plus a size-scaled allowance (1/sqrt(len)) could reach the fuzz threshold. Hunks with heavy in-line drift, such as an identifier renamed on half the lines of a 25-line function or on a third of the lines of a 100-line function, can now fall below the threshold where 1.6.4 matched them.
• Logging: Expanded trace and debug logging across parsing, path inference, anchor search, window scoring, candidate backtracking, and atomic application.
• Documentation: Extensively updated root and Python README.md files with guides and examples for atomic application, Aider search/replace with wildcard ellipsis matching, 3-way merges, inline word diffs, and stdin piping. Every README example is now exercised by Rust and Python integration tests.

Fixed

• Unified Diff Parsing & Exact Application: Recognize Git format-patch version footers after completed hunks, and preserve whitespace-only context lines verbatim during exact application.

Performance

• Histogram Sequence Comparisons: Switched word-level sequence comparisons in score_window, find_statement_match_in_block, and try_apply_hunk_at_location to similar::Algorithm::Histogram, eliminating quadratic Myers degradation on repetitive code.
• Linear-Time Patch Deduplication: Replaced the $O(N^2)$ same-file merge loop in parse_patches_from_lines with merge_patches, which uses index-mapped tracking and fast paths for single or empty patch sets; parse_aider_from_lines and parse_diffs use the same linear-time merge.
• $O(N + M)$ Single-Pass Inverted Indexing: Implemented single-pass inverted indexing for candidate anchor discovery, evaluating up to 100 high-entropy lines across hunks in $O(1)$ lookups instead of repetitive $O(N)$ linear scans.
• Upper-Bound Pruning: Added upper-bound pruning and lazy metric evaluation in score_window to skip expensive character- and word-level diff evaluations when line-level similarity plus the size-scaled word allowance cannot reach the fuzz threshold.
• Incumbent-Seeded Window Search: seed_best_score exactly scores the four most promising start positions before the main search, letting WindowScorer::score_candidate prune any window whose prefix-sum upper bound cannot beat that incumbent score and avoiding redundant line diffs on large hunks.
• Zero-Allocation Window Length Search: Refactored candidate window length exploration in WindowScorer::score_candidate radiating from nominal hunk length to use the zero-allocation WindowLengthIter, eliminating thousands of transient Vec heap allocations during sliding window fuzzy search.
• Tiered Fallback Search Architecture: Introduced a tiered fallback search architecture that evaluates narrow anchored ranges first and cascades to full-file search only if an anchor collision fails to reach the threshold, eliminating multi-second freezes on large hunks (>500 lines) in large files (>10,000 lines).
• Zero-Allocation Target Precomputations: Precomputed contiguous target slices, stripped string references, and character byte offsets in precompute_target and precompute_match, eliminating four heap allocations per candidate window.
• Short-Circuited Anchor Search: Short-circuited anchor occurrence search in find_search_ranges as soon as the maximum occurrence threshold is exceeded, avoiding full linear scans and unbounded vector allocations on repetitive target files.
• Zero-Copy Memory Reuse: Eliminated redundant target slice cloning and heap string allocations in try_apply_hunk_at_location by referencing target_lines directly and borrowing hunk addition slices.
• Backtracking Allocation Elimination: Eliminated redundant target_lines.clone() calls during candidate window backtracking in apply_hunk_to_lines.
• Buffer Pre-allocations: Replaced dynamic format! allocations with pre-allocated buffer pushes in Patch::from_texts, Hunk::invert, Hunk::required_match_span, and conflict marker parsing.
• In-Place Result Counting: Optimized ApplyResult::failure_count(), success_count(), and has_failures() to count in-place with zero heap allocations or error cloning.
• Linear-Time Discrepancy Check: Replaced $O(N^2)$ multiset subtraction in format_normalized_patch with a sorted $O(N)$ two-pointer pass.
• Lazy Python Indexing: Optimized ApplyResult.__getitem__ to convert individual hunk status items on demand rather than translating the entire status vector per indexed lookup.

Security

• Orphan Addition Guard: Fixed a critical bug in fuzzy reconstruction where additions attached to unchanged context lines that were missing or unaligned in the target file window (DiffOp::Delete and DiffOp::Replace) were blindly injected into preceding code. The applier now aborts with ContextNotFound to prevent syntax corruption.
• Head & Tail Context Preservation: Fuzzy reconstruction now rejects candidate windows (ContextNotFound) when missing or unaligned outer context contains a function or type definition, or when all substantive head or tail context is lost, preventing additions or modifications from being injected into wrong or unauthenticated functions. Minor drift is tolerated: stale outer lines such as a deleted doc comment ahead of a matched signature are skipped, and trailing comment lines may drift when every removal line is verified and the head context matched.
• Scope Delimiter Protection: Enforced primary identifier matching (extract_primary_identifier) and definition status consistency (is_definition) in statement and replacement matching (raising the statement threshold to 0.80), preventing fuzzy matches from crossing function boundaries or hijacking unrelated API calls. Definitions and calls are recognized across Rust, Python, Go, JavaScript/TypeScript, C/C++, Java, C#, Ruby, Lua, Shell, Lisp/Clojure, and SQL DDL, while YAML and JSON keys, assignments, preprocessor directives, SQL queries, CSS at-rules, and Markdown prose are not mistaken for definitions or calls.
• Low-Entropy Ambiguity Rejection: Fixed a bug where hunks with low-entropy match blocks (such as a single closing brace }) were incorrectly tie-broken using line-number hints across multiple ambiguous locations. Ambiguous low-entropy contexts, including bare block terminators such as end, fi, done, esac, and pass, are now cleanly rejected.

Fixed

• Zero-Context Addition Hunks: Pure-addition hunks with no context lines (such as @@ -N,0 +N+1,M @@ from git diff -U0, or appends at EOF) now apply to existing files, inserting at the hinted line or appending at EOF instead of failing with ContextNotFound. Line-0 additions into a non-empty file are still rejected as file-creation conflicts.
• Mixed Indentation in Additions: When a patch's context lines are unindented but the target block is indented, smart indentation now converts space-indented additions to hard tabs for tab-indented targets (keeping Makefile recipes valid) and tab-indented additions to spaces for space-indented targets, instead of prepending the target's indentation (which produced lines such as \t rm -f app.bin).
• Topological Anchor Interval Bounding: Implemented topological anchor interval bounding and cascading relaxation (resolve_hunk_line_hints) for patches lacking explicit line numbers (such as Aider search/replace blocks). Unambiguous hunks act as spatial anchors to soundly disambiguate intermediate hunks containing identical or repetitive code.
• Candidate Backtracking & Lenient Reconciliation: Implemented candidate location backtracking in apply_hunk_to_lines, testing all candidate windows strictly before falling back to lenient trailing delimiter reconciliation (normalize_line_delimiters) and re-anchoring additions on missing blank context lines.
• Semantic Statement Matching Across Line Breaks: Added statement matching across line breaks (find_statement_match_in_block) and required_match_span validation, allowing hunks with multi-line signatures or formatted statements to cleanly match single-line targets (and vice-versa) while pruning candidate windows too short to contain all edits. Window scoring adds a whitespace-insensitive character comparison for match blocks of up to 500 non-whitespace characters, and complete definition headers (balanced parentheses or block openers such as do, then, and begin) are never treated as split statements.
• Position-Aware Cumulative Delta Tracking: Added position-aware cumulative delta tracking in HunkApplier to ensure earlier hunk line additions or deletions only shift line numbers for subsequent hunks located downstream in the original file, correctly handling out-of-order or non-monotonic hunk applications.
• Entropy-Based Anchor Ranking: Enhanced candidate anchor ranking in find_search_ranges by entropy and distinctiveness, selected the anchor that most candidate anchors agree on (25-line spatial consensus) to avoid coincidental anchor collisions, bounded anchor search radius up to 120 lines, and aligned search window boundaries with maximum hunk expansion lengths.
• Multi-Anchor Expansion & Ratio Bounds: Replaced the fixed ±8-line fuzzy window ((len / 4).clamp(3, 8)) with proportional, threshold-aware bounds: windows may grow by half the hunk length plus a cushion scaled down for small hunks (at most 150 lines) and shrink by up to 50% of the hunk under the default 0.70 threshold (at most 150 lines). Line-based similarity ratios (ratio_lines and ratio_loose_lines) with expansion penalty limits let multi-anchor hunks spanning newly inserted documentation comments or code match without runaway window bloat.
• Discrepancy Check Whitespace Normalization: Trimmed whitespace during patch normalization in format_normalized_patch to prevent smart-indentation adjustments from triggering false-positive failures in the debug report discrepancy check (-vvvv).

🆚 Compare changes