iwe-org/iwe iwe-v0.12.0

v0.12.0

on GitHub

iwe

Added

• refs_text markdown option — preserve (default) keeps each markdown link's text as written; normalize makes iwe normalize and document output rewrite the text to the linked document's title.
• iwe stats similarity — list mutually near-identical page pairs across the store, each pair once and tab-separated in alphabetical order (forward matches computed once per page, concurrently).
• iwe stats gains an Orphans section listing every page with no incoming links (index pages are exempt as intentional entry points); per-document stats (-k) gain a Similar page line (markdown) / similarPages array (JSON/YAML) of near-identical documents.
• iwe schema validate — validate documents against the schemas bound to them by the [schemas] config section (each entry names a schema file in .iwe/schemas/ and a glob that binds it to document keys). Reports violations as -f text (default) or -f json, and accepts the universal filter flags to scope the check. Exits 1 when any document has violations, 2 on a config or schema-file error, 0 when clean. Bare iwe schema still infers the frontmatter schema.
• iwe schema validate --schema-file <PATH> — validate the selected documents (-k or the filter flags) against a schema file directly, bypassing the [schemas] config bindings, for ad-hoc checks; violations are reported under the file's stem.
• iwe schema validate --explain — print the binding trace (which section and block bound to which schema entry, additional for the rest) instead of validating, to see how the greedy matcher reads a document against a schema.

Changed

• iwe normalize and document rendering keep markdown link text as written by default; set refs_text to normalize to rewrite each link's text to the linked document's title (previously always rewritten).
• update --strict and delete --strict now reject a write that would leave a touched document violating its bound schema, aborting with exit 2 and the violation report before anything is written; previously --strict enforced only the --expect guards.
• update --strict and delete --strict also print non-blocking stats warnings to stderr after the schema gate — orphan pages and dangling links across the store, plus (on update) near-identical pages among the changed documents. They never change the exit code or block the write.

iwes

Added

• refs_text markdown option — set to normalize to make document formatting rewrite each markdown link's text to the linked document's title; preserve (default) keeps the text as written.

Changed

• Document formatting keeps markdown link text as written by default (previously each link's text was rewritten to the linked document's title on format).

iwec

Added

• iwe_create, iwe_update, and iwe_query (update / delete) now return non-blocking stats warnings alongside a successful result — orphan pages, dangling links, and (on create/update) near-identical pages — as warning: content blocks, each finding reported once per session. They never block a write; schema validation remains the only hard reject.
• iwe_stats with a key now returns a similarPages array of documents near-identical to that page.

Changed

• Markdown link text in rendered document content is kept as written by default; set the refs_text markdown option to normalize to rewrite each link's text to the linked document's title as before.
• Mutating tools (iwe_create, iwe_update, iwe_delete, iwe_query, iwe_rename, iwe_extract, iwe_inline, iwe_attach) now reject a change that would leave a touched document violating its bound schema, returning a detailed error carrying the readable report and a structured violations payload; nothing is written and the in-memory graph is left untouched. iwe_normalize and iwe_squash are unaffected.

liwe

Added

• refs_text field on MarkdownOptions and DjotOptions — a RefsText (preserve by default, or normalize) that controls whether a regular markdown link's text is rewritten to the linked document's title; read through FormatOptions::refs_text(), with InlinesContext and Graph gaining normalize_ref_text.
• schema module — document-schema validation: compile_schema compiles a YAML/JSON schema (frontmatter JSON Schema plus an ordered section tree with header, maxTokens, maxDepth, minContains/maxContains, allSections, additionalSections) into a CompiledSchema, and CompiledSchema::validate checks a Document and returns Violations carrying a breadcrumb, hint, schema pointer, and keyword. Unknown keywords are load errors (SchemaError).
• schema validation reaches block content: the document and every section, quote, and list item carry blocks / additionalBlocks / allBlocks — an ordered, greedy-matched array of block schemas discriminated by type (paragraph, bullet-list, ordered-list, code, quote, table, rule, or a list of these for a disjunction), each with text / lang identity, maxTokens, minContains / maxContains, list items / minItems / maxItems, and quote/item recursion; Document and Section now carry a blocks: Vec<Block> field and Crumb gains Block(usize) / Item(usize). An inclusion link is matched as a paragraph.
• CompiledSchema::explain renders the binding trace — which section and block bound to which schema entry, additional for the rest — for a Document. A sections or blocks array with an unreachable entry (a wildcard that is not last, or an exact duplicate of an earlier entry) is a load error.

Changed

• A regular markdown link's text is kept as written when rendering a document; set refs_text to normalize to re-derive it from the linked document's title (previously the text was always re-derived).

diwe

Added

• config::RefsText re-export and the refs_text field it sits on (MarkdownOptions/DjotOptions) — selects whether a markdown link's text is preserved (default) or normalized to the linked document's title.
• stats findings functions — graph_findings (whole-store orphan and dangling-link Findings, discriminated by Rule) and mutation_findings (the same plus a similar-page check for the created/updated keys), with orphan_keys and the now-public broken_links behind them.
• stats::SimilarityIndex — a search index plus per-key token counts, built once per run (SimilarityIndex::build) and reused for similar(key) (near-identical pages for one key, as stats::SimilarPage { key, score }) and pairs() (every mutually-similar pair across the store, each once, computed concurrently). Duplicate detection uses mutual BM25 similarity with a token-size gate and a high threshold.
• GraphStatistics.orphans — the list of orphan keys behind the existing orphaned_documents count; stats::KeyStatisticsReport pairs a KeyStatistics with its similar pages. index pages (root index or any <dir>/index) are treated as intentional entry points and excluded from both the orphan list and the count.
• search::Bm25Index point-score API — similar_to(key, floor) (documents whose self-normalized score against key's own embedding clears a floor, self excluded), self_score(key), and score_between(query_key, doc_key); search_query::corpus_text is now public.
• [schemas] config binding — config::SchemaBinding and config::Patterns types and the Configuration.schemas map bind document schemas to document keys by glob (a single glob or a list). The new schema module resolves and runs them: schema::SchemaBindings matches a key to its schema names, and schema::validate_documents compiles the bound schema files, validates a set of documents, and returns a schema::KeyReport per (key, schema) with violations.
• schema::validate_pending_documents (and schema::validate_pending_documents_in, which takes an explicit schemas directory) validate a set of pending (Key, content) documents against their bound schemas by building a throwaway graph, so a change can be checked before it is written. schema::pending_from_changes collects the touched documents from a Changes set, schema::render_reports_text renders a KeyReport list as text, and config::schemas_dir_in resolves the schemas directory under a given base path.
• schema::validate_documents_against_file — validate documents against one schema file directly, ignoring the [schemas] config bindings; reports are keyed by the file's stem.

Changed

• search now orders tied scores deterministically (score descending, then key ascending); previously ties came back in arbitrary order.