iwe-org/iwe iwe-v0.7.0

v0.7.0

on GitHub

iwe

Added

• retrieve --limit, and --max-tokens / --max-document-tokens on retrieve and find, to bound output for context-limited callers. 0 disables a limit. A warning: line is printed to stderr when output is truncated.

Changed

• find markdown output is now a compact index (one line per document) instead of full document blocks; a document body is rendered only when the projection includes $content (via --project / --add-fields). Use retrieve for full content.

Removed

• retrieve --no-content — removed; retrieve always returns content. Use find for a metadata-only index.
• retrieve --dry-run — removed.

iwec

Added

• iwe_retrieve and iwe_find accept max_tokens / max_document_tokens (and iwe_retrieve a limit) to bound output; all are unlimited unless set. When a limit trims the output, the tool appends a second content block with a JSON truncation summary (truncated, emitted, matched, clipped, tokens, budget, hint) alongside the unchanged primary JSON.

Removed

• iwe_retrieve tool no_content parameter — removed; the tool always returns content.

liwe

Added

• tokens module (count_tokens, truncate_to_tokens) backed by tiktoken-rs.
• RetrieveOptions gains limit, max_tokens, max_document_tokens; FindOptions gains max_tokens, max_document_tokens; RetrieveOutput / FindOutput gain a truncation summary.

Removed

• RetrieveOptions::no_content field — removed; DocumentReader always populates content.