Preview release for early testing. This release is cut from the preview/v0.2.6rc1 branch, not from main.
Highlights:
- Lark grammar compatibility with llguidance v1.8 (hex escapes, terminal set operations, complements, byte/Unicode patterns, text anchors, capture names, grammar options), plus faster Lark compilation. Thanks to the llguidance team for designing these Lark grammar extensions.
- JSON Schema string
pattern/minLength/maxLengthsemantics fixes, using the ECMAScript regex dialect (\d/\ware ASCII) as required by JSON Schema, while Lark keeps Unicode character classes. - Correctness fixes: indirectly nullable repetition ranges, compilation cache size accounting, structurally invalid compiled-grammar cache deserialization now reports an error, raw jump-forward bytes preserved for invalid UTF-8, partial three-byte UTF-8 range handling.
- Performance: dynamic JSON Schema compilation and token-mask generation improvements; token-mask hot path stack usage reduced.
Install with:
pip install xgrammar==0.2.6rc1Note: pip does not pick up pre-releases by default; pin the exact version as above or use pip install --pre xgrammar.