Summary
v0.1.21 brings significant performance improvements and resolves previous issues with infinite loops and system freezes. It introduces the Earley parser. The preprocessing time is now up to six times faster than before (See #308), and it no longer suffers from the previous problems of exponential growth in the number of states or infinite (or very deep) recursion.
It also fixes the cache corruption problem that occurs when the input is invalid in GrammarCompiler.
What's Changed
- [Feature] Use an earley parser to replace the current pushdown automata by @Seven-Streams in #308
- Fix: fix wrong LRU cache implementation by @DarkSharpness in #352
- [Refactor] Refactor file structure by @Ubospica in #353
- [Refactor] Refactor utils and FSM lib by @Ubospica in #354
- [Minor] Replace all Result<...>::Ok with ResultOk by @DarkSharpness in #355
- [Refactor] RuleExpr -> GrammarExpr by @Ubospica in #357
- Bump to v0.1.21 by @Ubospica in #358
Full Changelog: v0.1.20...v0.1.21