github guidance-ai/guidance 0.3.0

20 hours ago

Guidance 0.3.0

More model providers, better LARK exposure, re-done custom tools.

Breaking Changes

  • Removed old-style @guidance-based tool calls (in favor of more generic OpenAI-style tools)
  • Temp removal of calling support for local models (PR planned to restore support in-line with new tools)

Added

  • Added support for groq and mistralai APIs via litellm
  • Added experimental sglang model
  • Added experimental chat_completion() and chat_completion_streaming() APIs to the engine class, to enable applying a Lark grammar to an LLM without using the DSL
  • Added TTFT to model usage
  • Added support for OpenAI-style tools:function tools (using JSON schemas for parameters)
  • customtools including regex, lark, and (guidance native) grammar
  • Added support for GPT-5

Changed

  • Use official sentencepiece for Python 3.13, now that it is officially supported

Don't miss a new guidance release

NewReleases is sending notifications on new releases.