simonw/llm 0.32

on GitHub

LLM 0.32 is a major, backwards-compatible update to the way prompts, responses, tools and logs are represented. It adds structured messages and parts throughout the Python API, adopts the OpenAI Responses API for reasoning-capable models, substantially expands control over pausable and resumable tool loops and introduces a new content-addressed SQLite logging schema. Reasoning traces are now displayed on standard error by the llm command, for models that support them.

Structured messages and richer responses

Prompt inputs and model outputs are now represented as lists of Message objects, each containing typed Part objects for text, reasoning, tool calls, tool results and attachments.

• New messages= keyword argument on the prompt, conversation and chain APIs, including their asynchronous equivalents. For example, model.prompt(messages=[llm.user("Hello"),llm.assistant("Hi!"), llm.user("What can you do?")]). Existing prompt=, system=, attachments= and tool_results= arguments continue to work and are converted into the same structured representation.
• New structured streaming methods response.stream_events() and response.astream_events()expose mixed streams of text, reasoning, tool calls and tool results. Iterating over a response directly continues to yield text strings.
• response.messages() returns the assembled structured output. response.reply() continues from any response and can automatically execute pending tool calls before the next turn.
• response.to_dict() and Response.from_dict() provide JSON-safe persistence of complete turns, including reasoning and provider metadata, with matching TypedDict definitions in the new llm.serialization module.
• response.prompt.messages is now the canonical record of exactly what was sent to the model, including the full preceding conversation chain.

These APIs were introduced in 0.32a0. They are described in the Advanced model pluginsdocumentation.

OpenAI Responses API and reasoning

• Most reasoning-capable OpenAI models now use the /v1/responses endpoint by default, enabling interleaved reasoning across tool calls. The existing Chat Completions classes remain available, and -o chat_completions 1 selects that older path for an individual prompt. See 0.32a2 for the full list of affected models.
• OpenAI Responses API models now provide WebSearch and CodeInterpreter server-side tools, available from the CLI using -T WebSearch or -T 'CodeInterpreter(memory_limit="4g")'.
• Visible reasoning summaries are streamed to standard error by llm prompt and llm chat. Use -R/--hide-reasoning or the new hide_reasoning=True Python argument to hide them. Encrypted reasoning metadata is preserved for subsequent turns.
• The default model for users who have not selected one is now GPT-5.6 Luna, replacing GPT-4o mini. New built-in models include gpt-5.6-sol, gpt-5.6-terra and gpt-5.6-luna; models that are no longer available from OpenAI have been removed. See 0.32rc1 and 0.32rc2 for details.
• New llm openai endpoint command runs prompts and chats, or lists models, against an arbitrary OpenAI-compatible endpoint without configuring it first. These calls are not logged.
• OpenAI models now support a service_tier option. Use -o service_tier fast for faster responses at a higher price, or -o service_tier flex for slower, cheaper processing on supported models. See Fast mode and service tiers. #1585
• New llm -m model --options flag lists the options supported by a model. The Python prompt APIs now accept an explicit options= dictionary as well as the previous keyword-argument form.

More controllable tool loops

• Every tool call now has a unique tool_call_id, synthesized when the provider does not supply one. Tool implementations can accept an llm_tool_call parameter to inspect the current call and its ID.
• Tools can raise llm.PauseChain to pause execution for human approval or another external event. Chains can later resume from a message history ending in unresolved tool calls, without repeating calls that already have results.
• Conversations that use configured tools can be continued with llm -c or llm chat -c without repeating the toolbox configuration.
• llm tools now shows constructor signatures and docstrings for dynamic toolboxes. Passing a toolbox specification instantiates it and lists its runtime-generated tools, while llm tools --jsonidentifies dynamic toolboxes with a "dynamic" boolean. #1580
• Models can now declare the server-side tools they support using the instance-level supported_server_side_tools property and the new llm.ServerSideTool base class. Server-executed calls and results are captured as structured message parts, and llm -c restores configured server-side tools for continued conversations. #1592, #1593
• llm tools -m MODEL lists the server-side tools supported by that model. llm models --json returns model aliases, capability flags, attachment types and server-side tools, with option schemas included when combined with --options.
• OpenAI-compatible Responses endpoints can use provider-specific server-side tools with ServerSideTool(spec={...}), including OpenRouter’s web search implementation.

See 0.32a3 for more detail on pausing, resuming and inspecting tool calls.

New SQLite logging schema

LLM now logs prompts and responses using a new schema built around threads, turns and a content-addressed message store. Existing records in the legacy responses table are left untouched, and llm logs combines both generations of data. You can create a backup before upgrading using:

llm logs backup logs-backup.db

• Messages are stored once and referenced by their content hash, preserving structured text, reasoning, attachments and tool activity without duplicating repeated conversation history. See the message store documentation.
• Raw provider payloads are stored in turns.response_json, condensed using condense-json. llmlogs --json expands them back to their original shape, and LogStore.turn_response_json(turn_id) returns them from Python. #1586
• Model plugins can define json_replacements dictionaries to further improve payload compression.
• Full-text search, model and tool filters, and conversation views work across both the legacy and new tables. Logs now record which configured toolbox instance supplied each tool.
• New Response.log_to_db() Python API writes a response to a logs database. llm prompt --jsonoutputs the same structured representation as llm logs --json, even when persistent logging is disabled.
• New message_tree SQL view renders conversation threads as indented text outlines for direct SQL exploration.
• LLM now requires sqlite-utils 4.0 or higher and no longer depends on sqlite-migrate.

See 0.32rc1 for the detailed migration notes and complete list of logging changes.

Fixes since 0.32rc2

• Fixed streamed OpenAI Responses API calls recording two different ciphertexts of the same reasoning: the part’s encrypted_content was harvested from the response.output_item.done event while response_json came from response.completed, and OpenAI encrypts per event. Reasoning metadata is now re-emitted from the final payload, so both records agree on one blob.
• Attachments loaded from URLs now follow up to three redirects when detecting their content type or fetching their bytes. Thanks, Ojas Sharma. #1046, #1579
• Fixed a bug where llm openai endpoint --schema was ignored if the selected template also defined a schema. Thanks, ikatyal2110. #1588
• llm logs status now counts records in the new threads and turns tables, with legacy conversation and response counts shown separately when present.
• Response.to_dict() now executes an unconsumed synchronous response before serializing it instead of producing an empty assistant message list.
• Response.from_dict() now restores pending client-side tool calls so they can be inspected, executed or continued using response.reply(tools=[...]).
• Response.reply() now correctly passes attachments returned by tools to the next model call, for both synchronous and asynchronous responses.