github stanfordnlp/dspy 3.3.0
on GitHub

4 hours ago

DSPy 3.3.0

DSPy 3.3.0 is a feature release with a new experimental way to optimize programs as code, a native-tool-aware ReAct implementation, and the next stage of DSPy's move toward a typed, provider-neutral language-model system.

Most existing DSPy programs should keep working without changes. Review the API changes if you construct Image, Audio, or File values from paths or URLs; use NumPy-backed features from the base install; inspect detailed GEPA results; construct code interpreters directly; use RLM(max_iterations=...); consume raw Responses API tool-call outputs; or catch provider-specific LM exceptions.

We would especially appreciate feedback on Flex, ReActV2, and the typed LM path. These APIs expand what DSPy can optimize and how it can connect to model providers, and real-world usage will help shape their next iterations.

Highlights

Flex Optimizes Program Structure, Not Just Prompts — @michaelisaac-dev

Most DSPy modules fix the shape of a program up front: Predict makes one prediction, ReAct runs a tool loop, and RLM runs a code interpreter in a loop. Optimizers can improve the instructions around that structure, but the structure itself stays fixed. The new experimental dspy.Flex moves the implementation into the search space so GEPA can discover the decomposition instead.

Give Flex the same signature you would give Predict and it starts with the simplest working baseline: one dspy.Predict, or one dspy.RLM when tools are supplied. During compilation, GEPA can rewrite the complete module implementation—changing the predictors, control flow, DSPy primitives, and balance between Python and LM calls—against your metric. 

program = dspy.Flex("question -> answer")
optimized = dspy.GEPA(metric=metric, reflection_lm=reflection_lm).compile(
    program,
    trainset=trainset,
    valset=valset,
)

print(optimized.module_src)

Optimizer-authored source always runs in a CodeInterpreter sandbox, using dspy.PythonInterpreter by default. Predictor construction and LM calls bridge back to the host, broken candidates score as failures instead of crashing the search, and max_predictor_calls guards against runaway generated programs. Metrics can also accept a program_trace to score how a result was produced—for example, penalizing programs that make too many LM calls.

The optimized module_src is part of the program's serialized state, so saving and loading preserves the implementation GEPA discovered. Flex is experimental, and ordinary GEPA behavior is unchanged when a program does not contain a Flex module.

PR: #10047

ReActV2 and Native Tool-Calling History - @isaacbmiller

dspy.ReActV2 is a new version of ReAct built around native tool calling. It is currently marked as experimental.

The signature now uses dspy.History, dspy.Tool, and dspy.ToolCalls(which can now optionally store dspy.ToolCallResults), rather than the custom next_tool_args and custom trajectory syntax. Using dspy.History also means that messages are now broken up into user/assistant/tool groups rather than one long user message with the trajectory.

This changes the execution model in a few concrete ways:

  • parallel_tool_calls support: DSPy preserves each call/result pair by ID. You can do this in native mode or in non-native mode
  • Multi-turn native tool call support: Prior tool calls and results can be replayed as assistant and tool messages instead of being flattened into prompt text.
  • Each turn lives in dspy.History as structured messages rather than one ever-growing trajectory string, so providers with prompt caching can reuse stable prefixes more effectively. We have seen up to 50% decreases in cost for some tasks when testing this internally.

ReActV2 converts callables to dspy.Tool, adds an internal submit tool for final outputs, handles unknown tools and tool exceptions, accepts serialized history input, and can force final submission when the model does not call submit.

PRs: #9823, #9824, #9825, #9835

Typed, Provider-Neutral LM Boundary - @MaximeRivest

DSPy is moving from an untyped LM boundary based on prompt, messages, and provider-shaped kwargs toward a typed, provider-neutral contract: 

def forward(self, request: dspy.LMRequest) -> dspy.LMResponse:
    ...

The resulting API is a cleaner LM extension point:

  • LiteLLM can become an optional compatibility fallback in the planned 3.5+ path, instead of a required part of the core LM contract.
  • Custom LM authors can implement one typed LMRequest -> LMResponse path instead of guessing which OpenAI/LiteLLM-shaped inputs will arrive.
  • Custom LMs can translate between DSPy's typed objects and their own provider, local runtime, gateway, or inference stack.
  • Adapters can start to depend on DSPy's representation of messages, multimodal content, tool calls, reasoning, citations, usage, cache controls, metadata, and stream events.

Most users do not need to change anything in 3.3. Existing lm(...), modules, and programs keep their current behavior by default.

Try out the typed return path with dspy.context(experimental=True), and the public migration plan explains the staged transition for custom LM and adapter authors.

See the full plan here

PRs: #9786, #9802, #9828

BaseLM Runtime, Save/Load, Errors, and LiteLLM Decoupling - @MaximeRivest

BaseLM now owns shared runtime state and supports sanitized state serialization through dump_state() and load_state(). Serialized LM state excludes API keys, preserves legacy saved states, and requires explicit opt-in before importing trusted custom LM classes.

Saved programs with custom LMs are easier to reason about, LM copies isolate DSPy-owned mutable state, and callers can catch dspy.LMError or a narrower DSPy subclass instead of depending on provider-specific exception classes. LiteLLM imports are lazy, which keeps the core LM API less coupled to a specific provider bridge at import time.

PRs: #9752, #9820, #9821, #9826

LM and Responses API Updates Since 3.3.0b1 - @MaximeRivest, @isaacbmiller

Since the beta, DSPy has added an explicit BaseLM.forward() contract, exported the typed LM API, supported typed direct calls through BaseLM.__call__, made optional-provider imports thread-safe, and fixed LM state round trips for GPT-5 models.

The OpenAI Responses path now emits Responses-native tool and tool_choice request shapes. Legacy Responses outputs use the same Chat-style tool-call representation as the Chat Completions path, while typed LMToolCallPart objects preserve raw provider fields.

PRs: #9837, #9840, #9841, #9843, #9877, #9999, #10003, #10014, #10026, #10028

API Changes

Breaking Changes

Resource Construction and Validation No Longer Perform Implicit I/O

Constructing or validating dspy.Image, dspy.Audio, and dspy.File values no longer interprets locator-shaped strings as instructions to read a local file or fetch a remote URL. This prevents LM-output parsing, Pydantic validation, and deserialization from silently granting filesystem or network access merely because a value resembles a path or URL.

Resource loading now requires an explicit factory:

Before 3.3 DSPy 3.3 Behavior
Image(path) or Image(url=path) Image.from_path(path) Read and embed a local image
Image(url, download=True) Image.from_url(url) Download and embed a remote image
Image.from_url(url) or Image.from_url(url, download=False) Image(url) or Image(url=url) Keep a non-downloading provider-fetched URL reference
Audio(path) Audio.from_path(path) Read and embed local audio
Audio(url) Audio.from_url(url) Download and embed remote audio
File(path) File.from_path(path) Read and embed a local file
encode_image(path) Image.from_path(path) Explicitly read a local image
encode_audio(path_or_url) Audio.from_path(path) or Audio.from_url(url) Explicitly load audio
encode_file_to_dict(path) File.from_path(path) Explicitly read a local file

There are several related compatibility changes:

  • Image.from_url() now downloads the resource and returns an embedded data URI. Use Image(url) when the model provider should fetch the reference instead.
  • Image.from_url(..., download=...) and the download_images / verify options on encode_image() were removed. Choose reference construction or an explicit factory instead.
  • Pydantic payloads containing download or verify are rejected without fetching. The deprecated direct developer call Image(url, download=True) remains available with a warning through 3.3.
  • The deprecated compatibility call requires a positional source: Image(url, download=True). Validation-style calls such as Image(url=url, download=True) are rejected.
  • Image.from_file(), Image.from_PIL(), and Audio.from_file() remain as deprecated aliases through 3.3 and are scheduled for removal in 3.4. Use Image.from_path(), Image(pil_image), and Audio.from_path() respectively.
  • Safe in-memory inputs—including data URIs, bytes, PIL images, audio arrays, structured dictionaries, and existing resource instances—remain supported.

The explicit Image.from_url(url, verify=...) and Audio.from_url(url, verify=...) factories still accept TLS certificate verification controls. The removed verify option applies to encode_image().

Image.from_url() and Audio.from_url() make synchronous caller-initiated requests, follow redirects, and do not provide an SSRF allowlist. Applications remain responsible for validating or allowlisting destinations derived from untrusted input.

PR: #10111 by @isaacbmiller

numpy Is Now Optional

numpy is no longer installed with base dspy. Features that need NumPy now require the numpy extra: 

pip install "dspy[numpy]"

Affected areas include embeddings, KNN/KNNFewShot, SIMBA, and other NumPy-backed optimizer or retrieval paths. (#9659 by @isaacbmiller)

GEPA Result Shapes Changed With gepa[dspy]==0.1.1

The upstream GEPA 0.1.1 API changed several result structures, and DspyGEPAResult now mirrors those shapes. Users who inspect optimized_program.detailed_results may need to update code:

  • DspyGEPAResult.candidates is now a list of compiled DSPy modules, not instruction dictionaries.
  • DspyGEPAResult.best_candidate now returns a compiled DSPy module.
  • val_subscores is now list[dict[Any, float]], keyed by validation instance id.
  • per_val_instance_best_candidates is now dict[Any, set[int]].
  • best_outputs_valset is now dict[Any, list[tuple[int, Prediction]]] when tracked.
  • highest_score_achieved_per_val_task now returns a dictionary keyed by validation instance id.

If you pass custom GEPA reflection templates directly, note that GEPA 0.1.1 renamed default placeholders from <curr_instructions> / <inputs_outputs_feedback> to <curr_param> / <side_info>. In dspy.GEPA, passing reflection_prompt_template through gepa_kwargs now raises a clear ValueError; use instruction_proposer for custom proposal behavior instead. (#9673 by @BenMcH)

RLM.max_iterations Is Now RLM.max_iters

The RLM constructor now uses the same max_iters name as other iterative DSPy modules: 

# Before
rlm = dspy.RLM("context, query -> answer", max_iterations=10)

# DSPy 3.3
rlm = dspy.RLM("context, query -> answer", max_iters=10)

PR: #9920 by @isaacbmiller

RLM Rejects Colliding Names and Unexpected Inputs

RLM now validates its execution namespace up front. Construction fails for duplicate tool names, Python-keyword tool names, signature inputs that collide with built-in sandbox functions or tools, and output fields named trajectory or final_reasoning. Invocation also rejects unexpected input fields rather than ignoring them. Rename colliding fields or tools before constructing the module.

PR: #10020 by @isaacbmiller

RLM Sub-LM and Tool Execution Are Stricter

llm_query and llm_query_batched now require the sub-LM to return either a dspy.LMResponse containing text or a non-empty legacy output list whose first item is text. Arbitrary response objects are no longer converted with str(). Batched queries convert dspy.LMError failures to [ERROR] entries, but programming and response-contract errors now propagate.

RLM also invokes user tools through dspy.Tool, so tool argument validation and coercion, default handling, and tool callbacks now apply.

PRs: #10023, #10025 by @isaacbmiller

Code-Executing Modules Use an Interpreter Factory

ProgramOfThought, CodeAct, and RLM now accept interpreter_factory=, a zero-argument callable that creates a fresh CodeInterpreter for each invocation. This isolates concurrent calls and makes interpreter ownership explicit. 

program = dspy.ProgramOfThought(
    "question -> answer",
    interpreter_factory=MyInterpreter,
)

To use an existing interpreter, pass it as the first positional argument when invoking the module. DSPy does not shut down a caller-owned interpreter, and reuse is supported only for sequential calls to the same module instance.

These modules no longer expose a constructor-owned interpreter attribute or preserve its sandbox state across calls. Without a caller-owned interpreter, each invocation creates and shuts down a fresh interpreter. Interpreter process and protocol failures are terminal for that interpreter session rather than automatically restarting it, and submitted-code failures now raise CodeExecutionError, a subclass of CodeInterpreterError.

PRs: #10018, #10022 by @isaacbmiller

ToolCalls Uses DSPy's Native Serialized Shape

dspy.ToolCalls.format() and Pydantic serialization now represent each call as {"name": ..., "args": ...} rather than the prior OpenAI-style {"type": "function", "function": {"name": ..., "arguments": ...}} shape. Code that persists ToolCalls, calls .format() directly, or forwards the result to an OpenAI-compatible endpoint must update its conversion logic. Provider adapters still produce the wire shape required by their APIs.

PR: #9823 by @isaacbmiller

Direct BaseLM Defaults and Copy Semantics Changed

The direct BaseLM constructor now defaults temperature and max_tokens to None instead of 0.0 and 1000. This primarily affects custom LM subclasses that inherit or delegate to BaseLM.__init__; ordinary dspy.LM already used the provider-default None values.

BaseLM.copy() now shallow-copies subclass-owned attributes while separately copying DSPy-owned mutable runtime containers. Custom LM subclasses that relied on arbitrary mutable attributes being deep-copied should override copy() or copy that state explicitly.

PR: #9821 by @MaximeRivest

Responses Tool Calls Use the Chat-Compatible Legacy Shape

Legacy output from dspy.LM(..., model_type="responses") now represents tool calls in the same shape as the Chat Completions path: 

{
    "type": "function",
    "id": call_id,
    "function": {
        "name": tool_name,
        "arguments": arguments,
    },
}

The text key is now always present. Raw Responses item IDs, status, and provider fields remain available through LMToolCallPart.provider_data on the typed path.

PR: #10028 by @isaacbmiller

LM Error Types Are DSPy-Normalized

LM failures are now mapped into DSPy exception classes. This should make LM error handling more consistent, but code that catches provider-specific or LiteLLM-specific errors directly may need to catch dspy.LMError or a narrower DSPy subclass. (#9826 by @MaximeRivest)

ChatAdapter and JSONAdapter no longer retry with an alternate output format after an LMError; provider and transport failures propagate directly. TwoStepAdapter now raises dspy.AdapterParseError rather than ValueError for local extraction or parsing failures, while extraction-LM failures propagate as dspy.LMError.

New and Updated APIs

  • Added exported LMRequest, LMResponse, typed message/content classes, tool specifications, reasoning and cache configuration, usage records, history entries, and streaming types. (#9786, #9841, #9877 by @MaximeRivest)
  • Added forward_contract = "typed_lm" and an explicit BaseLM.forward(request) -> LMResponse contract for new custom LM implementations. (#9843, #9877 by @MaximeRivest)
  • Added Signature.append_instructions() for deriving a signature with additional instructions without mutating the original. (#9923 by @mathurk1)
  • Added experimental dspy.Flex and trace-aware GEPA metrics through the optional sixth program_trace parameter. (#10047 by @michaelisaac-dev)
  • Added experimental dspy.ReActV2, structured native tool-call history, and tool-result replay. (#9823, #9824, #9825, #9836 by @isaacbmiller)
  • Added explicit from_path() and from_url() resource-loading factories for Image and Audio, plus File.from_path(), while keeping construction and validation free of implicit host I/O. (#10111 by @isaacbmiller)
  • Made BaseLM own and serialize shared runtime state. copy() now makes a shallow object copy, resets history, and separately copies the callbacks list and kwargs dictionary. (#9820, #9821 by @MaximeRivest)
  • Excluded defaulted arguments from required tool parameters and stabilized open-ended tool argument schemas. (#9971 by @katherineahn, #10012 by @isaacbmiller)
  • Protected RLM-owned namespaces, normalized sub-LM responses, and routed tool execution through dspy.Tool. (#10020, #10023, #10025 by @isaacbmiller)
  • Made interpreter process and protocol failures terminal for the affected session and isolated interpreters per invocation. (#10018, #10022 by @isaacbmiller)
  • Improved interpreter transport for Pydantic objects, scalar values, dataclasses, named tuples, and non-finite floats. (#9753 by @Archelunch, #9991 by @Vinay152003, #10015 and #10049 by @migurski)
  • Made module load_state() transactional, isolated BootstrapFinetune data by predictor, validated random-search restrictions up front, and preserved callback lineage in parallel workers. (#9741 by @ashishSoni1234, #10005 and #10043 by @isaacbmiller, #10033 by @chuenchen309)

Compatibility Notices

  • OpenAI 1.66.2 is now the supported minimum. (#9999 by @isaacbmiller)
  • LiteLLM 1.65.8 is required for the reasoning-capability API. (#10003 by @isaacbmiller)
  • The LangChain extra now requires LangChain Core 0.3.0 or newer. (#10004 by @isaacbmiller)
  • The base install no longer depends directly on asyncer, xxhash, or typeguard; DSPy now uses AnyIO and standard-library equivalents. (#9733, #9734, #9735 by @isaacbmiller)
  • dspy.utils.hasher.Hasher now uses SHA-256 instead of xxhash64. Hash strings, hash-derived fine-tuning filenames, and deterministic bootstrap trace selection may change; LM response-cache keys are unaffected. (#9734 by @isaacbmiller)
  • Module.set_lm(), get_lm(), and state loading now include module-valued parameter leaves such as Flex, rather than only Predict. Custom classes that combine Module and Parameter should expose compatible LM state and accept allow_unsafe_lm_state= when overriding load_state(). (#10047 by @michaelisaac-dev)

Full PR List

Language Models and Adapters — 23 PRs

Agents, Tools, and Interpreters — 25 PRs

Core APIs and Optimizers — 10 PRs

Documentation and Community — 16 PRs plus one direct commit

CI, Testing, and Release Infrastructure — 17 PRs

Dependencies — 18 PRs

Contributors

Thank you to everyone who contributed to DSPy 3.3.0:

Automation contributions were made by @dependabot and @github-actions.

First-Time Contributors

Full Changelog: 3.2.1...3.3.0

Check out latest releases or
releases around stanfordnlp/dspy 3.3.0

Don't miss a new dspy release

NewReleases is sending notifications on new releases.

Get notifications