github stanfordnlp/dspy 3.4.1
on GitHub

4 hours ago

DSPy 3.4.1

DSPy 3.4.1 makes GEPA's structured InstructionProposer the default instruction proposer and lets dspy.RLM build DSPy sub-agents from inside its REPL. It also adds Python 3.15 support, fixes a 3.4.0 import crash, and includes streaming, adapter, and interpreter fixes. 

pip install --upgrade "dspy==3.4.1"

Upgrade notices. GEPA's default reflection request changes shape, so optimization results can differ from 3.4.0 even with the same seed. The reflection LM must now be a dspy.LM that can return JSON. Every RLM prompt now includes a sub-agent section, and max_llm_calls now also counts sub-agent LM calls. dspy.ReActV2 signatures can no longer declare inputs named history or tools. Programs saved on 3.4.0 with a null Score level no longer load. See API and Compatibility Changes.

The 3.5 LM-interface cutoff announced in 3.4.0 still applies. Legacy forward()/aforward() custom LMs, the legacy-engine wrappers, custom complete_legacy() shortcuts, and OpenAI-style messages= calls are deprecated and will be removed in 3.5. lm("hello") stays supported.

Highlights

InstructionProposer Is GEPA's Default Proposer — @dbreunig

dspy.GEPA now builds InstructionProposer() when you don't pass instruction_proposer=. The old default formatted reflective examples into one raw prompt and pulled the new instruction out of backticks. The new one calls the reflection LM through dspy.Predict with a JSONAdapter. Reflective examples stay structured, so dspy.History inputs, tool trajectories, and dspy.Type values such as dspy.Image (including nested images) reach the reflection LM as structured content instead of text placeholders. GEPA's reflection instructions are unchanged.

You can configure the proposer directly: 

import dspy
from dspy.teleprompt.gepa import InstructionProposer

optimizer = dspy.GEPA(
    metric=metric,
    reflection_lm=dspy.LM("openai/gpt-5"),
    instruction_proposer=InstructionProposer(
        skills=["./skills/prompt-engineering"],
        additional_instructions="Write instructions in imperative voice.",
        max_chars=1500,
        truncate_history_outputs=True,
    ),
    auto="medium",
)
  • skills: reference material for the reflection LM. Pass a path, a directory containing SKILL.md, an inline string, or a list of these. YAML frontmatter is parsed at construction.
  • additional_instructions: extra guidance applied to every proposal.
  • base_instructions: replaces the reflection prompt but keeps its input and output fields.
  • max_chars: a hard character limit. A draft that is too long gets one compression attempt; DSPy never truncates an instruction.
  • truncate_history_outputs: shortens long tool and REPL outputs inside history inputs before rendering, without changing the originals.
  • adapter: the adapter used for proposal calls. The default is JSONAdapter().

Custom instruction_proposer functions receive the same reflective dataset as before, and MultiModalInstructionProposer remains available.

This release also fixes reflection inputs that contain a value pydantic can't serialize, such as a numpy.int64 from a pandas row. That value now renders as str(value) in place, instead of turning every example into a Python repr.

PRs: #10555; follow-up fix #10634 by @isaacbmiller

RLM Can Build DSPy Sub-Agents — @michaelisaac-dev

Code that dspy.RLM writes can now import dspy and build sub-agents for subtasks that need structured inputs and outputs, tools, or their own recursion:

  • dspy.Predict and dspy.ChainOfThought
  • dspy.ReActV2, using the tools you gave the RLM
  • a nested dspy.RLM, which runs on the same interpreter backend as its parent

Sub-agents are built and run on the host. Only JSON crosses the interpreter boundary.

There is no new API to adopt; this is on for every RLM: 

import dspy

rlm = dspy.RLM(
    "logs, question -> answer",
    sub_lm=dspy.LM("openai/gpt-4o-mini"),  # used by llm_query and sub-agents
    max_llm_calls=30,                       # one budget for both
)

Inside the REPL, the model can now write code like this: 

import dspy

triage = dspy.ChainOfThought("log_chunk -> errors")
errors = [triage(log_chunk=chunk).errors for chunk in chunks]
  • One budget. max_llm_calls covers llm_query, llm_query_batched, and every sub-agent LM call. Each requested completion uses one slot, so n can't multiply it. Sub-agents use sub_lm, or dspy.settings.lm when it isn't set.
  • Restricted LM options. LM calls made for sandboxed code accept only generation parameters, such as temperature, max_tokens, response_format, reasoning effort, tool settings, and caching options. Sandboxed code can't set model, api_base, api_key, or headers.
  • Interpreters. The default PythonInterpreter and process-isolated interpreters such as LocalInterpreter host sub-agents. If an interpreter can't run the sub-agent shim and raises CodeExecutionError or SyntaxError, RLM logs a warning and runs that call without sub-agents. A RestrictedPython-based interpreter is one example.
  • In-process interpreters. Interpreters that run code with exec() in the host process also host sub-agents. DSPy binds the facade in that interpreter's namespace and leaves the host's dspy module in place. These interpreters isolate nothing, so the call budget and LM-option restrictions only bind well-behaved code. Use them only for trusted code.

dspy.Flex uses the same facade for its bridged predictors. Flex now also works with in-process interpreters you choose, without replacing the host's dspy module.

PRs: #10478; follow-up fixes #10625 and #10627 by @isaacbmiller

API and Compatibility Changes

GEPA Reflection Requests

  • Optimization results can differ from 3.4.0 even with the same seed. The default reflection request is now a system message with structured input fields and JSON output.
  • The reflection LM must return JSON that the adapter can parse. If the reflection model answers in plain text or a code fence, the proposal fails. GEPA records the failure in its log and keeps the current candidate, so a run can finish with the seed program unchanged. Use a reflection model that supports JSON or structured outputs.
  • reflection_lm must be a dspy.LM or another dspy.BaseLM. In 3.4.0, GEPA called reflection_lm(prompt) directly, so a plain function happened to work. Now every proposal made with a plain function fails, and the program stays unoptimized. Wrap a custom endpoint in dspy.LM, using a custom engine= if needed.
  • dspy.teleprompt.gepa.gepa_utils no longer re-exports InstructionProposalSignature. Import it from gepa.strategies.instruction_proposal instead.
  • pyyaml>=6.0.1 is a new dependency.

PRs: #10555

RLM Sub-Agents

  • Every RLM prompt now includes a sub-agent section. There is no flag to turn it off. Because RLM prompts differ from 3.4.0, LM caches and recorded responses from 3.4.0 won't match.
  • Sub-agent calls count toward max_llm_calls. An RLM that uses sub-agents reaches the limit sooner; raise it if needed.
  • New reserved names. An RLM input named dspy, or a tool named dspy or starting with _dspy or __dspy, now raises ValueError.
  • Interpreter authors: each RLM call now runs the sub-agent shim through execute() before the first iteration. Scripted or fake interpreters used in tests receive that shim as their first execute() call. Recognize it by comparing against dspy.primitives.facade.SHIM_SETUP.
  • Known limitation: an interpreter that raises anything other than CodeExecutionError or SyntaxError while running the shim aborts the RLM call. Raise CodeExecutionError for code your interpreter can't run. A follow-up will make RLM tolerate these interpreters when no sub-agent is used.

PRs: #10478, #10625

Flex Bridge Constants Moved to dspy.primitives.facade

Breaking change for sandbox backend authors. The shim constants and helper that dspy.predict.flex.bridge exposed now live in dspy.primitives.facade, which dspy.Flex and dspy.RLM share. The old module doesn't re-export them, so importing any of these names from dspy.predict.flex.bridge raises ImportError: SHIM_SETUP, CONSTRUCT_TOOL, CALL_TOOL, BRIDGEABLE_KINDS, SIGNATURE_MARKER, TOOL_MARKER, prediction_to_fields. 

# Before (3.4.0)
from dspy.predict.flex.bridge import BRIDGEABLE_KINDS, CALL_TOOL, CONSTRUCT_TOOL, SHIM_SETUP

# After (3.4.1)
from dspy.primitives.facade import BRIDGEABLE_KINDS, CALL_TOOL, CONSTRUCT_TOOL, SHIM_SETUP

The marker and tool-name constants keep their values. The private shim file moved from dspy/predict/flex/_sandbox_shim.py to dspy/primitives/_facade_shim.py, so dspy.predict.flex._sandbox_shim no longer imports. Read the shim through SHIM_SETUP. Code that only constructs and runs dspy.Flex programs is unaffected.

PR: #10478

Null Score Level Descriptions Are Rejected

Each Score level description must be a string, an object, or an array. DSPy 3.4.0 accepted null; 3.4.1 rejects it before any request is made:

  • Declaring Score[None, "Disruptive", "Blocking"] raises ValueError: Score level descriptions cannot be null. when the signature class is defined.
  • predictor.set_criteria(name, [None, ...]) raises ValueError.

Programs saved on 3.4.0 with a null Score level no longer load. program.load(path) raises ValueError: Invalid criteria for '<name>': must match the declared decision type and options. The message doesn't mention null. Whole-program saves (save_program=True) still load with dspy.load(), but raise the same error the first time the predictor runs or is saved.

To fix a saved JSON program, replace each null in the field's criteria list with a description. Alternatively, delete the criteria key so the field falls back to the levels declared in the signature: 

 "fields": {
   "severity": {
-    "criteria": [null, "Disruptive", {"what": "Blocking"}]
+    "criteria": ["Minor", "Disruptive", {"what": "Blocking"}]
   }
 }

For a whole-program save, call predictor.set_criteria(name, [...]) without null entries after dspy.load(), then save again.

PR: #10538

Bundled Model Metadata Now Follows LiteLLM 1.105.0

DSPy's bundled model-metadata snapshot moved from LiteLLM 1.102.1 to 1.105.0. Upstream added 803 entries and removed 264. Of those, 234 had passed their provider deprecation dates and 30 were renamed or retired.

Offline setups lose capability flags for some still-routable models. DSPy uses the bundled snapshot only when it can't use LiteLLM's live map: offline, when the fetch fails, or with LITELLM_LOCAL_MODEL_COST_MAP=true. In those setups, models whose entries were removed no longer report tool-calling, structured-output, or reasoning support. Examples are anthropic/claude-3-7-sonnet-20250219, gemini/gemini-2.0-flash, xai/grok-4, and groq/llama-3.3-70b-versatile. DSPy changes these requests without a warning:

  • JSONAdapter stops requesting structured outputs and falls back to JSON mode or prompt-only JSON.
  • Adapters stop sending tools as native function calls.
  • dspy.Reasoning no longer turns on native reasoning for models that lost the flag.

Online, 3.4.0 and 3.4.1 behave the same for these models, because LiteLLM's live map has dropped the same entries.

Family fallback rules are now provider-scoped. A capability rule that names its providers now fills missing flags only for those providers' entries. As a result, gemini/deep-research-preview-04-2026, gemini/deep-research-max-preview-04-2026, and gemini/deep-research-pro-preview-12-2025 no longer inherit supports_reasoning from the OpenAI deep-research rule.

What to do:

  • If you rely on structured outputs or native tool calls, move to current model IDs, and check lm.supports_response_schema and lm.supports_function_calling after upgrading.
  • To keep a removed model working offline, point LITELLM_MODEL_COST_MAP_URL at a metadata file that still contains its entry, and unset LITELLM_LOCAL_MODEL_COST_MAP. The file replaces the whole map, so it must cover every model you use.

PRs: #10517, #10539, #10598, #10624

Python 3.15 and Dependencies

  • DSPy now supports Python 3.15 (requires-python = ">=3.10, <3.16").
  • On Python 3.15, litellm and gepa are not installed, because both declare requires-python <3.15. LM calls run on the bundled native engine, and routes that need the LiteLLM fallback are unavailable there. dspy.GEPA raises ImportError on 3.15 until gepa supports it.
  • jsonschema>=4.0.0 is now a direct dependency.

PR: #10521

Custom Adapters and main_request

History replay now passes main_request= to format_user_message_content, so each historical turn keeps the output-format reminder from the original request. A custom adapter that overrides format_user_message_content without a main_request parameter raises TypeError when it formats dspy.History. Add main_request: bool = False to your override. TwoStepAdapter has been updated.

PRs: #10554, #10628

ReActV2 Reserves the history and tools Input Names

dspy.ReActV2 signatures can no longer declare input fields named history or tools; the constructor raises ValueError. ReActV2 supplies both itself: history is its structured run history, and tools is the tool list you pass to the constructor.

  • If your signature declares history: dspy.History, for example a chat agent that carries earlier turns, remove the field and keep passing history= when you call the agent. ReActV2 already used that argument to seed the run, so behavior is unchanged: 

    # Before (3.4.0)
agent = dspy.ReActV2("history: dspy.History, question -> answer", tools=[lookup_order])

# After (3.4.1)
agent = dspy.ReActV2("question -> answer", tools=[lookup_order])

result = agent(question=question, history=history)  # the call is unchanged

  • If your signature uses history or tools for your own data, for example history: str holding a ticket transcript, rename the field. In 3.4.0, ReActV2 replaced such fields with its own, and calls failed with a validation error or TypeError.

  • Tools are fixed when the agent is constructed. Passing tools= at call time has no effect.

PR: #10464

X | Y Annotations Render Like typing.Union

On Python 3.10–3.13, adapters now describe a field annotated int | None as Union[int, NoneType] instead of UnionType[int, NoneType]. This matches how Optional[int] already rendered and how Python 3.14 renders int | None. Parsing is unchanged, but prompts for signatures that use | unions differ from 3.4.0, so cached LM responses and recorded prompts from 3.4.0 won't match.

PR: #10575

Additional Fixes

  • dspy.Tool stores argument defaults in JSON form: an Enum becomes its value, Decimal a float, datetime and date ISO strings, Path a string, tuples and sets lists, and pydantic models and dataclasses dicts. Tool schemas with such defaults previously failed to serialize for native function calling. The tool still receives its original Python default when the argument is omitted. #10574
  • Fix the 3.4.0 import crash ImportError: cannot import name 'BaseModel' from partially initialized module 'openai._models', which could occur when DSPy lazily imported LiteLLM, for example through dspy.LM(...).supports_function_calling. #10520
  • Streamed ChatAdapter and XMLAdapter chunks no longer include the whitespace around field markers, so the joined chunks equal the parsed field value. #10553
  • Replayed dspy.History turns keep their output-format reminders, so each turn's prompt is a prefix of the next one for prompt caching. #10554
  • A JSON value that contains one item pydantic can't serialize now renders as JSON with that item as str(item), instead of the whole value becoming a Python repr. This applies to all adapters. #10634
  • BetterTogether raises a ValueError that names the student predictors without an LM. 3.4.0 failed later with AttributeError: 'NoneType' object has no attribute 'launch'. Call student.set_lm(lm); dspy.configure(lm=...) alone is not enough. #10569
  • Calling a module's forward no longer walks the whole call stack to decide whether to warn. #10534

Full PR List

GEPA and Optimizers

Agents, RLM, Flex, and Interpreters

  • Reject ReActV2 input fields named history or tools by @chenlichao (#10464).
  • Let RLM build DSPy sub-agents through a sandbox facade shared with Flex by @michaelisaac-dev (#10478).
  • Fall back to no sub-agents when an interpreter rejects the facade shim with SyntaxError by @isaacbmiller (#10625).
  • Host the facade in in-process interpreters without replacing the host's dspy by @isaacbmiller (#10627).

Adapters, Streaming, and Types

Core, Performance, and Platform

Model Metadata

Documentation

Testing, CI, Dependencies, and Release Engineering

  • Update main's version metadata after the 3.4.0 release (#10496).
  • Support the latest LiteLLM and MCP test fixtures by @isaacbmiller (#10593).
  • Deflake the LocalInterpreter timeout tests by @isaacbmiller (#10630).
  • Update the GitHub Actions upload-artifact action (#10541).
  • Update the documentation build's mkdocstrings-python dependency (#10543).
  • Update the documentation build's zensical dependency (#10544).

Contributors

Thank you to @adityashirsatrao007, @Aryan-Kochhar, @atharva-mashalkar, @chenlichao, @dbreunig, @ilia-iliev, @isaacbmiller, @jayhemnani9910, @jayzuccarelli, @kayzenline, @michaelisaac-dev, @migurski, @simpleqt, @thomasht86, @zamal-db, and @zh-hanlabs for contributing to this release. Dependency and version updates were automated by Dependabot and GitHub Actions.

Full Changelog: 3.4.0...3.4.1

Check out latest releases or
releases around stanfordnlp/dspy 3.4.1

Don't miss a new dspy release

NewReleases is sending notifications on new releases.

Get notifications