Genkit Python 0.13 is the last release with major breaking changes, and the last release before 1.0. Yes, there are a lot of breaking changes, and that's on purpose. The SDK we had going in carried many conventions borrowed from Genkit JS and Go. In this release we went through the whole API surface and made it feel native to Python, while keeping it consistent with what you'd expect from the other Genkit SDKs.
We chose clean breaks over deprecation shims and aliases because we want a clean, Pythonic API surface going into 1.0, not one that carries two names for everything. We didn't want to break you a little at a time before 1.0, and we won't break you after it. So everything that wasn't Pythonic comes to light in this one release. This is the API we're sticking with, and you shouldn't expect changes like these going forward.
Upgrading? Follow the 0.12 → 0.13 migration guide, or point your coding agent at it.
Evaluators are the one exception. They still ship in 0.13 and go away in 1.0, the next release, while we redesign them across all the Genkit SDKs. We kept them on purpose. Older plugin releases accepted any version of genkit, so removing evaluators now could let pip install an old genkit-evaluators next to a genkit that no longer has them. 0.13 adds version bounds to every plugin, and genkit-evaluators now requires genkit<1, so the next release can remove them cleanly (see below).
Why upgrade
Feels like Python
Your IDE and type checker now catch Genkit mistakes before you run anything. Contexts are generic, config fields are typed Literals, and signatures are keyword-only. Cancellation is plain asyncio, and flow input and output are validated by Pydantic.
@ai.flow()
async def plan(trip: Trip, ctx: FlowContext[Status]) -> Itinerary:
ctx.send_chunk(Status(step='search')) # chunk type checked by pyright
res = await ai.generate(
prompt=trip.ask,
config=GeminiConfig(thinking_level='HIGH'), # a typo here is a type error
)
...
task = asyncio.create_task(plan(trip))
task.cancel() # cancels the model call and any running tools, like any asyncio codeSafe by default
In 0.12, some mistakes failed silently or in surprising places. In 0.13 they fail where you made them, with an error that names the problem.
# 0.12 silently dropped the misspelled key. 0.13 raises before the model is called.
await ai.generate(prompt=p, config={'temprature': 0.2})
# 0.12 let a tool's catch-all swallow an approval request.
# In 0.13, Interrupt is a BaseException, so it reaches the caller.
@ai.tool()
async def refund(order: Order) -> dict:
try:
require_approval(order) # raises Interrupt
except Exception:
log.warning('refund check failed')- Approval interrupts. An interrupt isn't an error, so it's now a
BaseException, likeasyncio.CancelledError. A tool'sexcept Exception:can no longer turn an approval request into an error and let the turn continue without approval. - Truncated JSON. When a reply hit the token limit partway through a JSON object, 0.12 could return a partial object that looked valid. Now
response.outputisNoneandresponse.errorreportsINVALID_OUTPUT. - API keys. Per-request keys already belonged in
context.secrets, but some plugins silently dropped or ignored a key left in model config. Every plugin now raises instead. - Media URLs. Ollama no longer reads media from local file paths, and media downloads over 20 MB fail instead of loading into memory.
Errors are safe to serve
When you serve flows over HTTP, errors no longer leak. In 0.12, a served flow sent the client the message and details of any GenkitError, plus the text and stack trace of any other exception. In 0.13, the client sees only what you mark as public, and your own code still sees everything.
@ai.flow()
async def get_order(order_id: str) -> Order:
order = await db.find(order_id) # a DB or provider failure: client gets 500 Internal Error,
# your logs and except blocks get the full GenkitError
if order is None:
raise PublicError('NOT_FOUND', f'Order {order_id} not found') # client gets 404 and this message
return orderProvider errors, internal details and stack traces stay on your server, and you decide exactly which messages reach callers (see section 8 below).
Consistent concepts and names
The data types you pass around now share names with the other SDKs, mostly Genkit Go: model and embedder refs (ModelRef, EmbedderRef, BackgroundModel), model metadata (ModelSupports, ModelStage, GenerationUsage), evaluator results (EvaluatorResponse, ScoreStatus, ScoreDetails), Formatter, ToolContext and MultipartToolResponse. What you read in one SDK's docs maps onto these. The parts you write code against are Python's own: flow and model contexts, Failure and GenkitError, middleware classes, and plugins as classes with keyword arguments and refs like GoogleAI.gemini(...). Every provider plugin follows the same constructor, config and error conventions, so switching providers is a small, predictable change.
# Python 0.13
from genkit import MultipartToolResponse, ToolContext
from genkit.model import ModelRef, ModelSupports, GenerationUsage, BackgroundModel
from genkit.embedder import EmbedderRef
from genkit.evaluator import EvaluatorResponse, ScoreStatus
from genkit.formats import Formatter// Go: the same names, all in package ai
import "github.com/firebase/genkit/go/ai"
ai.ModelRef ai.ModelSupports ai.GenerationUsage ai.BackgroundModel
ai.EmbedderRef ai.EvaluatorResponse ai.ScoreStatus
ai.Formatter ai.ToolContext ai.MultipartToolResponseTyped config, with one escape hatch: extra
Every config class now declares its fields and rejects anything else. For provider settings a plugin doesn't model yet, every provider plugin has the same escape hatch, extra. Its contents go to the provider under the provider's own field names, merged into the request Genkit builds, and they win on conflict.
# 0.12: an unknown key was silently dropped or forwarded, depending on the plugin,
# so a typo and a real provider option looked the same
config={'temprature': 0.2, 'reasoning': {'effort': 'low'}}
# 0.13: declared fields are typed and checked; provider-only fields are explicit
config={'temperature': 0.2, 'extra': {'reasoning': {'effort': 'low'}}}- A typo can no longer pass for a provider option. It raises before the request is sent.
- A new provider feature works the day it ships, without waiting for a plugin release.
- It's the same pattern in every provider plugin: Google GenAI (Gemini, Veo, Interactions), OpenAI, Anthropic, Ollama and Bedrock. Each plugin's README says where
extralands in its request.
API changes
Every public API change in 0.13, grouped by area. Each one shows the 0.12 code and what replaces it.
1. One home for every symbol
The public API is now a small set of namespaces. Each one's __init__.py lists exactly what it exports, each is stripped down to its essentials, and every public symbol lives in exactly one of them. You, and your coding agent, can predict where an import lives without searching.
from genkit import Genkit, Message, Part, FlowContext, GenkitError # building an app
from genkit.model import Model, ModelRef, ModelRequest # defining models
from genkit.embedder import Embedder, EmbedderRef # defining embedders
from genkit.evaluator import Evaluator, Score # writing evaluators
from genkit.middleware import Middleware, MiddlewareContext # writing middleware
from genkit.formats import Formatter # custom output formats
from genkit.plugin_api import Plugin, Action, GenkitConfig # writing plugins
from genkit.web import RequestData, to_sse # serving flows
from genkit.telemetry import Instrumentation # tracing backends
from genkit.testing import define_scripted_model # tests
from genkit.exp.agent import Agent # experimentalThe rule for where a symbol lives follows who uses it:
- App developers import from
genkit. Anything an application calls or handles lives there, including types that plugin authors share with apps, such asMessage,PartandDocument. The top level went from 49 names to 27 (#6390, #6669). - Anyone defining a resource, usually a plugin author, imports from that resource's namespace:
genkit.model,genkit.embedder,genkit.evaluator,genkit.middlewareorgenkit.formats. - Plugin authors get framework primitives from
genkit.plugin_api. - Specialized jobs have their own namespace:
genkit.webfor serving,genkit.telemetryfor tracing backends,genkit.testingfor fake models, andgenkit.expfor experimental APIs. - Plugin packages expose only their top-level module, such as
genkit_google_genai.
Model helpers also lost their _model suffix: GoogleAI.gemini(...), Anthropic.claude(...). The migration guide lists every symbol that moved and where it went (#6486, #6512, #6550, #6662, #6671, #6682, #6690, #6698, #6699, #6766, #6777).
2. Keyword-only arguments
Constructors, and every option after an action's input, are keyword-only. If you see TypeError: takes N positional arguments, name the argument.
Genkit (#6536)
- ai = Genkit([GoogleAI()], GoogleAI.gemini_model('gemini-flash-latest'))
+ ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest'))Running actions (#6769)
- await my_flow.run(data, on_chunk, ctx)
+ await my_flow.run(data, on_chunk=on_chunk, context=ctx)- VertexAI(project_id='my-proj')
+ VertexAI(project='my-proj') # matches the google-cloud-* clients
- Ollama(server_address='http://localhost:11434')
+ Ollama(base_url='http://localhost:11434') # every plugin takes one base_url=Evaluators (#6672)
- ai.define_evaluator(name='byo/accuracy', fn=accuracy)
+ ai.define_evaluator('byo/accuracy', accuracy) # name and function positional, the rest keyword-only3. Config
Every config class now extends GenkitConfig. Provider config is flat, and an unknown key raises before the model is called. Dict keys may be snake_case or camelCase.
- from genkit_google_genai import GeminiConfigSchema, GeminiImageConfigSchema, GeminiTtsConfigSchema, GemmaConfigSchema
+ from genkit_google_genai import GeminiConfig, GeminiImageConfig, GeminiTtsConfig, GemmaConfigGemini: thinking, search and voice are flat fields (#6593, #6698)
- config={'thinking_config': {'thinking_level': 'HIGH', 'include_thoughts': True}}
+ config=GeminiConfig(thinking_level='HIGH', include_thoughts=True)
- config={'google_search_retrieval': True}
+ config=GeminiConfig(google_search=True)
- config={'speech_config': {'voice_config': {'prebuilt_voice_config': {'voice_name': 'Kore'}}}}
+ config=GeminiTtsConfig(voice='Kore')Anthropic: one flat config (#6596, #6699)
- config={'thinking': {'enabled': True, 'budgetTokens': 1024}}
+ config=AnthropicConfig(thinking='enabled', thinking_budget=1024)
+ # effort, task_budget and user_id are flat too
+ config=AnthropicConfig(thinking='adaptive', effort='high', task_budget=20000, user_id='diner-42')Tool choice moves to generate (#6699)
- config={'tool_choice': {'type': 'any'}}
+ await ai.generate(prompt=p, tools=[...], tool_choice='required') # 'auto' | 'required' | 'none'Provider settings without a field go in extra (#6634, #6644, #6646)
- config={'some_new_provider_option': 1}
+ config=BedrockConfig(extra={'some_new_provider_option': 1})options= is now config=, like generate (#6569)
ai.generate has always taken config=. ai.embed and ai.embed_many called the same thing options=. Now every call that configures a model or embedder uses config=.
await ai.generate(prompt=p, config={'temperature': 0.2}) # unchanged
- await ai.embed(embedder=e, content=doc, options={'task_type': 'RETRIEVAL_DOCUMENT'})
+ await ai.embed(embedder=e, content=doc, config={'task_type': 'RETRIEVAL_DOCUMENT'})How config layers combine (#6697, #6712, #6714, #6978)
Config for a call can come from four layers. Later layers win, one key at a time:
- the config class's defaults
- the model ref:
GoogleAI.gemini('...', config=...) - the prompt:
define_prompt(config=...)or a.promptfile - the call:
config=
A layer only overrides the keys it sets. Everything else passes through from the layer below.
gemini = GoogleAI.gemini('gemini-flash-latest', config=GeminiConfig(thinking_level='HIGH', temperature=0.7))
recipe = ai.define_prompt(name='recipe', model=gemini, config={'temperature': 0.4}, prompt='...')
await recipe(config={'max_output_tokens': 512})
# sent: thinking_level='HIGH' (ref), temperature=0.4 (prompt), max_output_tokens=512 (call)
await recipe(config={'thinking_level': None, 'thinking_budget': 1024})
# None removes the inherited thinking_level, so it doesn't clash with thinking_budgetNoneis a value. It clears a key inherited from a lower layer. To inherit a key, leave it out.- Each layer is checked as you pass it, so a typo fails at that call. Rules that span fields, like
thinking_levelwiththinking_budget, are checked once on the merged config, after middleware runs. - Switching a prompt to another model: set a prompt-level key to
Noneon the call, and it's dropped before the new model sees it.
Typos raise now (#6587, #6675)
await ai.generate(prompt=p, config={'temprature': 0.2})
# raises GenkitError (INVALID_ARGUMENT) naming 'temprature'; the model is never calledAlso removed: EmbeddingTaskType, VertexAIEvaluationMetricType (pass the string value), Anthropic's ThinkingConfig / OutputConfig / TaskBudget / RequestMetadata / *ToolChoice, and OpenAIConfig.user.
4. API keys
Per-request keys already belonged in context.secrets; Google GenAI read them from there in 0.12. What changes in 0.13 is that every plugin now fails loudly when a key is in model config. Before, Ollama silently dropped it, and Anthropic could ignore it with only a log warning, so a request might quietly run on the wrong key or none at all. Now Google GenAI, OpenAI, Anthropic and Ollama all raise before anything is sent.
- await ai.generate(model=m, prompt=p, config={'api_key': user_key}) # now raises
+ # one key for the whole app
+ ai = Genkit(plugins=[OpenAI(api_key=os.environ['OPENAI_API_KEY'])])
+ # or a different key per request
+ await ai.generate(model=m, prompt=p, context={'secrets': {'api_key': user_key}})Vertex AI Model Garden authenticates with ADC and rejects a per-request key. ai.embed() takes no context, so embedders always use the plugin's key (#6595, #6596, #6597, #6643, #6644).
5. Typed contexts
ActionRunContext splits into one typed context per kind of action, and the chunk type moves into the context's type parameter. Genkit finds the context by its annotation.
- from genkit import ActionRunContext
- @ai.flow(chunk_type=Status)
- async def run(n: int, ctx: ActionRunContext) -> str:
+ from genkit import FlowContext
+ @ai.flow()
+ async def run(n: int, ctx: FlowContext[Status]) -> str:
ctx.send_chunk(Status(step='search'))Tools (which can now stream too) (#6314)
- from genkit import ToolRunContext
- async def prepare(dish: str, ctx: ToolRunContext) -> str:
+ from genkit import ToolContext
+ async def prepare(dish: str, ctx: ToolContext[Progress]) -> str:
+ ctx.send_chunk(Progress(stage='searing', percent=40))Models you implement (#6768)
- from genkit import ActionRunContext
- async def generate(request: ModelRequest, ctx: ActionRunContext) -> ModelResponse:
+ from genkit.model import ModelContext
+ async def generate(request: ModelRequest, ctx: ModelContext) -> ModelResponse:The new contexts subclass ActionRunContext, so an old annotation still type-checks. Only from genkit import ActionRunContext breaks.
6. Cancellation
Python already has a cancellation model: asyncio delivers cancellation to whatever code is running, try/finally cleans up, and every async library follows the same rules. 0.12 layered a second, JS-style mechanism on top that your code had to opt into. 0.13 drops it and uses the one Python developers already know.
The generate loop switches back and forth between framework code and code you supply, such as tools and middleware. In 0.12, cancelling it followed the JS pattern: Genkit set an abort signal, and your code had to check it. A tool that didn't check kept running after the caller gave up. Its side effects still happened, and its result was thrown away.
In 0.13, cancellation is native asyncio cancellation. Cancelling a call raises asyncio.CancelledError at whatever await your code is paused on, so your tools stop without checking anything. ctx.abort_signal is removed.
That changes how you write tools. Instead of polling for an abort, put cleanup in finally, which runs whether the tool finishes, fails or is cancelled.
@ai.tool()
- async def book_trip(trip: Trip, ctx: ToolRunContext) -> Booking:
- hold = await reserve_seats(trip)
- if ctx.abort_signal.is_set(): # easy to forget, and only checked here
- await release(hold)
- raise GenkitError(status='ABORTED', message='aborted')
- return await confirm(hold)
+ async def book_trip(trip: Trip) -> Booking:
+ hold = await reserve_seats(trip)
+ try:
+ return await confirm(hold) # cancellation can land at any await
+ finally:
+ if not hold.confirmed:
+ await release(hold)If you catch asyncio.CancelledError yourself, re-raise it. Swallowing it keeps the work running.
All of these cancel the whole call, including the model request, any Retry backoff, and every running tool:
task = asyncio.create_task(ai.generate(prompt=p))
task.cancel() # cancel a call
stream = my_flow.stream(x)
async with stream: # leaving the block cancels the flow
async for chunk in stream:
if enough(chunk):
breakA served flow is also cancelled when the client disconnects from its SSE stream. When one tool in a turn fails, Genkit cancels the other tools in that turn and waits for their finally blocks to finish (#6685, #6952, #7001, #7056).
7. Errors
There are two kinds of error, and they serve different purposes:
Failureis a returned value. When a generation or agent turn fails at runtime, the call doesn't raise. You get the failure back onres.errorand inspect it.GenkitErroris the exception you catch. When a call raises, you handle it withtry/except GenkitError.
Returned failures (#6572, #6767)
- from genkit import RuntimeErrorReason
+ from genkit import ErrorReason, Failure # Failure is now a public, importable type
res = await ai.generate(prompt=p)
if res.error: # same fields: status, message, details, reason
- if res.error.reason == RuntimeErrorReason.INVALID_OUTPUT: ...
+ if res.error.reason == ErrorReason.INVALID_OUTPUT: ...
+ def report(err: Failure) -> None: ... # you can annotate with it now except GenkitError as e:
- log(e.original_message, e.http_code, e.trace_id, e.cause)
+ log(e.message, e.status, e.__cause__)
- from genkit import RuntimeErrorReason
+ from genkit import ErrorReasonInterrupts are no longer Exceptions (#6763)
An interrupt isn't an error. It's a tool pausing to ask for something, such as human approval. Python already uses BaseException for signals like this (KeyboardInterrupt, asyncio.CancelledError), so that except Exception: handlers written for failures don't swallow them. Tools often wrap their work in except Exception: to report errors back to the model. In 0.12, that same handler also caught the interrupt, so an approval request could turn into an error message and the turn would carry on without approval.
@ai.tool()
async def transfer(req: Transfer) -> dict:
try:
approve_or_interrupt(req) # raises Interrupt
except Exception:
... # no longer catches Interrupt, which is now a BaseExceptionIf you catch interrupts on purpose, catch Interrupt by name.
Plugin exception types are gone (#6710)
- except GenkitError as e:
- if isinstance(e.cause, OllamaConnectionError): ...
+ except GenkitError as e:
+ if e.status == 'UNAVAILABLE': ...8. Errors sent to HTTP clients
When you serve flows over HTTP, with Flask, FastAPI, Django or genkit.web, errors now cross the wire safely by default. Inside your process, nothing is hidden (#6499).
In 0.12, a served flow sent the client the message and details of any GenkitError, plus the text and stack trace of any other exception. Whoever called your endpoint could see provider error messages, internal details and stack traces.
0.13 treats the two audiences differently:
- Your code, in process, still sees the full error: a
GenkitErrorwithstatus,message,detailsand__cause__, or your own exception, unwrapped. Logs, tests, middleware and the Dev UI see everything. - The HTTP client sees only what you mark as public. Raise
PublicErrorwith a status and a message written for the caller, and the client gets that message with the matching HTTP status, such as 404. Anything else becomes a 500 with{"status": "INTERNAL", "message": "Internal Error"}.
from genkit import PublicError
@ai.flow()
async def get_order(order_id: str) -> Order:
order = await db.find(order_id) # if this times out, the client sees 500 Internal Error
if order is None:
raise PublicError('NOT_FOUND', f'Order {order_id} not found') # the client sees 404 and this message
return orderA provider failure, such as an expired API key, is UNAUTHENTICATED in your process but reaches the client as a 500. A 401 there would hint at your credentials. Streaming endpoints follow the same rule for their error event. A PublicError raised in a context provider is handled the same way, which is how auth returns a 401 or 403 the caller can act on.
Invalid input to a served flow still returns a 400 that names the failing fields, without the values the caller sent. Every failure is still logged on the server with the flow's name: 5xx errors with the full traceback, 4xx errors as warnings. For your own routes, genkit.web.error_body() and error_status() apply the same rules.
9. Serving and telemetry
Serving types now live in genkit.web, and the Flask, FastAPI and Django adapters all behave the same way. Telemetry is pluggable OpenTelemetry.
Context providers (#6534, #6709)
- from genkit.plugin_api import RequestData
- async def auth(req: RequestData[HttpRequest]) -> dict:
- return {'user': req.request.META.get('HTTP_AUTHORIZATION')}
+ from genkit.web import RequestData
+ async def auth(req: RequestData) -> dict:
+ return {'user': req.headers.get('authorization')} # also req.method, req.inputGoogle Cloud telemetry (#6591, #6630, #6633)
- add_gcp_telemetry(project_id='my-project', force_dev_export=True)
+ enable_google_cloud_telemetry(project='my-project')
+ # export a sample instead of every trace
+ enable_google_cloud_telemetry(project='my-project', sampler=ParentBased(TraceIdRatioBased(0.1)))The FIREBASE_PROJECT_ID fallback is gone; set project= or GOOGLE_CLOUD_PROJECT.
Custom exporters (#6132)
- from genkit.plugin_api import tracer, add_custom_exporter, AdjustingTraceExporter
+ from genkit.telemetry import Instrumentation, configure_instrumentationNew in this release: genkit.web.to_sse() for custom streaming routes, genkit_django_handler, and the genkit-otel package, whose GenAiInstrumentation emits OpenTelemetry GenAI spans and metrics (#6388, #6655).
10. Provider plugins
Each plugin package exposes only its top-level module and takes model names as plain strings. Config changes are in section 3.
Google GenAI (#6300, #6305, #6363, #6826, #7049)
- GoogleAI.embedding('gemini-embedding-001')
+ GoogleAI.embedder('gemini-embedding-001') # VertexAI too
- GoogleAI.imagen_model('imagen-4.0-generate-001')
+ GoogleAI.gemini_image('gemini-2.5-flash-image') # the Imagen catalog is retired
+ # new client options
+ GoogleAI(headers={...}, timeout=30, max_retries=2, http_client_factory=make_client)
+ await ai.generate(prompt=p, config=GeminiConfig(timeout=30)) # per-request, in secondsVertex AI Model Garden (#6500, #6594)
- from genkit_vertexai.model_garden import ModelGardenPlugin
- plugins=[ModelGardenPlugin()]
+ from genkit_vertexai import ModelGarden
+ plugins=[ModelGarden()]uv add 'genkit-vertexai[anthropic]' # Claude
uv add 'genkit-vertexai[openai]' # Llama, Mistral and other OpenAI-compatible models- from genkit_openai import openai_model
- model=openai_model('gpt-4o')
+ model='openai/gpt-4o' # KnownGpt and genkit_openai.models.* are gone tooOllama (#6637, #6643, #6706, #6707)
- from genkit_ollama import EmbeddingDefinition, ModelDefinition, Ollama
- Ollama(
- models=[ModelDefinition(name='llama3.2')],
- embedders=[EmbeddingDefinition(name='nomic-embed-text')],
- server_address=os.getenv('OLLAMA_HOST'),
- )
+ from genkit_ollama import Ollama
+ Ollama(base_url=os.getenv('OLLAMA_HOST')) # reads pulled models and capabilities from the serverOllama no longer reads media from local file paths.
Amazon Bedrock (#6644, #6706, #6977)
- from genkit_amazon_bedrock import Bedrock, ModelDefinition, BedrockImageConfig
- Bedrock(models=[ModelDefinition(name='us.amazon.nova-lite-v1:0')])
+ from genkit_amazon_bedrock import Bedrock, StabilityImageConfig
+ Bedrock(models=['us.amazon.nova-lite-v1:0'])Flask (#6631)
# genkit-flask no longer installs genkit-google-genai for you
uv add genkit-google-genaiFor every provider, media URLs over 20 MB now fail instead of downloading (#6637).
Evaluators: last release before the redesign (#7029)
We're redesigning evaluators as a concept across all the Genkit SDKs, so 1.0, the next release, removes the current evaluator APIs and the genkit-evaluators package. We kept them in 0.13 on purpose. Until now, plugins depended on genkit with no upper bound, so removing something from genkit could leave pip or uv pairing an older plugin with a genkit it can't run on. 0.13 is the first release where every plugin declares the range of genkit versions it supports, and genkit-evaluators 0.13.0 requires genkit<1. That gives the removal a clean boundary: installs that pin the 0.13 line keep working, and the next release can drop evaluators without breaking anyone's resolver. If you rely on evaluators, stay on 0.13 until the new design ships.
Behavior changes
These run without edits, but they behave differently.
Templates only render in prompts (#6553)
await ai.generate(prompt='Say hi to {{name}}') # now sent literally
greet = ai.define_prompt(name='greet', prompt='Say hi to {{name}}') # templates belong hereOnly complete JSON becomes output (#6573, #7003)
res = await ai.generate(prompt=p, output_schema=City)
res.output # None, with an INVALID_OUTPUT error, if the reply was cut offchunk.output never raises while streaming.
Prompt calls take generate's keyword names (#6539, #6590)
recipe = ai.prompt('recipe')
await recipe({'dish': 'pie'}, config={'temperature': 0.2}) # merges over the prompt's config
# the output schema is fixed by the prompt and can't be overridden per callPrompt input is validated, .prompt file defaults are filled in, and file schemas return the registered class.
Flow input and output are validated (#6575)
@ai.flow()
async def greet(name: str = 'world') -> str: ...
# a served call with {"data": null} runs with the default
# returning the wrong type now raisesActions raise what their body raised (#6576, #6586)
try:
await my_flow.run(x)
except TypeError: # your own exception arrives unwrapped
...Retry and Fallback only retry failures a plugin has classified, such as UNAVAILABLE.
Smaller changes
await tool(x)returns what the function returned (#6681).- A
Messagenested in your own Pydantic model serializes like it does on its own: camelCase, no nulls (#6559). - Thinking signatures are stored under
thoughtSignaturefor every provider (#6588). - Generation returns once it reaches
max_turns(#6501). response.outputis typedOutputT | None, for type checkers only (#6537).
Plugin and middleware authors
Skip this if you only call Genkit. genkit.plugin_api is now a small surface, and builder functions became classes.
- from genkit.middleware import BaseMiddleware, GenerateMiddlewareContext
- @ai.middleware(name='pii_redact')
- class PiiRedact(BaseMiddleware[Cfg]):
- async def model(self, params, ctx: GenerateMiddlewareContext, next_fn): ...
+ from genkit.middleware import Middleware, MiddlewareContext
+ @ai.middleware
+ class PiiRedact(Middleware[Cfg]):
+ name = 'pii_redact'
+ async def model(self, params, ctx: MiddlewareContext, next_fn): ...A middleware's tools() returns Tool handles. GenerateMiddleware, MiddlewareRef, MiddlewarePlugin and new_middleware are gone.
Defining models and embedders (#6697, #6757, #6777)
- from genkit.plugin_api import model, model_ref, background_model, embedder, embedder_ref
+ from genkit.model import Model, ModelRef, BackgroundModel
+ from genkit.embedder import Embedder, EmbedderRef
- from genkit.plugin_api import ModelConfig
+ from genkit.plugin_api import GenkitConfig # base for every config classResolving models (#6574)
async def resolve(self, kind, name):
- model_id = name.removeprefix('myplugin/')
+ model_id = name # resolve() now gets the id without the plugin prefixProvider errors (#6586, #6710)
- from genkit.plugin_api import wrap_http_error, from_http_code, parse_retry_after_ms
- raise wrap_http_error(resp)
+ from genkit.plugin_api import provider_error
+ raise provider_error(...) # classifies the failure so Retry and Fallback can act on itClients, headers and logging (#6674, #6764, #7033)
- from genkit.plugin_api import get_cached_client, GENKIT_CLIENT_HEADER, GENKIT_VERSION
+ from genkit.plugin_api import loop_local_client, GENKIT_USER_AGENT
+ from genkit import get_loggerTracing (#6132)
- from genkit.plugin_api import tracer, add_custom_exporter, AdjustingTraceExporter, RedactedSpan
+ from genkit.telemetry import Instrumentation, configure_instrumentationAlso (#6562, #6689, #6757, #6765, #7049): Plugin.model() / Plugin.embedder() and Supports.context are removed. generate_operation receives only what a background start uses, and BackgroundModel subclasses the generic BackgroundAction. Custom formatters import Formatter, FormatterConfig and FormatHandler from genkit.formats. New: genkit.evaluator.Evaluator / BatchEvaluator, and fake models in genkit.testing.
Experimental APIs
Anything under an exp module can change in any release, and it stays experimental after 1.0 (#6549, #6571, #6649).
- from genkit.exp import Agent, Session, InMemorySessionStore, AgentError
+ from genkit.exp.agent import Agent, Session, InMemorySessionStore
- from genkit_fastapi import serve_agent
+ from genkit_fastapi.exp import serve_agent
- from genkit_middleware import Artifacts
+ from genkit_middleware.exp import ArtifactsAgentError is gone: a failed turn returns res.error (a Failure), and a call that produces no turn raises GenkitError. A detached turn is cancelled with await task.abort().