xAI, DeepSeek, DashScope, Kimi, Z.ai, and OpenRouter join the OpenAI-compatible family, all of it rebuilt on typed per-provider configs that the framework validates before a request is billed. Failures now carry a status from the line that raised them through the retry middleware to the HTTP response, and provider SDK errors arrive already classified. Logs attach to the span that produced them. Prompt content functions are typed against the prompt's own input. Options that used to reject a repeat now merge.
go get github.com/firebase/genkit/go@v1.12.0Six providers join the OpenAI-compatible core
Genkit Go ships plugins for xAI, DeepSeek, DashScope (Qwen), Kimi, Z.ai (GLM), and OpenRouter. They sit beside openai and the OpenAI-compatible anthropic plugin on a rebuilt compat_oai core.
Each of the six declares a ChatConfig covering exactly the fields its provider documents: Kimi's K-series takes no temperature, Z.ai caps it at 1, DeepSeek carries a user_id that partitions its context cache. The plugin advertises the JSON schema inferred from that struct and the framework enforces it at the action boundary, so an out-of-range value fails before the request is billed, and the Dev UI renders the same schema as a form.
g := genkit.Init(ctx, genkit.WithPlugins(&deepseek.DeepSeek{})) // DEEPSEEK_API_KEY
model := deepseek.ModelRef("deepseek-v4-pro", &deepseek.ChatConfig{
ReasoningEffort: deepseek.ReasoningEffortMax,
})g := genkit.Init(ctx, genkit.WithPlugins(&kimi.Kimi{})) // KIMI_API_KEY
model := kimi.ModelRef("kimi-k3", &kimi.ChatConfig{
Thinking: &kimi.ThinkingConfig{Type: kimi.ThinkingTypeEnabled},
})g := genkit.Init(ctx, genkit.WithPlugins(&xai.XAI{})) // XAI_API_KEY
model := xai.ModelRef("grok-4.6", &xai.ChatConfig{
ReasoningEffort: xai.ReasoningEffortXHigh,
})g := genkit.Init(ctx, genkit.WithPlugins(&zai.ZAI{})) // ZAI_API_KEY
model := zai.ModelRef("glm-5.1", &zai.ChatConfig{
Thinking: &zai.ThinkingConfig{Type: zai.ThinkingTypeEnabled},
})g := genkit.Init(ctx, genkit.WithPlugins(&dashscope.DashScope{})) // DASHSCOPE_API_KEY
// openai.Ptr is the openai-go SDK helper for optional fields.
model := dashscope.ModelRef("qwen-plus", &dashscope.ChatConfig{
EnableThinking: openai.Ptr(true),
ThinkingBudget: openai.Ptr(2048),
})OpenRouter reaches the rest
OpenRouter fronts hundreds of models from dozens of vendors. The plugin curates nothing: you name a model, and the ID keeps its upstream vendor prefix, which puts two slashes in an action name such as openrouter/openai/gpt-5. The gateway controls are typed at the call site. Choose which providers may serve the request, chain fallback models, set reasoning effort.
g := genkit.Init(ctx, genkit.WithPlugins(&openrouter.OpenRouter{})) // OPENROUTER_API_KEY
resp, err := genkit.Generate(ctx, g,
ai.WithModel(openrouter.ModelRef("openai/gpt-5", &openrouter.ChatConfig{
// Try these in order if gpt-5 is unavailable.
Models: []string{"anthropic/claude-sonnet-4.5", "deepseek/deepseek-v4-pro"},
Provider: &openrouter.ProviderRouting{
Sort: openrouter.ProviderSortThroughput,
DataCollection: openrouter.DataCollectionDeny,
},
Reasoning: &openrouter.ReasoningConfig{Effort: openrouter.ReasoningEffortHigh},
})),
ai.WithPrompt("Work through this step by step."),
)The family reads a provider's non-standard reasoning field, reasoning_content first and reasoning second, back as a Genkit reasoning part, so resp.Reasoning() covers DeepSeek, Kimi, and OpenRouter alike. A gateway that reports what it charged puts the figure under the cost key of Usage.Custom. Test for the key rather than a nonzero value: a free-tier request is priced at an explicit zero.
fmt.Println(resp.Reasoning())
if cost, ok := resp.Usage.Custom["cost"]; ok {
fmt.Printf("this answer cost %.5f\n", cost)
}Google and Claude, closer to the metal
Vertex AI Express Mode authenticates with an API key alone: no project, no location, no ADC. Both Google backends take BaseURL, Headers, and HTTPClient, so a proxy or a custom transport is a struct field, and GoogleAI takes APIVersion as well. Client() hands back the genai SDK client for Files, Caches, Batches, and Tunings. When a 429 names its own backoff, RetryDelay reads it.
gemini := &googlegenai.GoogleAI{
APIVersion: "v1alpha",
Headers: http.Header{"X-Team": {"platform"}},
}
// Express Mode: a Vertex AI key, no project, no location, no ADC.
vertex := &googlegenai.VertexAI{APIKey: "YOUR_VERTEX_API_KEY"}
g := genkit.Init(ctx, genkit.WithPlugins(gemini, vertex))
client, err := gemini.Client()
if err != nil {
return err
}
file, err := client.Files.UploadFromPath(ctx, "photo.jpg", &genai.UploadFileConfig{
MIMEType: "image/jpeg",
})
_, err = genkit.Generate(ctx, g,
ai.WithModelName("googleai/gemini-flash-latest"),
ai.WithPrompt("Explain how neural networks learn."),
)
if err != nil {
if delay, ok := googlegenai.RetryDelay(err); ok {
time.Sleep(delay) // the service named its own backoff
}
return err
}On the native Claude plugin, Opts carries anthropic-sdk-go request options into the client: retries, timeouts, middleware, and the SDK's Bedrock and Vertex routing helpers. ModelRef binds a *anthropic.MessageNewParams to a model ID, so thinking, effort, and server-side tools are typed at the call site.
g := genkit.Init(ctx, genkit.WithPlugins(&anthropic.Anthropic{
Opts: []option.RequestOption{option.WithMaxRetries(5)},
}))
resp, err := genkit.Generate(ctx, g,
ai.WithModel(anthropic.ModelRef("claude-sonnet-5", &sdk.MessageNewParams{
MaxTokens: 4000,
Thinking: sdk.ThinkingConfigParamUnion{
OfAdaptive: &sdk.ThinkingConfigAdaptiveParam{},
},
OutputConfig: sdk.OutputConfigParam{Effort: sdk.OutputConfigEffortHigh},
})),
ai.WithPrompt("Plan the migration."),
)Every model plugin, Google and Claude included, takes a Models map keyed by model ID that overrides the capabilities the plugin resolves. Fields left at zero keep what the plugin already knows, so one entry describes a model released after the plugin without forking it.
Classify a failure once, and it stays classified
Genkit has one error type and one status vocabulary: InvalidArgument, NotFound, PermissionDenied, Unauthenticated, ResourceExhausted, Unavailable, DeadlineExceeded, Internal, and nine more. The names follow the Google API error model and mean the same thing in the JS and Python runtimes. Classify a failure at the point where you know what it is.
// Keeps NOT_FOUND from its parent, and matches both itself and status.ErrNotFound.
var ErrRecipeNotFound = status.ErrNotFound.Subtype("recipe not found")
func lookupRecipe(dish string) (string, error) {
recipe, ok := cookbook[dish]
if !ok {
return "", status.PublicErrorf(ErrRecipeNotFound, "no recipe for %q", dish)
}
return recipe, nil
}
func plateUp(dish string) (string, error) {
recipe, err := lookupRecipe(dish)
if err != nil {
return "", fmt.Errorf("plating %q: %w", dish, err)
}
return recipe, nil
}Subtype derives a sentinel that keeps its parent status. Errorf builds a message for your logs, PublicErrorf marks one safe to hand a caller. Adding context with %w changes nothing else: the sentinel, the status, and the public message all survive the trip up the stack. No re-wrapping, no matching on message text.
switch {
case errors.Is(err, ErrRecipeNotFound): // this exact failure
case errors.Is(err, status.ErrNotFound): // anything missing
case errors.Is(err, status.ErrResourceExhausted): // rate limited or out of quota
}
status.Of(err) // status.NotFound
status.Of(err).HTTPCode() // 404
msg, public := status.PublicMessage(err) // `no recipe for "lasagna"`, trueCode you did not write reads the classification
Serve the flow with genkit.Handler and the response code comes from the classification. The message only leaves the process when you built it with PublicErrorf.
genkit.DefineFlow(g, "recipe", func(ctx context.Context, dish string) (string, error) {
if dish == "" {
// 400, body: dish must not be empty
return "", status.PublicErrorf(status.ErrInvalidArgument, "dish must not be empty")
}
if dish == "escargot" {
// 400, body: invalid argument
return "", status.Errorf(status.ErrInvalidArgument, "supplier for %q is offline", dish)
}
// 500, body: internal
return "", errors.New("db at 10.0.0.3: password rejected")
})Everything else is redacted to the generic label for its status, and the full text goes to the server log. Set GENKIT_ENV=dev and the real message comes back instead; the code is the same either way.
The reach of that classification is what changed. Every plugin now classifies what its provider SDK returns, so a 401 from Anthropic or a 429 from Gemini arrives already carrying Unauthenticated or ResourceExhausted. Retry and Fallback have always read a classification and disagreed on purpose: Retry reissues a ResourceExhausted or Unavailable call, leaves an InvalidArgument alone, and retries an unclassified error because a dial timeout deserves another attempt, while Fallback propagates an unclassified error rather than spending a second billed model on it. What is different is that provider failures now reach them classified instead of opaque.
resp, err := genkit.Generate(ctx, g,
ai.WithModelName("googleai/gemini-flash-latest"),
ai.WithPrompt("Draft the menu."),
ai.WithUse(
&middleware.Retry{MaxRetries: 3},
&middleware.Fallback{Models: []ai.ModelRef{
ai.NewModelRef("googleai/gemini-3.5-flash", nil),
}},
),
)Logs that land on the trace
Every core/logger call takes a context first and carries the active span, so a line written inside a flow attaches to that run instead of scrolling past in stdout. Attributes are structured, and a status classification is just one more attribute.
genkit.DefineFlow(g, "summarize", func(ctx context.Context, doc string) (string, error) {
logger.Info(ctx, "summarizing", "chars", len(doc))
resp, err := genkit.Generate(ctx, g,
ai.WithModelName("googleai/gemini-flash-latest"),
ai.WithPrompt("Summarize: %s", doc))
if err != nil {
logger.Warn(ctx, "generate failed", "status", status.Of(err), "err", err)
return "", err
}
logger.Debug(ctx, "summarized", "chars", len(resp.Text()))
return resp.Text(), nil
})Genkit logs through that same path, so the stream is full before you write a line of your own: span start and finish with state and duration, resolved generate requests, each model turn with its finish reason and token counts, tool batches, and middleware hooks with their duration and whether they short-circuited.
Console verbosity and span correlation are separate knobs. Quiet the terminal and the debug narrative still reaches the trace. Attributes bound to a context flow downstream with no extra plumbing.
// The terminal stays at warn; the trace keeps every debug record.
logger.SetLevel(slog.LevelWarn)
ctx = logger.WithContext(ctx, logger.FromContext(ctx).With("requestId", requestID))
logger.Debug(ctx, "request accepted")Strings, ints, and bools travel natively; other values render to text. Per-span log inspection arrives in the Developer UI shortly, listing these records beside the trace they belong to.
Fill a prompt's slots from one typed input
A prompt's content functions are generic over its input type. WithSystemFn, WithMessagesFn, and WithPromptFn take func(context.Context, In) (...), so the compiler checks each one against the type WithInputType declares, whether the call arrives from Go, from the Dev UI, or over HTTP. The system and user prompt slots also take []*ai.Part, static through WithSystemParts and WithPromptParts or computed through WithSystemPartsFn and WithPromptPartsFn. WithDocsFn resolves context documents from that same input, so retrieval lives in the prompt definition instead of at each call site.
support := genkit.DefinePrompt(g, "support",
ai.WithModelName("googleai/gemini-flash-latest"),
ai.WithInputType(SupportRequest{Tier: "free"}),
ai.WithSystemFn(func(ctx context.Context, in SupportRequest) (string, error) {
if in.Tier == "enterprise" {
return "You are a support agent. Be thorough, and offer to escalate.", nil
}
return "You are a support agent. Answer from the reference material.", nil
}),
ai.WithMessagesFn(func(ctx context.Context, in SupportRequest) ([]*ai.Message, error) {
turns := ai.HistoryFromContext(ctx)
if len(turns) > 6 {
turns = turns[len(turns)-6:]
}
return turns, nil
}),
ai.WithPromptPartsFn(func(ctx context.Context, in SupportRequest) ([]*ai.Part, error) {
parts := []*ai.Part{ai.NewTextPart(in.Question)}
if in.Screenshot != "" {
parts = append(parts, ai.NewMediaPart("image/png", in.Screenshot))
}
return parts, nil
}),
ai.WithDocsFn(func(ctx context.Context, in SupportRequest) ([]*ai.Document, error) {
return retrieve(ctx, in.Area)
}),
)
resp, err := support.Execute(ctx,
ai.WithInput(SupportRequest{Area: "billing", Tier: "pro", Question: "Why {{two}} charges?"}),
ai.WithMessages(history...),
)Whatever a function returns is sent verbatim. Only WithSystem, WithPrompt, and WithMessagesTemplate compile as templates, so a customer question carrying {{#if}} reaches the model as written.
Where the conversation lands
The prompt places the messages passed to Execute, by one of three rules:
- The prompt declares no conversation: they sit between the system message and the user prompt.
- The prompt declares
WithMessagesorWithMessagesFn: the prompt owns them, and reads them withai.HistoryFromContextto trim, summarize, or reorder before returning them. A prompt that carries few-shot examples this way receives the caller's history only when it asks for it. - The prompt declares
WithMessagesTemplate: they land at{{history}}, or, with no marker, just before the template's final user turn.
triage := genkit.DefineDataPrompt[SupportRequest, Triage](g, "triage",
ai.WithModelName("googleai/gemini-flash-latest"),
ai.WithSystem("Classify the {{area}} question from this {{tier}} customer."),
ai.WithMessagesTemplate(`{{role "user"}}Deploys fail with a 401 after I rotated keys.
{{role "model"}}{"category": "bug", "urgency": "high"}
{{history}}`),
ai.WithPrompt("{{question}}"),
)ai.NewHistoryContext is the writing half of that pair, for code that drives Prompt.Render and GenerateWithRequest by hand. Scope it to the render call, never the generate call.
Documents follow a related rule: documents passed to Execute replace the prompt's own and suppress WithDocsFn, so the retriever is never called for a result that would be discarded.
Options that compose instead of collide
Passing an option twice used to fail the call with INVALID_ARGUMENT. Options merge instead, left to right, each by what it means, and applying them cannot fail.
Collections accumulate across repeats: tools, middleware, messages, documents, resources. Single-value slots take the last value set: model, config, system, prompt, output schema.
That is what makes a helper worth writing. Hand back a slice of options and let the caller build on it.
func supportOptions(tools ...ai.ToolRef) []ai.GenerateOption {
return []ai.GenerateOption{
ai.WithModelName("googleai/gemini-flash-latest"),
ai.WithSystem("You are a support agent."),
ai.WithTools(tools...),
ai.WithMiddleware(logging),
}
}The caller extends what should stack and overrides what should not, without knowing which options the helper already set.
opts := append(supportOptions(searchTool),
ai.WithTools(refundTool), // accumulates: the model sees both tools
ai.WithMiddleware(retry), // accumulates, in call order
ai.WithSystem("You are a support agent. Answer in one sentence."), // one slot: last wins
)
resp, err := genkit.Generate(ctx, g, append(opts, ai.WithPrompt("Where is my order?"))...)Two of those slots are shared by four options each. WithSystem, WithSystemParts, WithSystemFn, and WithSystemPartsFn all fill the system message, so the last one set replaces the others rather than adding to them. WithPrompt, WithPromptParts, WithPromptFn, and WithPromptPartsFn work the same way on the user message. The Parts variants build those messages from multimodal parts instead of a string.
One combination is refused instead of merged. ai.WithMessagesTemplate lays out the whole conversation as a template, down to where {{history}} puts the caller's, so messages passed beside it have no position relative to it. Handing DefinePrompt both panics at the call site.
Rows arrive once in a JSONL stream
The JSONL handler hands over the rows that completed since the previous chunk, including a row that finished before its trailing newline arrived. Append every chunk and each completed row lands once, with no dedupe on the caller's side. A trailing row that is still half written is the exception: it arrives as a partial and fills in over later chunks.
for val, err := range genkit.GenerateDataStream[[]Character](ctx, g,
ai.WithModelName("googleai/gemini-flash-latest"),
ai.WithOutputFormat(ai.OutputFormatJSONL),
ai.WithPrompt("Invent four characters for a story about a lighthouse keeper."),
) {
if err != nil {
return err
}
if val.Done {
return nil
}
// Rows that finished since the last chunk. The trailing row, if it is
// still half written, arrives again as it fills in.
for _, c := range val.Chunk {
render(c)
}
}Config arrives typed and already validated
Build a model, embedder, retriever, or evaluator with a constructor that carries a Config type parameter. Genkit derives the config's JSON schema from the Go type, validates every request against it, and hands your function a deserialized value. A struct from an application, a pointer to it, and JSON from the Dev UI all land as the same typed value.
type MyConfig struct {
Temperature float32 `json:"temperature,omitempty"`
MaxTokens int `json:"maxTokens,omitempty"`
}
// ResolveAction builds the model a request names, so the plugin registers
// nothing up front.
func (p *MyPlugin) ResolveAction(atype api.ActionType, id string) api.Action {
if atype != api.ActionTypeModel {
return nil
}
return p.newModel(id)
}
func (p *MyPlugin) newModel(id string) *ai.ModelAction {
return ai.NewModelAction(api.NewName(p.Name(), id), &ai.ModelOptions{
Label: "My Model " + id,
Supports: &ai.ModelSupports{Multiturn: true, Tools: true},
}, func(ctx context.Context, req *ai.ModelRequest, cfg MyConfig, cb ai.ModelStreamCallback) (*ai.ModelResponse, error) {
return callMyAPI(ctx, id, req, cfg, cb)
})
}*ai.ModelAction satisfies api.Action, so a plugin hands it back from ResolveAction, ListActions, or Init with no assertion. A request carrying a key the schema does not declare fails at the action boundary with config: Additional property bogus is not allowed, before the provider is billed. A plugin whose wire contract differs from the reflected one sets ModelOptions.ConfigSchema and keeps the typed parameter.
Applications reach the same constructors through genkit, defined and registered in one call:
genkit.DefineRetrieverAction(g, "local/menuDocs",
&ai.RetrieverOptions{Label: "Menu Docs"},
func(ctx context.Context, req *ai.RetrieverRequest, cfg SearchConfig) (*ai.RetrieverResponse, error) {
return search(ctx, req.Query, cfg.K)
})
docs, err := genkit.Retrieve(ctx, g,
ai.WithRetrieverName("local/menuDocs"),
ai.WithTextDocs("what soup is on today?"),
ai.WithConfig(&SearchConfig{K: 3}))Two audiences, two shapes. A constructor takes identity positionally, one options struct for every descriptor slot, and the implementation function last, so an optional hook lands as a field instead of a signature break. A caller composes variadic With* options. The same shape runs one level down: core.NewActionOf, NewStreamingActionOf, NewBidiActionOf, and NewBackgroundActionOf take the action type first and a single core.ActionOptions covering input, output, and stream schemas.
DefineSchemasFor registers a batch of Go types under their type names, which prompt frontmatter and generate calls then reference by name:
genkit.DefineSchemasFor(g, MenuQuestion{}, MenuAnswer{})
resp, err := genkit.Generate(ctx, g,
ai.WithModelName("googleai/gemini-flash-latest"),
ai.WithOutputSchemaName("MenuAnswer"),
ai.WithPrompt("What is the soup of the day?"))Each name belongs to one type, so run this at startup.
Seventeen samples, one standard
go/samples/basic* is seventeen programs written to one standard. Each opens with a package comment that teaches the concept, names every flow, and gives three ways to run it: go run ., genkit start -- go run . for the Developer UI, and a working curl per flow. Inputs are structs whose jsonschema tags carry a description and a default, so the Developer UI renders a pre-filled form and any flow runs from a browser.
// jsonschema tags give the Developer UI a form to pre-fill.
type JokeRequest struct {
Topic string `json:"topic" jsonschema:"description=What the joke should be about,default=airplane food"`
}
genkit.DefineStreamingFlow(g, "streamingJokesFlow",
func(ctx context.Context, input JokeRequest, sendChunk ai.ModelStreamCallback) (string, error) {
resp, err := genkit.Generate(ctx, g,
ai.WithModelName("googleai/gemini-flash-latest"),
ai.WithPrompt("Share a joke about %s.", input.Topic),
ai.WithStreaming(sendChunk),
)
if err != nil {
return "", fmt.Errorf("could not generate joke: %w", err)
}
return resp.Text(), nil
})Where to look for what:
basic: the two flow shapes, one returning its answer whole and one forwarding the model's chunks.basic-structuredandbasic-formats: typed output throughGenerateDataandGenerateDataStream, then the formats underneath it.jsonstreams a growing value,jsonlhands over each finished row once,enumconstrains the answer to one label.basic-prompts: every prompt defined twice, inline withDefinePromptand as a.promptfile looked up by name, so the pair shows what moves out of code.basic-prompt-contentfills all four content slots from one typed input.basic-media: describe a picture, edit one, generate one, and animate one through a polling background model.basic-tools: a multipart tool that answers with a*Rolloutand an attached latency chart, which reaches the model and the Developer UI both.basic-tool-interrupts: human in the loop. A tool pauses a transfer for approval, and a second turn restarts it with the answer attached.basic-middleware:Retrywrapped aroundFallbackover a deliberately broken model id;Filesystem, whose four file tools are confined toRootDirbyos.Root; andSkills, where the model loads aSKILL.mdbody on demand.basic-agents: six agents in six styles behind one CLI, snapshotting to disk.basic-agents-serverserves agents over plain HTTP, one holding session state on the server and one handing it back to the client.basic-errors: classify once withstatus.Errorf, add context with%w, branch witherrors.Is, and watch the HTTP boundary redact the one failure nobody classified.basic-durable-streaming-exp: drop the connection mid-run, reconnect with the stream ID, and read the buffered chunks before live ones resume.
Tools and interrupts ship twice, once against the stable API and once against the in-preview API in genkit/exp. Same rollout, same approval, so diff between the pair is the lesson.
Smaller things worth knowing
- Custom output formats resolve correctly, so a format registered with
genkit.DefineFormatsis found whenai.WithOutputFormatnames it. - Partial JSON completion closes structures by nesting order rather than by counting braces, so a truncated stream of an array of objects parses into a growing value.
- A blocked or truncated response reaches the caller with its finish reason intact instead of failing as a schema mismatch.
- Ollama reads each local model's capabilities from the server and caches them by digest.
- Anthropic message conversion and tool choice are corrected, and multipart tool responses map to
tool_resultblocks. - Streaming through the OpenAI-compatible family returns the complete conversation from
resp.History(). - A gateway whose upstream fails part-way through a stream now ends it with a classified error rather than handing back a short answer that reads as complete.
- A tool short-circuited by middleware is attributed to that tool in traces.
- Mistral serves from Vertex AI Model Garden through
modelgarden.Mistral.
The documentation caught up
Every Go page on genkit.dev was read against this release rather than patched around it. Snippets compile. Deprecated helpers no longer appear as the primary way to do anything. Output formats, multipart tools, typed prompt content, and the status vocabulary are documented where a reader looks for them instead of only in godoc, and every provider in the OpenAI-compatible family has a page, including the four that never had one.
The sample suite is linked from the sections that teach each concept, so the path from reading about a feature to running it is one click.