π Kargo v1.11.0 has arrived! This release teaches auto-promotion to respect your intention to pin a Stage on older Freight, makes promotions dramatically more responsive, lightens the load on your Git servers, and continues our steady stream of general improvements.
π¨ Breaking Changes
-
Auto-Promotion May Move Pinned Stages on Upgrade (#6334): Before v1.11.0, a Stage with auto-promotion enabled that had been manually pointed at older Freight stayed put only by an incidental behavior: auto-promotion declined to re-promote any Freight for which a Promotion already existed. Auto-promotion holds (see below) replace that incidental protection with an explicit, durable one β but holds are recorded only for Promotions created from v1.11.0 onward, so pre-existing, incidental pins are not protected. On the first reconciliation after upgrade, any Stage previously pinned this way will be auto-promoted to its origin's current auto-promotion candidate. To avoid this, disable auto-promotion for affected Stages before upgrading. To restore a pin after upgrading, simply promote the older Freight again β this time, a hold will be recorded.
-
allowTags/ignoreTagsEnforcement (#6487): As announced in the v1.9.0 release notes, artifact discovery now fails with an error for anyWarehousesubscription still using the deprecatedallowTagsorignoreTagsfields. Migrate toallowTagsRegexesandignoreTagsRegexesbefore upgrading. The deprecated fields will be removed entirely in v1.13.0. -
Corrected OpenAPI Spec and New Generated Go Client (#6647): This one PR fixed two independent problems. First, the REST API's OpenAPI spec (
swagger.json) previously mistyped several Kubernetes types.QuantityandMicroTimefields are now plain strings β matching their actual JSON wire format β andIntOrStringfields are now untyped. If you've generated a client of your own from the spec, regenerate it from the corrected version. Second, and independent of the spec, the go-swagger-generated Go client (pkg/client/generated) has been removed outright and replaced by an OpenAPI Generator-generated client atpkg/x/client/generated: the old generator emitted "phantom" empty objects for optional fields, visible as junk inkargo get ... -o yamloutput and hazardous to anything round-tripping that output, and the new generator does not. As with everything underpkg/x/, Go programs importing the new package should expect no stability guarantees. We recommend upgrading the CLI to v1.11.0 alongside your server. The CLI uses a generated client, and older versions predate both fixes β and, having used the old generator, can emit the phantom-object junk. A v1.10.x CLI remains mostly compatible, but upgrading is the safer choice. -
Removed Chart Values (#6318): The
rbac.useDefaultServiceAccountandrbac.disableAutomountServiceAccountTokenHelm chart values introduced in v1.10.0 have been removed. These existed to support internal plans that have since changed and are unlikely to have been used by anyone else. If you had set either, remove them from your values before upgrading.
ποΈ Deprecations
- Legacy gRPC (ConnectRPC) API β Removal in v1.12.0: This is not a new deprecation, but a reminder of an existing and important one. The Connect-based gRPC API was deprecated back in v1.9.0 in favor of the REST API. With the UI now fully transitioned to the REST API (see UI Improvements below), the legacy API no longer has any first-party consumers and will be removed entirely in v1.12.0. If you maintain integrations built against it, migrate them to the REST API before upgrading to v1.12.0.
βΈοΈ Auto-Promotion Holds
Ever promoted an older piece of Freight to a Stage β only to watch auto-promotion immediately "correct" your correction? Not anymore. When a Promotion you initiate manually succeeds for Freight other than the current auto-promotion candidate, Kargo now records a hold that pauses auto-promotion of other Freight from that Freight's origin to that Stage, thereby keeping the Stage in the state you deliberately restored it to. Holds are scoped to a single origin, so Freight from other origins matching a Stage's other Freight requests continues to auto-promote as usual.
When you're ready to move forward again, just promote the origin's current auto-promotion candidate and auto-promotion resumes. Depending on auto-promotion policy, that's either the newest available Freight or the Freight in use immediately upstream of the Stage.
The UI makes this effortless: Stages with paused auto-promotion display an alert with a Resume button, and a resume action is also available from the Stage's action menu. Either opens a drawer that identifies the held origin and its current auto-promotion candidate, resuming with a single click. (#6628, #6670)
With the CLI, the current auto-promotion candidate can be resolved automatically by just specifying a Warehouse instead of specific Freight:
kargo promote --project my-project --stage test --origin Warehouse/my-warehouseβ‘ Performance & Responsiveness
This release takes a big bite out of the two most common sources of waiting in Kargo: artifact discovery and polling for external conditions.
-
Cheaper Git Discovery: At steady state, most
Warehousereconciliations discover nothing new β yet previously still performed a full clone and history walk per Git subscription. A singlegit ls-remoteround-trip now detects when remote ref state is unchanged and skips the clone entirely, a huge win for large monorepos subscribed to by manyWarehouses. (#6369) -
Blobless Clones Are Back: Blobless clones (
--filter=blob:none) were disabled in a previous release because some Git servers don't support partial clones. They've been restored with a graceful fallback: if a server rejects the filter, Kargo automatically retries without it. (#6014) -
Instant Wake-Up for PR Merges: When a GitHub
pull_requestwebhook reports a PR has closed, Kargo now immediately re-reconciles any running Promotion whosegit-wait-for-prstep is waiting on that PR β reducing detection latency from minutes to near-instant, with polling retained as a fallback. (docs, #5852) -
Event-Driven Wake-Up for Selector-Based Argo CD Steps:
argocd-updateandargocd-waitsteps that targetApplications by label selector (rather than by name) previously progressed only on the periodic reconciliation interval.Applicationstate changes now wake these Promotions through the same event-driven path as name-based steps. (#6401) -
Configurable (and Faster) Polling: The three steps that poll external conditions β
http,git-wait-for-pr, andgit-merge-prβ now accept an optionalpollIntervalin their configurations, and ship with much snappier defaults (30s, 30s, and 10s respectively) than the previous five-minute requeue interval. (#6493)
β‘οΈ Promotion Improvements
-
New
file-writeStep: Renders evaluated content directly into a file β useful for templating arbitrary content without a dedicated step for the format at hand. New YAML and pretty-JSON expression helpers round out the picture. (docs, #6293) -
httpStep: A newerrorExpressionoption extracts a meaningful error message from a failed response and surfaces it in the step result, and a newproxyoption specifies an HTTP proxy for the request, overridinghttps_proxyand related environment variables. (docs, #6414, #6264) -
git-tagStep: A newforceoption permits an existing tag to be moved to the currentHEADβ supporting workflows where a well-known tag tracks the commit from which a GitOps agent syncs. (docs, #6457) -
git-merge-prStep: ThemergeMethodoption introduced in v1.10.0 is now supported for Bitbucket Cloud, and the step's GitHub provider is now resilient to races with concurrent merges to the same base branch, retrying rather than failing the Promotion. (docs, #6315, #6490) -
git-open-prStep: Thelabelsoption is now supported for Gitea. (#6262) -
deleteStep: Thepathoption now accepts glob patterns, so a long series ofdeletesteps that cluttered the promotion view can collapse into a single step. (docs, #6533) -
Freight Alias in Expressions: The human-friendly alias of the Freight being promoted is now accessible as
ctx.targetFreight.aliasin the expression language. (#6496) -
Multiple Authorized Stages per Argo CD
Application: Thekargo.akuity.io/authorized-stageannotation now accepts a comma-separated list of<project>:<stage>entries, so more than one Stage can runargocd-updateagainst the sameApplication. (docs, #6398) -
Promotions Labeled with Their Stage: The mutating webhook now applies a
kargo.akuity.io/stage: <stage-name>label to every Promotion, makingkubectl get promotions -l kargo.akuity.io/stage=my-stagejust work. (#6417) -
Consistent Promotion Creation: Promotion construction has been consolidated in the defaulting webhook, so Promotions created with
kubectlnow behave exactly the same as those created via the UI or CLI. (#6215)
π¦ Freight Improvements
-
discoveredAtTimestamp: Freight now records when its artifacts were first discovered, and this timestamp β rather than resource creation time β is used everywhere Freight is ordered or aged: auto-promotion, API sorting, CLI age display, and garbage collection. This makes time-based ordering robust even whenFreightresources may have been recreated from a backup. (#6494) -
FreightCreatedEvents: Kargo now records an event whenever Freight is produced by aWarehouseor created manually, complete with the name of theWarehouseit originated from. (docs, #6321, #6374) -
kargo delete freight: The CLI can now delete Freight by name or alias. (#6178)
π₯οΈ UI Improvements
-
UI Fully Transitioned to the REST API: The UI now communicates exclusively through Kargo's REST API, completing its migration off the legacy gRPC (ConnectRPC) API. (See Deprecations above for that legacy API's removal timeline.)
-
Clearer Promotion Retry: In the Stage details drawer's Promotions tab, retrying a Promotion is now a clearly labeled action of its own. (#6624)
-
Freight Timeline Redesign: A cleaner freight timeline with cards that size themselves to fit the row, smoother paging, tidier truncation of long names, and filters tucked into a popover when not in use. (#6267)
-
Freight Diffs in the Promote Drawer: When promoting, the drawer now shows a diff between a Stage's current Freight and the incoming Freight, so you can see exactly what's about to change. (#6234)
-
More Readable Pipelines: Pipeline graph edges are now smooth curves (with a toggle for the old step lines), with better node spacing, ordered connection points that reduce edge crossings, and subtler line styling. Plus, a new search box makes finding a Stage in a large pipeline easy. (#6206, #6320)
-
Deep Links from Step Outputs: Promotion steps can emit a
linksarray in their output, and the UI renders them as clickable icons or labeled links inline on step and promotion views β connecting a promotion to PRs, Jira tickets, dashboards, or anything else. (#6223, #6271) -
Friendlier Freight Details: The Stage drawer now surfaces the currently-running Freight in each requested-freight card, Promotion and Verification tabs show Freight aliases alongside hashes, and internal UIDs are no longer displayed. (#6473)
-
Step Output and Config as YAML: Promotion step configuration and output are now rendered as YAML instead of JSON for improved readability. (#6471)
-
Secret Types for Generic Secrets: The generic secrets flow now supports the Kubernetes
Secrettypefield (e.g.kubernetes.io/dockerconfigjson), which previously requiredkargo apply. (#6492) -
Azure DevOps Commit Links: Commit hashes on Freight sourced from Azure DevOps repositories are now clickable deep links. (#6327)
β Helm Chart Improvements
-
Prometheus Metrics Made Scrapeable: The chart can now expose metrics
Services and optionalServiceMonitors for the controller, management controller, and webhooks server, following the same pattern as the Argo CD and Argo Workflows charts. (docs, #6486) -
Serve Kargo Under a Subpath: New
api.basePathandexternalWebhooksServer.basePathvalues allow the API server, UI, and external webhooks server to be served behind a configurable URL path prefix. (#6404) -
PodDisruptionBudgets: Optional PDBs can now be configured for Kargo components. (#6082) -
Extra Containers and Volumes: Every pod-bearing component now supports extra
containers,initContainers,volumes, andvolumeMounts, andtplis applied toenv/envFromvalues. (#6318, #6427) -
Labels, Annotations, and Namespaces: Additional configurability for custom labels and annotations across resources, including on the non-main namespaces the chart creates β whose RBAC resources can now also be conditionally disabled. (#6238, #6102)
-
Odds and Ends:
appProtocolis configurable on the APIService(#6314), the Dex server's log format is configurable (#6227), and the Kubernetes webhook server's port can be changed via theWEBHOOK_SERVER_PORTenvironment variable for environments where 9443 conflicts with other host-network services (#6358).
π Notable Bug Fixes
-
Approvals No Longer Cross Origin Boundaries (#6562): Manually approving Freight for a Stage previously made it promotable to that Stage even if the Stage did not request Freight from that origin. Approval now grants an exemption from verification requirements only β not from origin boundaries.
-
Fewer Intermittent "Repository Not Found" Errors with GitHub Apps (#6630): GitHub replicates newly minted App installation access tokens to its edge infrastructure asynchronously, so a token used immediately after being minted was sometimes rejected β masked by GitHub as a 404 on private repositories. Kargo now validates every new token before caching and using it, eliminating a long-standing source of spurious "repository not found" errors during git operations.
-
Tolerance for Transient Argo CD Discovery Failures (#6583, #6606, #6664, #6671): Kubernetes API discovery hiccups β most common on fresh clusters or shortly after startup β could previously cause
argocd-updateandargocd-waitsteps to fail spuriously. The controller now retries them. -
Argo Rollouts v1.9 Compatibility (#6472): Kargo's embedded Argo Rollouts type stubs were a mix of v1.7 and v1.8 vintage. They've been synced with Rollouts v1.9, ensuring verification works smoothly with recent
AnalysisTemplatefeatures. -
CLI Token Refresh with Dex (#6523): The CLI now refreshes OIDC tokens using the CLI-specific client ID it logged in with, eliminating forced re-authentication for identity providers that distinguish web and CLI clients.
-
Reliable GAR/GCR Credentials on GKE (#6129): The controller now retries Workload Identity Federation initialization instead of permanently disabling Google Artifact Registry support when the instance metadata server isn't reachable at startup β an occasional occurrence on GKE Autopilot.
π Special Thanks
Thank you to community members who made their first contributions in this release or a recent v1.10 patch release!
@ajaytikoo @anaplan-kh @audip @badstreff @cgarcia-l @hhrygim @jessebot @jinjiaKarl @johnsmith-ayx @jz-wilson @kamalmarhubi @mail2sudheerobbu-oss @nickvollmar @rhysmcneill @sjoukedv @snuggie12 @tbartik
Full Changelog: v1.10.9...v1.11.0