akuity/kargo v0.8.0

on GitHub

💥  Kargo v0.8.0 is here!

As with all our minor releases, v0.8.0 ushers in numerous improvements and bug fixes, but also has a heavy focus on one major new feature: Multi-Pipelines.

Multi-Pipelines

"Pipeline" has never been a well-defined term in Kargo. It's been intuitively and widely understood to represent the path that Freight takes from a Warehouse, through a DAG of Stages, and into production.

Until now, only one such pipeline could be thought of as passing through each Stage. Although a Stage could subscribe to multiple upstream Stages, this only ever expressed that Freight should be considered available for promotion to that Stage after being verified in any one of its upstream Stages. Ultimately, Stages (or, more accurately, their underlying environments) were still constrained to hosting only one piece of Freight at a time. Some users with advanced use cases have found this limiting and have long been asking for Stages to be able to host multiple pieces of Freight (of different types) at once.

Why would someone want this? In general terms it is the product of a desire to advance different artifacts through the same Stages in parallel, and likely at different cadences. Examples:

• CI may publish many new Docker images in a day, and a user may typically wish to advance these all the way through to production rather quickly. By contrast, they may wish for new manifests to advance more slowly and "soak" for some time in each Stage.

• An application may be composed of many microservices that can be delivered in parallel. Kargo has handled this well for some time now if users manage a separate set of Stages for each microservice, but this can be cumbersome and fails to visually convey that the microservices are components of a larger whole.

In short, Freight, as meta-artifacts, have been fantastic for delivering sets of related artifacts as a unit, but sometimes users need to deliver related artifacts (or sets of artifacts) independently of one another.

Kargo v0.8.0 introduces the ability for Stages to host multiple pieces of Freight, of different types, simultaneously. Stages can now be thought of as possessing multiple "slots" for different types of Freight, with multiple pipelines passing through each Stage, dropping off different types of Freight to fill those slots -- and potentially doing so at different cadences.

Breaking Changes

With Stages gaining the ability to host multiple pieces of Freight simultaneously, as discussed above, there are obvious implications for the spec and status of Stage resources. Where Stages could previously subscribe to a Warehouse, or to one or more upstream Stages, permitting Stages to host multiple pieces of Freight simultaneously necessitates that Stages can reference Warehouses and/or Stages as sources for multiple types of Freight. Similarly, while the status of a Stage previously tracked which one piece of Freight's artifacts were deployed to the underlying environment, it now needs to track this information for multiple pieces of Freight of different types.

These changes have forced Kargo to incur some breaking changes -- which the team is avoided whenever all but unavoidable.

A StageSpec that may previously have subscribed to a Warehouse, like so:

spec:
  subscriptions:
    warehouse: kargo-demo

Now expresses a "request" for a particular type of Freight (as described why where it originated from) and indicates a willingness to accept it directly from that origin (its Warehouse):

spec:
  requestedFreight:
  - origin:
       kind: Warehouse
       name: kargo-demo
  sources:
    direct: true

Similarly, a StageSpec that may previously have subscribed to some upstream Stages, like so:

spec:
  subscriptions:
    upstreamStages:
    - name: test
    - name: qa

Now expresses a "request" for a particular type of Freight and indicates a willingness to accept it from certain upstream Stages (but not directly from its origin, in this case):

spec:
  requestedFreight:
  - origin:
  	kind: Warehouse
  	name: kargo-demo
    sources:
      stages:
      - test
      - qa

Fortunately, the team has (as we've done with previous releases) been able to automate the migration process from v0.7.x to v0.8.0. There is no automatic migration from versions prior to v0.7.0.

Although this migration process will keep things running smoothly in the near term, it is suggested that, especially if you GitOps your Kargo Projects, that you update your StageSpecs so that your GitOps agent (e.g. Argo CD) won't find itself perpetually at odds with Kargo's automated upgrade process. v0.9.0 can also be expected to remove all fields deprecated in the course of this change, creating further impetus to update your StageSpecs.

Credentials

Kargo v0.8.0 introduces a variety of new integrations for obtaining credentials for accessing Git and OCI repositories.

• Building off of v0.7.0's support for using EKS Pod Identity to access image
  repositories hosted in ECR:
  • Kargo now supports using IRSA as well.
  • EKS Pod Identity and IRSA now work for accessing chart repositories hosted in ECR.
  • If Kargo is unable to assume a Project-specific IAM role, it will fall back on directly using the IAM role associated with the Kargo controller. This can relieve the overhead of managing many Project-specific IAM roles for users without strict tenant isolation requirements.
• Improved and well-validated support for accessing Git repositories hosted in GitHub Enterprise and in self-hosted GitLab instances.

UI

As usual, there are too many UI improvements to list, but here are some highlights:

• Projects view is now paginated and searchable.
• Sidebar navigation now contains links to documentation and CLI binaries.
• "Freightline" has been renamed to "Freight Timeline" in an attempt to emphasize that it is simply a stream of Freight arranged chronologically.
• Freight Timeline can now be filtered by Warehouse. Freight older than the oldest still in use can be hidden to improve UI performance.
• The image history table can be hidden.

Miscellaneous Improvements

• When evaluating Stage health:
  • Associated Argo CD Applications in a "Suspended" state now cause the Stage to be counted as "Progressing."
  • A grace period is now observed after an associated Argo CD Application first becomes "Healthy" after a promotion. This is to reduce the impact of Argo CD Applications that falsely report that they are healthy in the moments immediately following a sync.
• Improved Freight-comparison logic to be less dependent on deterministically calculated Freight ID, as the algorithm for calculating this ID has sometimes changed from one release to the next.

Where We Go From Here

It's a good time to start preparing our community for the breaking changes we anticipate in v0.9.0.

The extremely varied and complex nature of the use cases we are seeing among Kargo users has made it clear that our promotion mechanisms require fewer opinions and more flexibility. To this end, we are planning to replace our tree of opinionated promotion mechanisms with the ability to execute an ordered list of promotion directives. These directives will be more granular, thereby permitting users to describe more complex promotion processes. Perhaps more importantly, we anticipate this pivot paving the way for user-defined or third-party promotion directives, upon which alternative and more sophisticated storage, configuration management, notification, and approval capabilities will be built.

As usual, you can expect the Kargo team to make every effort to automate the upgrade process such that any breaking changes incurred in v0.9.0 will pose minimal disruption to your Kargo Projects.

New Community Contributors

Last, but certainly not least, Kargo would be nothing without its community, so we'd like to recognize @anubhav06 for their first contribution, which is included in this release. Thank you for helping to make Kargo better!

Full Changelog: v0.7.1...v0.8.0