moby/buildkit v0.10.0

on GitHub

Welcome to the v0.10.0 release of buildkit!

Please try out the release binaries and report any issues at
https://github.com/moby/buildkit/issues.

Notable changes:

• New LLB operation MergeOp allows efficiently chaining groups of layers together without the need to access the individual files. This allows to build layers separately and merge them together later without making them depend on each other. MergeOp can work with remote references, for example, it can be used to rebase an image on top of a newer base image without ever pulling or pushing any layers. #2335 #2614

• New LLB operation DiffOp allows computing a difference between two points in LLB graph containing the files that have been added and whiteouts for files that have been removed. When DiffOp inputs are based on the same layer chain DiffOp can work directly with remote layer references and doesn't need to pull layers to access individual files. Files of the DiffOp result can be accessed directly or used as input to a MergeOp. #2517 #2434 #2563

• New build information structures are generated with build metadata that allows you to see all the sources (images, git repositories) that were used by the build with their exact versions and also the configuration that was passed to the build. This information can also be embedded into the image configuration if one is generated. Build sources are added to the image config by default. The build configuration is not currently embedded by default to avoid credential leaks in poorly written Dockerfiles but the intention is to enable it in the future. #2311 #2476 #2654 #2657 #2645 #2684

• Empty layer removal feature on exporting images has been removed because it caused history to change after remote cache import #2651

• When possible, blobs are now created with native OverlayFS differ with increased performance. This Differ can directly use files in OverlayFS upper directory instead of scanning for differences between snapshots. #2181 #2491 #2480 #2388 #2390 #2662

• Frontends can now send warning messages that are shown on the progressbar. Warnings can be associated with specific LLB vertex and contain additional information like URL to documentation or location in original source code. #2482 #2498

• Layer blobs can now be exported in Zstd compression format with -o compression=zstd. Zstd provides smaller files and faster decompression than gzip. #2344

• Layers can now be exported with eStargz compression type -o compression=estargz #2246 #2603 #2352 #2674

• A compression level can be set with -o compression-level=N to configure the compressor when new layer blobs are created. This can be used to create smaller blobs by spending more time on the compression step. #2591

• Remote cache inlined in image configuration now supports arbitrary configuration of image layers as cache sources #2501

• Enable eStargz-based lazy pulling on registry cache importer #2648

• Support exporting non-distributable blob descriptors. -o prefer-nondist-layers=true exports layer with a non-distributable mediatype and external URL if such URL was provided when blob was pulled from the remote registry. Descriptors with non-distributable mediatypes are skipped on push. #2561

• Build metadata now provides access to OCI descriptor of the result if one was generated. Previously only digest of the descriptor could be accessed. #2610

• Builder now understands AMD64 Microarchitecture levels, e.g. linux/amd64/v2, linux/amd64/v4. The default variant remains v1 and is normalized to linux/amd64. These variants allow creating more optimized versions of your images that run when the container is running on a more modern CPU. #2588

• LLB now supports progress groups for grouping multiple steps together so they appear as the same row in build progress #2513

• LLB ExecOp now supports build secrets that are exposed as environment variables #2579

• Interactive container API now supports sending signals to processes from the client #2590

• Logs now use a rolling buffer to show the last logs for a process that ended with an error even if regular logs have been clipped because they have reached the max-logs limit. #2577

• Remote cache exporting now support setting the same compression options available to image outputs #2685

• Frontends can now access their own definition and call additional processes on their own image #2443

• Tmpfs mounts can now set a size limit #2411 #2422

• Custom Cgroup parent can now be set the LLB ExecOp #2430

• Add support for Ulimit resource limits in LLB ExecOp #2398

• Add extra hosts supports to Gateway Exec API #2294

• Fix security mode config being dropped on Gateway Exec start #2290

• Git source can now correctly clone annotated tag references #2570

• OpenTelemetry tracing propagation environment variables have been updated to TRACEPARENT and TRACESTATE to match the changes in upstream. Old variables are deprecated and will be removed in the next release. #2572

• Make sure supplementary groups are loaded for the default user configuration #2428

• Allow exporting inline cache when blobs exist in multiple compressions #2405

• Github cache backend retry logic on hitting rate limits has been improved #2506

• Color schema on TTY progressbar has been enhanced on Windows for better readability #2368

• Build status stream now supports ProgressGroup object to group multiple LLB steps into a single progress item #2668

• Fixes to progress stream handling of multiple "complete" events during cache import #2675 #2641

• Fix possible out of order indexes in plain progress mode #2688

• Extra progress step has been added to the step where blobs are prepared for inline cache #2658

• Allow insecure security mode to work on environments where all capabilities are not available #2394

• Use standard user umask for Git processes #2356

• Fix tracing indicators showing up in logs even when tracing is not enabled #2351

• Handling of doublestar (**) pattern has been improved on transporting local sources. #2319 #2486

• QEMU embedded emulators have been updated to v6.2.0 #2634

• Alpine has been updated to 3.15 on release image #2582

• External registry requests now show BuildKit major version in User-Agent header (previously Containerd) #2593

• Fix caching of weak ETag references when pulling HTTP URLs #2629

• Avoid gRPC size limits when transferring lots of logs #2456

• Correct FileOp.Rm to not follow symlinks to the target #2474

• Validate manifest blobs mediatypes against their content #2469

• Make Git checkouts more deterministic for better cache reuse #2397

• Containerd worker now supports rootless mode #2660

• Fix handling tokens with multiple scopes #2431

• Fix possible leaking processes when using external decompressor #2620

• Fix possible issues when converting blobs to another compression #2600

• Fix symlink handling on doing copy with includePath filters #2318

• Performance of creating layer blobs has been improved in some cases #2601

• Request token first with a POST request and fall back to GET request if needed #2553

• Gracefully handle the case where a crash happens after snapshot commit #2564

• Improve Authority pseudo-header handling with new gRPC #2518

• Clean up temporary mounts on a restart that might have leaked after crash #2652 #2670

• Fix possible panic on deduplicating mounts #2519

• Fix shared cache mounts resulting in overlay corruption #2637

• Fix remote cache imports when invoking builds through gateway API #2659

• Fix possible panic on loading invalid config #2650

• Fixes for session handling for parallel builds from local sources #2585

• Fixes for scheduler inconsistency detector #2556

• Allow listening buildkitd on multiple sockets with --addr #2649

• Systemd definitions have been updated with Rootless and notify support #2473

Contributors

• Tõnis Tiigi
• CrazyMax
• Erik Sipsma
• Akihiro Suda
• Kohei Tokunaga
• Sebastiaan van Stijn
• Aaron Lehmann
• Koichi Shiraishi
• Alex Couture-Beil
• Brian Goff
• Justin Chadwell
• Cory Bennett
• Anders F Björklund
• Davis Schirmer
• Edgar Lee
• Jacob MacElroy
• Maxime Lagresle
• Andrey Smirnov
• Christian Weichel
• Csaba Apagyi
• Edgar Lee
• Hans van den Bogert
• Jonathan Giannuzzi
• Morlay
• Shingo Omura
• Shiwei Zhang
• Solomon Hykes
• Vlad A. Ionescu
• lugeng
• sunchunming

Dependency Changes

• github.com/Azure/go-ansiterm d6e3b3328b78 -> d185dfc1b5a1
• github.com/Microsoft/go-winio v0.4.17 -> v0.5.1
• github.com/Microsoft/hcsshim v0.8.18 -> v0.9.2
• github.com/armon/circbuf 5111143e8da2 new
• github.com/containerd/cgroups v1.0.1 -> v1.0.3
• github.com/containerd/console v1.0.2 -> v1.0.3
• github.com/containerd/containerd v1.5.3 -> v1.6.1
• github.com/containerd/continuity v0.1.0 -> v0.2.2
• github.com/containerd/go-cni v1.0.2 -> v1.1.3
• github.com/containerd/stargz-snapshotter v0.6.4 -> v0.11.2
• github.com/containerd/ttrpc v1.0.2 -> v1.1.0
• github.com/containernetworking/cni v0.8.1 -> v1.0.1
• github.com/dimchansky/utfbom v1.1.1 new
• github.com/docker/cli v20.10.7 -> v20.10.12
• github.com/docker/distribution v2.7.1 -> v2.8.0
• github.com/docker/docker ef4d47340142 -> c78f6963a1c0
• github.com/docker/docker-credential-helpers v0.6.3 -> v0.6.4
• github.com/go-logr/logr v1.2.2 new
• github.com/go-logr/stdr v1.2.2 new
• github.com/gogo/googleapis v1.4.0 -> v1.4.1
• github.com/golang-jwt/jwt/v4 v4.1.0 new
• github.com/golang/groupcache 8c9f03a8e57e -> 41bb18bfe9da
• github.com/google/go-cmp v0.5.6 -> v0.5.7
• github.com/google/uuid v1.2.0 -> v1.3.0
• github.com/grpc-ecosystem/go-grpc-middleware v1.2.0 -> v1.3.0
• github.com/hanwen/go-fuse/v2 v2.1.0 -> f57e95bda82d
• github.com/hashicorp/errwrap v1.0.0 -> v1.1.0
• github.com/hashicorp/go-cleanhttp v0.5.1 new
• github.com/hashicorp/go-retryablehttp v0.7.0 new
• github.com/klauspost/compress v1.12.3 -> v1.15.0
• github.com/mitchellh/hashstructure/v2 v2.0.2 new
• github.com/moby/sys/signal v0.6.0 new
• github.com/moby/term bea5bbe245bf -> 3f7ff695adc6
• github.com/opencontainers/image-spec v1.0.1 -> 693428a734f5
• github.com/opencontainers/runc v1.0.0-rc93 -> v1.1.0
• github.com/opencontainers/runtime-spec e6143ca7d51d -> 1c3f411f0417
• github.com/opencontainers/selinux v1.8.0 -> v1.10.0
• github.com/pelletier/go-toml v1.9.4 new
• github.com/prometheus/client_golang v1.7.1 -> v1.12.1
• github.com/prometheus/common v0.10.0 -> v0.32.1
• github.com/prometheus/procfs v0.6.0 -> v0.7.3
• github.com/tonistiigi/fsutil d72af97c0eaf -> b19f7f9cb274
• github.com/tonistiigi/go-actions-cache b93d7f1b2e70 -> 9642704158ff
• github.com/tonistiigi/go-archvariant v1.0.0 new
• github.com/urfave/cli v1.22.2 -> v1.22.4
• github.com/vbatts/tar-split v0.11.2 new
• go.etcd.io/bbolt v1.3.5 -> v1.3.6
• go.opencensus.io v0.22.3 -> v0.23.0
• go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc d010b05b4939 -> v0.29.0
• go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace d010b05b4939 -> v0.29.0
• go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp d010b05b4939 -> v0.29.0
• go.opentelemetry.io/otel v1.0.0-RC1 -> v1.4.1
• go.opentelemetry.io/otel/exporters/jaeger v1.0.0-RC1 -> v1.4.1
• go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.4.1 new
• go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.0.0-RC1 -> v1.4.1
• go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.0.0-RC1 -> v1.4.1
• go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.0.0-RC1 -> v1.4.1
• go.opentelemetry.io/otel/internal/metric v0.21.0 -> v0.27.0
• go.opentelemetry.io/otel/metric v0.21.0 -> v0.27.0
• go.opentelemetry.io/otel/sdk v1.0.0-RC1 -> v1.4.1
• go.opentelemetry.io/otel/trace v1.0.0-RC1 -> v1.4.1
• go.opentelemetry.io/proto/otlp v0.9.0 -> v0.12.0
• golang.org/x/crypto 0c34fe9e7dc2 -> 5770296d904e
• golang.org/x/net e18ecbb05110 -> fe4d6282115f
• golang.org/x/sys 5e06dd20ab57 -> da31bd327af9
• golang.org/x/time 3af7569d3a1e -> 1f47c861a9ac
• google.golang.org/genproto 8816d57aaa9a -> 3a66f561d7aa
• google.golang.org/grpc v1.38.0 -> v1.44.0
• gopkg.in/yaml.v3 9f266ea9e77c -> 496545a6307b

Previous release can be found at v0.9.0