cilium/ebpf v0.10.0

v0.10.0 - BTF Marshaling

on GitHub

We're pleased to announce the first ebpf-go release of 2023! There is one breaking change, so please read through the release notes carefully. All users are encouraged to upgrade.

Breaking Changes

Support for BTF marshaling

So far, the library would only load BTF into the kernel that was originally generated by clang or pahole. As of this release, the library will instead marshal btf.Types directly into BTF blobs. For example, it's now possible to create an ebpf.Map from Go code by populating the MapSpec.Key and .Value fields with the map's k/v types to make bpftool map dump nicely pretty-print its contents for you.

As expected, there are a few important side effects and breaking changes to be aware of:

• Setting MapSpec.BTF and ProgramSpec.BTF to nil can no longer be used to disable BTF during loading, which was the fields' last remaining purpose. The fields have been removed to make sure this doesn't slip through library upgrades unnoticed.
• If your userspace program replaces individual instructions in ProgramSpec.Instructions before loading a program, make sure to copy the old Instruction's Metadata using Instruction.WithMetadata(old.Metadata). Not doing so may cause you to discard the original instruction's func_info or line_info, leading to the verifier rejecting your program with e.g. invalid argument: missing bpf_line_info for func#0. (see cilium/cilium#21933 for how this can go wrong)
• Disabling BTF for program loads now requires calling btf.WithFuncMetadata(ins, nil) on the first instruction of each (sub)program, as well as Instruction.WithSource(nil) on each Instruction containing lineinfo.
• Disabling BTF for map loads now requires MapSpec.Key and .Value being set to nil.

We're interested in hearing your use cases for explicitly disabling BTF during map/prog loads. We hope the need for disabling BTF altogether should arise less frequently, given the library falls back gracefully when the underlying kernel doesn't support BTF. In case you do believe this to be necessary, please raise an issue to discuss this further.

Fixes

NewProgram returns an unwrapped VerifierError

Since commit 148c76c ("internal: make VerifierError more succinct") the library has defaulted to omitting most of the verifier log when loading a program failed. The intention was that callers would use formatting with the %+v verb to output as much context as necessary. Due to how error wrapping with fmt.Errorf works this meant that the error had to be unwrapped with errors.As, which led to confusion.

NewProgram and friends now return an unwrapped VerifierError so that callers can format the error more easily.

Reliable VerifierError.Truncate field

Prior to this release, the VerifierError.Truncate field would only be set in case ProgramOptions.LogLevel was left to the default value of 0, among a few other quirks. This has now been resolved, making the VerifierError.Truncate field a reliable driver for a retry loop that automatically grows the verifier log buffer and retries loading the program(s). See cilium/cilium@934bccf for an example implementation.

Additions

Expanded and overhauled feature probes

features.HaveProgramType() can now conclusively probe for the program types ebpf.LSM, ebpf.Tracing and ebpf.Extension without relying on recognizing specific error return values.

Package features has been refactored to use internal.FeatureTest, unifying error wrapping and result caching with the library's internal machinery. ErrNotSupported returned from features now includes the minimum required kernel version and a feature name.

Kretprobes allow setting maxactive

The kernel's kretprobe implementation has a limitation where only a fixed number of concurrent calls to a probed function are handled. Usually the kernel chooses a sufficient default value, but for very busy functions this default is too low. This leads to missed kretprobe events.

There is an (unfortunate) workaround for this: the user can specify how many concurrent calls they want to support via a maxactive parameter. This comes with a lot of drawbacks however. maxactive is only supported when using an obsolete interface to kretprobes, and it's not at all clear how to arrive at the correct maxactive setting. link.KprobeOptions now exposes this setting to the user, since it is the only partial fix.

The authors of the library recommend to not use maxactive unless absolutely necessary. Incorrect use will make your application more brittle and may have system-wide performance impact.

What's Changed

• map, prog: fix broken links by @boratanrikulu in #816
• program: clarify how to use VerifierError by @lmb in #819
• link: Allow kprobe multi to be disabled in kernel by @arthurfabre in #812
• Add IsPinned() to RawLink by @boratanrikulu in #817
• fix bad link to GitHub Discussions by @dmitris in #824
• internal: detect if /proc/self/auxv is not readable due to file caps by @lmb in #825
• btf: add support for marshaling Type and use it for Program and Map by @lmb in #796
• btf: support enum64 by @willfindlay in #820
• Add stringer to ebpf-builder by @lmb in #827
• asm: add .WithMetadata() for conveniently replacing individual Instructions by @ti-mo in #832
• prog: populate VerifierError.Truncate when LogLevel > 0 by @ti-mo in #834
• elf_reader: check if ELF is for BPF data by @florianl in #830
• map_test: clean up after tests and close maps by @florianl in #841
• btf: check for compatibility first when searching for a CO-RE field by @lmb in #852
• Set Program.name when constructing from file descriptor by @aibor in #849
• btf: do Datasec fixup on inflated types by @lmb in #860
• btf: distinguish 'map' and 'program' BTF by @ti-mo in #855
• program: return unwrapped VerifierError by @lmb in #851
• map: export MapSpec.Compatible by @olsajiri in #858
• btf: Remove deprecated {Map,Program}Spec.BTF field by @ti-mo in #864
• btf: fix some split BTF shortcomings by @lmb in #861
• Update dependencies for current Go versions by @thaJeztah in #866
• map: include generated pin path in newMapWithOptions error by @ti-mo in #870
• features: reuse internal.FeatureTest instead of open coding probes by @lmb in #776
• feat: support LSM prog type by @daemon1024 in #885
• bpf2go: write dependencies to temporary file to support Windows by @junjiexing in #865
• program: block SIGPROF during BPF_PROG_RUN by @lmb in #887
• btf: fix function doc typo by @rgo3 in #889
• link: add QueryPrograms API by @rgo3 in #867
• link: add maxactive for kretprobe by @alahaiyo in #755
• Add missing program type feature probes by @rgo3 in #890

New Contributors

• @boratanrikulu made their first contribution in #816
• @willfindlay made their first contribution in #820
• @aibor made their first contribution in #849
• @daemon1024 made their first contribution in #885
• @junjiexing made their first contribution in #865
• @alahaiyo made their first contribution in #755

Full Changelog: v0.9.3...v0.10.0