Breaking changes

This release requires at least Go 1.19.

github.com/cilium/ebpf/btf:

• Handle.Spec() now takes a base *Spec argument.
  nil is accepted if the Handle is for vmlinux. If Handle is for a (split BTF) kernel module, pass the result of LoadKernelSpec().
• NewHandle() now takes a *Builder instead of a *Spec.
  Loading BTF into the kernel now goes via a new Builder type. See the 'Additions' section below.

github.com/cilium/ebpf/link:

• KprobeMultiOptions.Addresses: changed from []uint64 to []uintptr
  The old API didn't take 32-bit architectures into account.

Fixes

Work around the kernel rejecting some Datasec

The kernel erroneously rejects Datasec where a Typedef, Volatile, Const, Restrict or typeTag follows a Pointer, Struct, Union or Array. There is now a workaround in place, see #954.

Marshaling Map keys and values is more efficient

The marshaling code in the library now uses sync.Pool to re-use bytes.Buffer, which makes common map operations cheaper, see #1053.

CO-RE relocations are a lot more efficient

The CO-RE code now does much less copying of BTF types, which makes CO-RE relocation a lot faster, especially against large types such as sk_buff. See #1084.

Additions

__ksym (kfunc) support

It's now possible to use new-style BPF helpers aka kfunc with the library. Going forward, all new BPF 'helper' functionality in the kernel will be exposed as kfuncs, and new helpers won't be added. See #966 and #996.

__kconfig support

Tracing programs often need to vary their behaviour based on kernel configuration, such as CONFIG_HZ. Such references to __kconfig variables are now automatically populated from a variety of sources such as /proc/config.gz. Note that if you run your application implementing ebpf-go in a container, it will need access to the host's /boot on some distributions that don't ship /proc/config(.gz). (notably, Debian-based distros)

As a special mention, the LINUX_HAS_SYSCALL_WRAPPER kconfig is also supported. This allows writing portable kprobes using the BPF_KSYSCALL macro from bpf_tracing.h.

See #951, #960 and #995.

Overwritable perf buffer support

The perf reader now allows creating "overwritable" perf buffers, which always contain the most recent events in case the buffer ever gets full. This is in contrast to regular perf buffers which drop recent events if there is no more space. This is useful to implement "flight recorder" type functionality for events sourced from BPF.

See #953.

btf.Builder

Constructing custom BTF type blobs is now possible through the new btf.Builder type. Call btf.NewBuilder() to obtain one, Builder.Add(t btf.Type) to add any types you need, followed by btf.NewHandle(b btf.Builder) to load the types into the kernel. Builder also has a Marshal() method for serializing the type collection into the canonical BTF format so it can be stored for later use.

What's Changed

• cs.RewriteConstants: define error MissingConstantsError by @alban in #904
• internal: add Memoize by @lmb in #909
• Fix ProgramInfo.MapIDs for programs without maps on old kernels by @lmb in #905
• link: make KprobeMultiOptions.Addresses a []uintptr by @lmb in #913
• btf: export API to create BTF from scratch by @lmb in #859
• Update CI dependencies by @lmb in #916
• map: replace hacky unsafe.Pointer conversion with PROT_NONE page by @lmb in #915
• link: use strings.Map in sanitizeSymbol by @lmb in #919
• btf: skip .data..percpu in TestRoundtripVMlinux by @lmb in #922
• link: close perfEventLink.fd before perf event in Close() by @ti-mo in #918
• map: ignore BPF_F_RDONLY_PROG for DevMap compatibility check by @aibor in #930
• Remove deprecated +build Go build tags by @ti-mo in #888
• link: use /sys/kernel/tracing if available by @spikat in #906
• collection: work around excessive copying of kernel BTF by @alban in #920
• sys: add fd leak tracing instrumentation by @ti-mo in #732
• program: set name from object info for pinned file by @aibor in #932
• Allow specifying the tracefs prefix in options by @brycekahle in #842
• Fixed broken links pointing to Cilium doc by @PhilipSchmid in #942
• link: use statfs for tracefs mount detection by @brycekahle in #944
• btf: ensure that TypesIterator is not affected by Spec.Add by @lmb in #937
• run-tests: allow using local kernel by @lmb in #952
• require Go 1.19 by @lmb in #950
• Support __kconfig macro for LINUX_KERNEL_VERSION by @eiffel-fl in #951
• btf: work around kernel Datasec bug by @lmb in #954
• link: always include requested symbol in kprobe creation errors by @ti-mo in #959
• Fix expected fs type for /sys/kernel/debug/tracing by @brycekahle in #958
• collection: ensure LINUX_KERNEL_VERSION variable is a btf.Int. by @eiffel-fl in #961
• add tests for FSType and getTracefsPath by @lmb in #962
• Link to maintainers file by @xmulligan in #957
• docs: update various bits of documentation by @lmb in #936
• fix some comments by @cuishuang in #972
• link: new link from fd by @Asphaltt in #971
• Add support for loong64 by @zhaixiaojuan in #975
• Allow debugfs fstype for /sys/kernel/debug/tracing by @brycekahle in #976
• Add error case for creating unspecified map type. by @tommyp1ckles in #974
• perf: Add support for overwritable buffer. by @eiffel-fl in #953
• update platformPrefix list with libbpf values by @paulcacheux in #982
• link: document AttachCgroup and flags by @ti-mo in #980
• perf: clean up tests by @lmb in #979
• btf: remove Spec.firstTypeID() by @lmb in #990
• elf_reader, linker: add kfunc support by @rgo3 in #966
• CI: enable gofmt linter by @lmb in #992
• link: use BPF links to attach Tracing and LSM prog types by @mmat11 in #837
• bpf2go: spring cleaning by @lmb in #910
• btf: fix off-by-one in Spec.TypeByID by @lmb in #993
• link/executable: lazy load symbol table by @mmat11 in #991
• btf: remove type alias for []Type by @lmb in #989
• sys: introduce TypeID by @lmb in #912
• elf: freeze .kconfig map by @lmb in #1000
• btf: clean up handling of split BTF in loadRawSpec and inflateRawTypes by @lmb in #999
• elf: use per-instruction metadata for .kconfig references by @lmb in #994
• btf: avoid repeated kernel BTF copies in Handle.Spec by @lmb in #1001
• docs: document API stability by @lmb in #1003
• bpf2go: Fall back to default module name when debug.ReadBuildInfo is not available by @folbricht in #1004
• update github.com/frankban/quicktest by @lmb in #1010
• asm: update eBPF built-in functions by @florianl in #1011
• ProgramInfo provides CreatedByUid by @zachcheu in #1006
• btf: optimize Spec.Copy by @lmb in #1002
• Add support for LINUX_HAS_SYSCALL_WRAPPER kconfig by @paulcacheux in #995
• program: improve error handling in haveSyscallWrapper by @lmb in #1022
• fix container-all make invocation by @paulcacheux in #1021
• make: allow interrupting container-all with ctrl-c by @lmb in #1024
• refactor tracefs event handling into an object by @lmb in #1023
• Print out the relocation name in the error message by @kuroa-me in #1026
• allow skipping kernel version check for feature tests if test skipped via env variable by @abhipranay in #1025
• replace deprecated rand.Seed calls by @kwakubiney in #978
• elf: support kernel module kfunc calls by @shun159 in #996
• tracefs: minor cleanups by @lmb in #1027
• elf_reader: Add support for XDP frags by @ptzianos in #1037
• program: move examples to VerifierError in main package by @lmb in #1040
• sys: prevent clobbering existing entries in fd tracing registry by @ti-mo in #1043
• btf: avoid triggering feature probe on import by @lmb in #1045
• program: update Benchmark doc by @lmb in #1046
• collection: Add code to parse kernel config files by @eiffel-fl in #960
• btf: move LoadKernelSpec into separate package by @lmb in #1015
• link: add test for UDP iter by @lmb in #1044
• marshalers: Pool bytes.Buffers in marshalbytes by @ti-mo in #1053
• CI: try to fix job timeouts by @lmb in #1056
• Pre-allocate map sys.Errors by @hmahmood in #1052
• internal/sys: don't panic in init() by @lmb in #1055
• internal/kconfig: Add possibility to filter kconfig file by @eiffel-fl in #1051
• bpf2go: argument parsing unit test by @mejedi in #1054
• bpf2go: some options take defaults from the environment by @mejedi in #1047
• map: disentangle per-CPU handling by @lmb in #1067
• btf: move Spec.Add to dedicated type by @lmb in #1069
• Keep code multi OS aware by @florianl in #1075
• Benchmark batch lookups vs. MapIterator, remove MapIterator.prevKey by @ti-mo in #1077
• testutils: print newline after random seed by @lmb in #1085
• btf: don't copy types during CO-RE relocation by @lmb in #1084
• Makefile: require user to provide extract-vmlinux by @lmb in #1088
• btf: fix CO-RE bitfield relocations by @lmb in #1086

New Contributors

• @alban made their first contribution in #904
• @spikat made their first contribution in #906
• @PhilipSchmid made their first contribution in #942
• @eiffel-fl made their first contribution in #951
• @cuishuang made their first contribution in #972
• @Asphaltt made their first contribution in #971
• @zhaixiaojuan made their first contribution in #975
• @tommyp1ckles made their first contribution in #974
• @zachcheu made their first contribution in #1006
• @kuroa-me made their first contribution in #1026
• @abhipranay made their first contribution in #1025
• @kwakubiney made their first contribution in #978
• @shun159 made their first contribution in #996
• @ptzianos made their first contribution in #1037
• @hmahmood made their first contribution in #1052
• @mejedi made their first contribution in #1054

Full Changelog: v0.10.0...v0.11.0