hyperlight-dev/hyperlight v0.19.0

Release v0.19.0

on GitHub

What's Changed

Added

• Sandbox::snapshot_with_options takes a snapshot with SnapshotOptions. SnapshotOptions::scratch_size gives the sandbox, and later restores of the snapshot, a different scratch size. A guest can boot with the scratch its initialisation needs and run guest function calls from a snapshot with only the scratch they need, so each restore costs less. It can also give a guest more scratch than it booted with.
• Sandbox::scratch_allocated_bytes returns how much scratch the guest has allocated since the sandbox was last restored or snapshotted. The guest controls the value it is read from.
• hyperlight-component-csharp crate for generating C# host bindings from WIT worlds by @ludfjig in #1946

Changed

• Core dumps no longer record the guest binary path. The process name and command line are always <unknown>, as they already were for sandboxes created from snapshots by @ludfjig in #1975

• Breaking: Generated libc bindings use usize for size_t arguments and return values, including strlen and memcpy. Rust callers that use u64 or c_ulong need explicit conversions. The C ABI is unchanged.

• Breaking: Snapshots use ABI 9 and config schema v5. Existing snapshots
  must be regenerated.

• Breaking: PtRootFinder receives a SnapshotMemoryReader and returns a
  Result<Vec<u64>>. Read snapshot bytes with snapshot.read_exact(gpa, buffer)?
  and return roots with Ok(roots).

• Related snapshots share immutable memory layers. This reduces capture time
  and memory use. A guest can read data it overwrote before a snapshot from the
  older layers of that snapshot.

• KVM: deleting a memory slot flushes only that slot's guest mappings on Linux
  6.12 and later. This speeds up the first guest call after a snapshot,
  restore, or unmap changes memory slots.

• Breaking: Sandbox::snapshot installs the captured snapshot into the
  sandbox and removes map_region and map_file_cow regions, which only
  restore removed before. Region pages the guest has not mapped into its page
  tables become unreachable at the snapshot instead of at the next restore.

• vmem::walk_va_spaces merges adjacent mappings of the same kind into one
  mapping.

• A new sandbox writes only the guest code, PEB and init data into its memory.
  The host populates other pages on first guest touch, which speeds up sandbox
  creation.

• Use talc as the guest heap allocator. Guest init writes only the start and
  end of the heap, which speeds up sandbox creation.

• Breaking: Snapshots no longer contain a zero-filled heap region, which
  makes them smaller and faster to create, save, and load. The guest heap is
  backed by scratch and has no fixed size. Running out of scratch aborts the
  guest with MallocFailed instead of failing the allocation.

• Breaking: The transport arena sits one unmapped guard page below the reserved scratch-top pages, at an address independent of the scratch size, and the page tables at the scratch base. The guest's page allocator stops at the arena GPA the host publishes in the scratch-top metadata, so guests must be rebuilt. hyperlight_common::layout::scratch_allocator_limit_gpa is renamed scratch_reserved_top_gpa, the start of the reserved pages. An allocator that stops there runs into the arena. The minimum scratch size grows by that guard page. Snapshot ABI 7: existing snapshots must be regenerated.

• Breaking: hyperlight-component-macro: host_bindgen! and guest_bindgen! encode values little-endian and qualify function names with their interface, such as wasi:http/types@0.2.0#[method]fields.get by @ludfjig in #1946

Removed

• Breaking: SandboxConfiguration::set_heap_size,
  SandboxConfiguration::DEFAULT_HEAP_SIZE, SandboxBuilder::heap_size, and
  SandboxBuilder::get_heap_size.

Fixed

• Guest libc accepts null buffers for zero-length writes.
• Fix calls to guest functions with more than 11 parameters. Registration accepts up to 32, but every call with more than 11 failed.
• Fix restore on KVM not rewinding the guest TSC when restores are less than a second apart.

Full Changelog (excl. dependencies)

• Run guard page crash tests on Windows by @DanielTangnes in #1899
• Test MSHV 6.18 weekly and benchmark it daily by @ludfjig in #1941
• feat(virtq): use Rc pointer equality instead if generation for pools by @andreiltd in #1945
• Add platform support to README by @yoshuawuyts in #1967
• Make PrimeCaches the only Rust cache writer and remove sccache by @ludfjig in #1963
• Remove SandboxRuntimeConfig by @ludfjig in #1975
• feat: restore snapshots with a different scratch size by @danbugs in #1947
• test(sandbox): check held buffers survive resized snapshots by @danbugs in #1995
• fix(guest): allow guest functions with more than 11 parameters by @ludfjig in #1998
• Layered/incremental snapshots by @ludfjig in #1926
• Write initial snapshot memory in place by @ludfjig in #1927
• Use talc as the guest heap allocator by @ludfjig in #1928
• HIP 0005 - Structured Labels by @yoshuawuyts in #1978
• Fix flaky reset_vcpu_simple and log cpu in CI by @ludfjig in #2018
• fix(trace): read each guest trace batch once by @ludfjig in #1991
• Restore the guest TSC exactly on KVM by @ludfjig in #2015
• Expose how much scratch a sandbox has allocated by @danbugs in #2019
• Add crate for host-side .net component bindings by @ludfjig in #1946
• Remove the legacy heap region from snapshots by @ludfjig in #1968

Full Changelog (dependencies)

• chore(deps): bump toml from 1.1.6+spec-1.1.0 to 1.1.7+spec-1.1.0 by @dependabot[bot] in #1936
• chore(deps): bump toml from 1.1.7+spec-1.1.0 to 1.1.8+spec-1.1.0 by @dependabot[bot] in #1942
• chore(deps): bump uuid from 1.26.1 to 1.27.0 by @dependabot[bot] in #1924
• chore(deps): bump tokio from 1.53.1 to 1.53.2 by @dependabot[bot] in #1920
• chore(deps): bump serde_json from 1.0.151 to 1.0.154 by @dependabot[bot] in #1948
• chore(deps): bump opentelemetry-semantic-conventions from 0.33.0 to 0.33.1 by @dependabot[bot] in #1949
• chore(deps): bump syn from 3.0.6 to 3.0.8 by @dependabot[bot] in #1953
• chore(deps): bump proptest from 1.11.0 to 1.12.0 by @dependabot[bot] in #1951
• chore(deps): bump smallvec from 1.16.2 to 1.16.3 by @dependabot[bot] in #1952
• chore(deps): bump uuid from 1.27.0 to 1.28.0 by @dependabot[bot] in #1954
• chore(deps): bump actions/upload-artifact from 7.0.1 to 7.0.2 by @dependabot[bot] in #1935
• chore(deps): bump dtolnay/rust-toolchain from 7e38f4b43b4db5c8dd498af069a4f6196df1d067 to e2a55d2ffb04f378e9626c28d38b36d230d1e12f by @dependabot[bot] in #1914
• chore(deps): bump cc from 1.6.0 to 1.8.0 by @dependabot[bot] in #1950
• chore(deps): bump kvm-ioctls from 0.25.0 to 0.25.1 by @dependabot[bot] in #1944
• chore(deps): bump syn from 3.0.8 to 3.0.9 by @dependabot[bot] in #1960
• chore(deps): bump actions/download-artifact from 8.0.1 to 8.0.2 by @dependabot[bot] in #1957
• chore(deps): bump dtolnay/rust-toolchain from e2a55d2ffb04f378e9626c28d38b36d230d1e12f to 683b588c487a69d01f6c6ddf40f036ff682cc2c8 by @dependabot[bot] in #1958
• chore(deps): bump crate-ci/typos from 1.50.3 to 1.51.1 by @dependabot[bot] in #1934
• chore(deps): bump actions/create-github-app-token from 3.2.0 to 3.3.0 by @dependabot[bot] in #1965
• chore(deps): bump tokio from 1.53.2 to 1.53.3 by @dependabot[bot] in #1966
• Changes to allow different toolchains for host and guest by @simongdavies in #1962
• chore(deps): bump taiki-e/install-action from 2.87.21 to 2.87.28 by @dependabot[bot] in #1959
• chore(deps): bump sha2 from 0.11.0 to 0.11.1 by @dependabot[bot] in #1961
• chore(deps): bump libc from 0.2.190 to 0.2.191 by @dependabot[bot] in #1981
• Bump ci-setup-workflow to v2.3.1 by @ludfjig in #1984
• chore(deps): bump cc from 1.8.0 to 1.9.0 by @dependabot[bot] in #1986
• chore(deps): bump taiki-e/install-action from 2.87.28 to 2.87.29 by @dependabot[bot] in #1977
• chore(deps): bump bindgen from 0.72.1 to 0.73.2 by @dependabot[bot] in #1829
• chore(deps): bump rust-lang/crates-io-auth-action from 1.0.5 to 1.0.6 by @dependabot[bot] in #1990
• chore(deps): bump taiki-e/install-action from 2.87.29 to 2.87.30 by @dependabot[bot] in #1989
• chore(deps): bump tracing-opentelemetry from 0.33.0 to 0.34.0 by @dependabot[bot] in #1867
• chore(deps): bump oras-project/setup-oras from 2.0.1 to 2.0.2 by @dependabot[bot] in #1887
• chore(deps): bump rust-embed from 8.12.0 to 8.13.0 by @dependabot[bot] in #1931
• chore(deps): bump arbitrary from 1.4.2 to 1.5.0 by @dependabot[bot] in #1933
• chore(deps): bump docker/login-action from 4.6.0 to 4.7.0 by @dependabot[bot] in #2001
• chore(deps): bump taiki-e/install-action from 2.87.30 to 2.87.31 by @dependabot[bot] in #2000
• chore(deps): bump tokio from 1.53.3 to 1.54.1 by @dependabot[bot] in #1993
• chore(deps): bump smallvec from 1.16.3 to 1.16.4 by @dependabot[bot] in #2008
• chore(deps): bump cc from 1.9.0 to 1.10.0 by @dependabot[bot] in #2006
• chore: prepare release v0.19.0 by @danbugs in #2024

New Contributors

• @DanielTangnes made their first contribution in #1899

Full Changelog: v0.18.0...v0.19.0