What's Changed
⚠️ Breaking Changes
- SandboxConfig now holds multiple named versions with a default by @zoez7 in #2333
- Replace SnapshotContentScope with the SnapshotFidelity ladder and preferred_fidelity by @dberkov in #2364
- Add sandboxClasses with an optional SandboxConfig ref to WorkerPool by @zoez7 in #2332
- gvisor: move the sandbox asset to release 20260907.0 by @ygao-g in #2413
- Health probes move to their own server on :9091, and :9090 serves only /metrics by @JeffLuoo in #2424
- Rename ContainerWakeupProbe to WakeupProbe by @laoj2 in #2575
- Remove the substrate version label from the install by @HavenXia in #1812
- Block link local egress in atunnel by @keithmattix in #2319
- stop naming every cached sandbox asset runsc- by @BenTheElder in #2627
- Changed SandboxClass from enum to string to allow extensibility by @laoj2 in #2586
Bug Fixes
- objectstore: finish snapshot cleanup when the bucket is missing by @cloudonly in #1973
- Components honor OTEL_TRACES_EXPORTER=none and treat OTEL_METRICS_EXPORTER=prometheus as none where they serve /metrics by @QuentinBisson in #2190
- Return the caller's context error when an atelet call fails by @eren23 in #2236
- ateom-gvisor: attribute runsc output for the pause container by @cloudonly in #2330
- Benchmark automation runners can reach ate-api-server again under ate-system's default-deny ingress by @haiyanmeng in #2637
- ateapi: fail /readyz while the store is unreachable by @ericcurtin in #1989
- run ateom under minimal init by @BenTheElder in #2626
- atunnel: bound the tunnel open by a deadline by @ygao-g in #2654
Features: Networking and Egress
- atenet: keep the router pod unready until its Envoy is serving by @LiorLieberman in #2325
- Egress gateway can mint and cache actor JWTs by @thompsonmax in #2135
- Egress policies can inject actor JWTs by @thompsonmax in #2141
- More robust Actor DNS Relay by @bowei in #2370
- Add E2E HTTPS egress tests and reject WebSocket upgrades by @swiftdiaries in #1634
- Add actor egress benchmarking tool by @yanavlasov in #2505
- Reduce number of egress atunnels by multiplexing CONNECTs over H/2 by @yanavlasov in #2425
- atenet-router: reach atunnel through mTLS CONNECT tunnels by default by @LiorLieberman in #2618
- Router status private by @LiorLieberman in #2573
- Block DNS if there is no egress policy configured by @bowei in #2524
Features: Security and Identity
- atespace isolation tests by @siyanshen in #1966
- security: prevent path traversal in snapshot storage locations by @BenTheElder in #2372
- atelet: introduce declarative validation for the AteomHerder RPCs by @shrutiyam-glitch in #1997
- ate-system now denies ingress by default; in-cluster API clients must label their namespace ate.dev/api-access=true by @destijl in #2522
- add support for connect from atenet (envoy) to atunnel by @LiorLieberman in #2531
Features: Observability
- ateom-gvisor: emit actor usage records by @baizhenyu in #2194
- ateom-microvm: emit actor usage records by @baizhenyu in #2195
- atelet: drop its actor usage records and count each activation's CPU once by @baizhenyu in #1984
- benchmarking/analysis: aggregate suspend/resume phase breakdown log r… by @Oneimu in #2009
- Drop per-pod server.address label from rpc.client.call.duration metrics by @JeffLuoo in #2576
Features: Workers and Actors
- fake-dataplane: let the fake data plane host actors by @haiyanmeng in #2300
- Build the OCI spec in ateom instead of atelet by @HavenXia in #2293
- Allow ActorTemplates to disable golden snapshot creation by @cloudonly in #2328
- Benchmark external volumes with the DurDir load by @anishgang in #2057
- ateom-microvm: boot the guest image over virtio-pmem with DAX by @jiayimeow in #2218
- microvm: evict snapshot page cache on restore to prevent cgroup memory pressure by @chw120 in #2030
- Move static runsc flags to configuration file by @fvoznika in #2281
- drop wall-clock TestWaitForSocketReturnsPromptly, it flakes on loaded ci runners by @BenTheElder in #2625
- fix kind cluster delete hanging when nfs volumes are mounted by @BenTheElder in #2621
Features: Install and Operations
- benchmarking: pace boomer to a target rate and add the resumepaused user class by @haiyanmeng in #2350
- ate-setup: have EnsureCRDs create only the missing CRDs by @HavenXia in #2585
- Add --fake-data-plane to install a fake data plane for control-plane benchmarks by @haiyanmeng in #2469
Dependencies
- Envoy, Alpine and dev-tool dependency bumps clear pre-GA CVEs by @annapendleton in #2568
Documentation
- Lifecycle metrics omit empty template labels, and a crash during a delete reports the delete operation by @JeffLuoo in #2356
- Upgrade Runbook GA by @HavenXia in #1844
Other Changes
- Build the JWK set that publishes actor JWT verification keys by @thompsonmax in #2132
- ate-setup: do not record demo and benchmark deploys by @HavenXia in #2320
- ate-setup should record the Substrate version an install used by @HavenXia in #2321
- Volume E2E tests wait for actor restoration before checking mounts by @cloudonly in #2294
- docs(router): fix the request parking guide link by @ktsoator in #2327
- Authenticate bundled PostgreSQL logins with separate pod certificates by @iplay88keys in #2266
- ateapi: record one scheduler assignment for each resume by @JeffLuoo in #2244
- Delete unused
actor_template_reffromActorAssigmentsub-resource by @juli4n in #2322 - benchmark(ateom-gvisor): log per-actor checkpoint and restore phase b… by @Oneimu in #2302
- Don't default name in store layer. by @juli4n in #2359
- kubectl ate admin can rotate the actor JWT signing key by @thompsonmax in #2140
- cdiinject: confine device paths and host mounts to allowed roots by @BenTheElder in #2315
- Add sections about sub-resources and singletons to the API style guide. by @juli4n in #2354
- ate-setup: choose the StorageClass of the bundled PostgreSQL by @ericcurtin in #2314
- Move snapshot transfers behind the object-store snapshot plugin by @ILLUM1N0X in #2224
- Fail snapshot plugin calls with Unavailable after 30s when the plugin is down by @aramase in #2380
- docs(api): fix systeminfo volume examples by @ktsoator in #2346
- ateom-gvisor logs a warning when a running actor's sandbox dies by @git286 in #2258
- Update Snapshots API to support Actor Lifecycle v2 by @msau42 in #2289
- Reject requests with duplicate ate-target-actor headers across HTTP and CONNECT by @destijl in #2308
- docs: add substrate security guide and principles by @destijl in #2326
- Rename some
ExternalVolumefields by @juli4n in #2404 - kubectl-ate: add --source to select container or lifecycle log lines by @erain in #2337
- ate-setup sets the actor JWT issuer (ACTOR_JWT_ISSUER), and ate-api-server requires it by @thompsonmax in #2257
- Rename target_ateom_uid to worker_pod_uid by @laoj2 in #2423
- Rename
MintActorJWTRequest.audiencetoaudiences. by @juli4n in #2421 - Remove makefile by @ahmedtd in #2374
- Rename the object-store plugin proto package to objectstore.v1 by @aramase in #2432
- podcertcontroller: do not rate-limit PCRs assigned to other replicas by @chw120 in #2353
- Add an eviction section to the API guide by @HavenXia in #2417
- Worker Self-Reported Sandbox & Hardware Compatibility by @shrutiyam-glitch in #2381
- Update COLLABORATING.md to reflect our stronger processes by @ahmedtd in #2437
- atenet-router no longer exposes its plaintext xDS server on the pod IP by @destijl in #2443
- cmd/atenet: Remove the router's "all" mode by @ahmedtd in #2441
- test: Fix macos unit test failures by @timflannagan in #2429
- Update x/net to v0.60.0 to fix govulncheck failures by @cloudonly in #2401
- ROOTFS snapshot fidelity: micro-VM capture and restore, API opened for micro-VM templates by @dberkov in #2448
- fix(kubectl-ate): validate atespace flags before connecting by @ktsoator in #2464
- atunnel: strip X-Ate-Assignment-Stale from upstream actor responses by @panchoruy in #2410
- WorkerPools resolve their SandboxConfig and protect it from deletion by @zoez7 in #2393
- Simplify Snapshot API for Actor Lifecycle v2 by @msau42 in #2449
- Singleton subresource Get and Delete requests take a name by @ericcurtin in #2452
- Fix a couple issues with plural commands in kubectl-ate by @thockin in #2509
- e2e: skip the egress credential injection test by @yufan-su in #2518
- atecontroller: add --worker-sync-concurrency flag by @chw120 in #2334
- ate-api-server no longer links the GCS and S3 SDKs by @ericcurtin in #2398
- Make
DrainWorkerreturnDrainWorkerResponseby @juli4n in #2501 ate-setup delete ate-systemnow deletes the ClusterTrustBundles Substrate published by @aramase in #2418- Update gVisor to 2026-10-10 nightly and use fs/ split checkpoint for durable dirs by @nybidari in #2455
- ate-setup: roll ate-api-server once when its environment changes by @HavenXia in #2371
- e2e: update credential injection e2e to call an in-cluster echo server by @yufan-su in #2535
- Fetch private sandbox assets through the object-store plugin by @aramase in #2439
- router authn/z part 1: Add simple mTLS client authentication to the ingress router by @ahmedtd in #2387
- ateapi: [Part 3] authorize actor lifecycle, egress policy, and worker RPCs with OpenFGA by @SHRUTI6991 in #2185
- Move the OIDC JWT verifier to internal/ so binaries other than ate-api-server can use it by @ILLUM1N0X in #2519
- Rename the snapshot-plugin sidecar, image and flag to objectstore-plugin by @aramase in #2540
- Report missing Kind and control-plane prerequisites before deployment by @cloudonly in #2403
- fix(objectstorage): handle download reader close errors by @ktsoator in #2446
- atecontroller: the :8080 metrics listener follows OTEL_METRICS_EXPORTER by @ericcurtin in #1976
- Use TitleCase for initialisms in API proto names. by @juli4n in #2520
- hack: warn when the host's neighbor table is at the kernel default by @dims in #2497
- Credential provider tests no longer fail with text file busy by @ericcurtin in #2549
- benchmarking: append the suspend/resume phase percentiles to stats.jsonl from the runner by @Oneimu in #2148
- Align timestamp fields with AIP-142 naming conventions by @ganeshkumarashok in #2438
- e2e: fetch in-cluster echo origins in the egress MITM suite by @dims in #2561
- cmd/atelet: Join the pause-image pull in the prewarm download test by @dims in #2564
- Fix ETXTBSY test flake in credentialprovider by @BenTheElder in #2365
- ate-setup: refuse to point an external-database install at the bundled one by @HavenXia in #2588
- ateapi: include actor identity in ateom certificates by @christian-posta in #2582
- Add and use a CLI verbosity flag by @thockin in #2475
- docs: document demo & benchmark install settings with their demos by @stshrive in #2572
- Update atelet Snapshot API for to support asynchronous uploads by @msau42 in #2592
- atelet: preserve tar entry mtimes when unpacking image layers by @jiayimeow in #2216
- e2e: Update egress e2e to use in-cluster test server and remove every external dependencies by @yufan-su in #2590
- Add WorkerService.ReportActorSnapshotState API for atelet snapshot reports by @igooch in #2612
- ateapi: grant system component roles to Substrate's own components by @SHRUTI6991 in #2220
- Wrap page size and next page token in a
ListOptionsmessage. by @juli4n in #2593 - atelet prepares each local worker's sandbox assets and versions file by @zoez7 in #2521
- Define a paused actor by its snapshots and delete the PAUSED and PAUSING states by @zoez7 in #2620
- benchmarking: add the personal-assistant-light agent-session script by @aditya-shantanu in #2583
- Mark workers unavailable if it's pod status changes by @sfunkenhauser in #2055
- Stamp snapshots with SandboxRuntime and schedule restores on runtime-compatible workers by @shrutiyam-glitch in #2536
- csi: require serverName when TLS is enabled. by @lekhoavu2909 in #2603
- WorkerService.MintAteomActorJwt mints ateom-for-actor JWTs for atelet; actor JWT claims move from ate.dev to actor.ate.dev by @ILLUM1N0X in #2523
- Move nighthawk benchmark server out of ate-system namespace by @yanavlasov in #2656
- Add proto message options to attach resource metadata. by @juli4n in #2454
- benchmarking: full request history, histogram restore/checkpoint, active_actors by @Nishanth29 in #2221
- fix(atelet): fall back to anonymous GCS read with objectstore-plugin Internal by @annapendleton in #2614
- atelet: Prefer local snapshot on Restore by @msau42 in #2658
- Fix macOS DNS relay tests by @swiftdiaries in #2667
- ateapi: [Part 4] deny RPCs without an authz rule and authorize tags by @SHRUTI6991 in #2562
- Consolidate API validation for sandbox class across services by @laoj2 in #2587
New Contributors
- @eren23 made their first contribution in #2236
- @siyanshen made their first contribution in #1966
- @erain made their first contribution in #2337
- @panchoruy made their first contribution in #2410
- @ganeshkumarashok made their first contribution in #2438
- @christian-posta made their first contribution in #2582
- @jiayimeow made their first contribution in #2218
- @fvoznika made their first contribution in #2281
- @lekhoavu2909 made their first contribution in #2603
Full Changelog: v0.4.0...v0.5.0
Committers in this release
- @aditya-shantanu
- @ahmedtd
- @aiqubits
- @anishgang
- @annapendleton
- @aramase
- @baizhenyu
- @BenTheElder
- @bowei
- @christian-posta
- @chw120
- @cloudonly
- @dberkov
- @destijl
- @dims
- @erain
- @eren23
- @ericcurtin
- @fvoznika
- @ganeshkumarashok
- @git286
- @haiyanmeng
- @hajiler
- @HavenXia
- @igooch
- @ILLUM1N0X
- @iplay88keys
- @JeffLuoo
- @jiayimeow
- @juli4n
- @keithmattix
- @ktsoator
- @laoj2
- @lekhoavu2909
- @LiorLieberman
- @msau42
- @Nishanth29
- @nybidari
- @Oneimu
- @panchoruy
- @QuentinBisson
- @sfunkenhauser
- @SHRUTI6991
- @shrutiyam-glitch
- @siyanshen
- @stshrive
- @swiftdiaries
- @thockin
- @thompsonmax
- @timflannagan
- @yanavlasov
- @ygao-g
- @yufan-su
- @zoez7