github kubernetes-sigs/agent-sandbox v1.0.9
on GitHub

4 hours ago

🚀 Announcing Agent Sandbox v1.0.9!

We're excited to announce the release of Agent Sandbox v1.0.9! Highlights: native gRPC proxying in sandbox-router (including sandboxd's ProcessService), a Helm chart for the router, working WebSockets through the router again (broken since v1.0.7), directory watching in sandboxd, fixes for warm-pool adoption and suspend in the controllers, streaming command execution in the Python SDK, and fail-fast, liveness, and pre-pull improvements in examples/agent-sandbox-rl.

⚠️ Action Required

  • Router NetworkPolicy now allows egress to TCP 9090 (#1880): The NetworkPolicy in sandbox-router/deploy/ (applied by kubectl apply -k sandbox-router/deploy/) now allows egress on TCP 9090 as well as 8888, so the router can reach sandboxd's gRPC port. The rule uses namespaceSelector: {}, so it covers every Pod in every namespace. With the default --authz-mode=allow-all, any client that can reach the router can target port 9090 on any Pod IP through X-Sandbox-Pod-IP.
    • Action: If you don't proxy gRPC through the router, remove the 9090 entry before applying. Otherwise, limit the rule to sandbox Pods with a podSelector on agents.x-k8s.io/sandbox-name-hash (Exists), as the new router Helm chart does. OLM installs don't include this NetworkPolicy.
  • Go label constants moved (#1878): SandboxNameHashLabel moved from sigs.k8s.io/agent-sandbox/controllers to sigs.k8s.io/agent-sandbox/api/v1beta1. PodSandboxNameHashLabel and PodWarmPoolLabel were removed from sigs.k8s.io/agent-sandbox/sandbox-router/cache. Label values are unchanged.
    • Action: Go code that imports these constants must switch to v1beta1.SandboxNameHashLabel and v1beta1.SandboxWarmPoolLabel.

Highlights

Core Controller

  • volumeClaimTemplates validated at admission (#1886): Sandbox, SandboxTemplate, and SandboxClaim now reject volumeClaimTemplates entries with a missing or empty metadata.name, duplicate names, or names longer than 63 characters, as well as lists of more than 128 entries. Apart from the 128-entry cap, these manifests were accepted before but never produced a running Pod: the claim reported Ready=False (VolumeClaimTemplatesError), or PVC or Pod creation failed. Existing objects aren't re-validated. Helm doesn't upgrade CRDs, so Helm users must run kubectl apply -f helm/crds/ to get the new rules.
  • Claims no longer stuck at Ready=False after warm adoption (#1944): During bursts of concurrent claims, a SandboxClaim could stay Ready=False after adopting a Ready warm Sandbox. Its status write hit a 409 conflict and was dropped, and nothing triggered another reconcile, so SDK readiness waits timed out. On a conflict, the controller now re-reads the claim from the API server and retries the status write once. It also reconciles a claim when the owner references of a Sandbox it owns change, such as the handoff from the SandboxWarmPool to the claim.
  • No second Sandbox after warm adoption (#2010): A SandboxClaim reconciled from a stale cache could cold-create a second Sandbox after it had already adopted a warm one, and it kept both until the claim was deleted. Before a cold create, the controller now re-reads the claim from the API server and skips the create if a Sandbox is already assigned.
  • Suspend no longer reports Finished=True (#2079): Suspend deletes the Pod, and since Kubernetes 1.27 the kubelet marks a deleted Pod Failed or Succeeded before removing it. The controller reported that as Finished=True until the Pod was gone. SandboxClaim mirrors Finished to start ttlSecondsAfterFinished, so suspending a claimed Sandbox with a very small TTL (such as 0) could expire the claim and delete the Sandbox and its PVCs. While spec.operatingMode is Suspended, Finished is no longer derived from the Pod phase.
  • Claim name on Pods (#1974, #1997): Pods of Sandboxes owned by a SandboxClaim now carry the agents.x-k8s.io/claim-name annotation with the claim's full name, next to the existing agents.x-k8s.io/claim-uid label. A service that holds a Pod can find its claim without reading the Sandbox's owner references. Warm-pool Pods get the annotation shortly after adoption. It's informational, not proof of identity. See SandboxClaim identity on Pods.
  • Controller flags from a ConfigMap (#1301): The controller now reads 19 allowlisted tuning flags (API client, worker concurrency, warm-pool, event, and write-behind settings) from the optional, user-created agent-sandbox-config ConfigMap in its namespace. This is for installs such as OLM, where you can't edit the Deployment's args. Command-line flags take precedence, unknown keys are logged and ignored, and an invalid value stops the controller from starting. The ConfigMap is read once at startup, so restart the controller after changing it. The core-only sandbox.yaml doesn't mount it, so it has no effect there. See Controller flag overrides.
  • Sandbox lifecycle metrics (#1775): New metrics, all labeled by namespace, sandbox_template, and owned_by:
    • the agent_sandbox_suspend_total and agent_sandbox_resume_total counters (by result)
    • the agent_sandbox_suspend_latency_ms and agent_sandbox_resume_latency_ms histograms
    • agent_sandbox_finished_total (by reason)
    • agent_sandbox_expired_total (by shutdown_policy), which counts Sandboxes that reach spec.shutdownTime. SandboxClaim lifecycle expiry isn't counted.

Sandbox Router

  • Native gRPC proxying (#1880): The Go sandbox-router now proxies gRPC to sandbox services, unary and streaming, including sandboxd's ProcessService. It's always on:

    • Port 8080 now also accepts cleartext HTTP/2 (h2c), and gRPC also works over the optional TLS listener. The router dials sandboxes over h2c.
    • Clients send the usual routing headers as gRPC metadata, with an explicit x-sandbox-port. There's no 8888 default for gRPC; sandboxd uses 9090. Each RPC is authorized separately.
    • The new --grpc-proxy-timeout (default 0s, no limit) caps RPC duration. --proxy-timeout doesn't apply to gRPC.
    • --path-routing-prefix and gRPC-Web aren't supported.
    • The router strips authorization, so requests through it can't authenticate to a sandboxd started with --auth-token-file or --auth-token-sha256.
    • The SDKs still connect to sandboxd by port-forward or Pod IP.

    See examples/sandbox-router-grpc/, and the NetworkPolicy change under Action Required.

  • WebSockets through the router work again (#2065): Since v1.0.7, every WebSocket or other Upgrade request through the Go sandbox-router (code-server, for example) got a 502 instead of a 101. The byte counters added in v1.0.7 (#1502) wrapped the upgraded connection in a read-only body, so the proxy rejected it. The router always enables metrics, so there was no workaround. Upgraded connections now also count toward sandbox_router_upstream_rx_bytes_total and sandbox_router_upstream_tx_bytes_total.

  • Helm chart for the router (#2044): New chart in sandbox-router/helm/ (chart version 0.1.0). It renders the same resources, with the same names, as sandbox-router/deploy/.

    • It isn't published to a chart repository. Install it from a checkout of the release tag with helm install sandbox-router ./sandbox-router/helm/ --namespace agent-sandbox-system --create-namespace --set image.tag=v1.0.9 (image.tag is required).
    • Values cover TLS (tls.enabled), authz.mode (tokenreview binds system:auth-delegator), a namespace-scoped Pod cache (router.cacheNamespace), and NetworkPolicy peers.
    • Its NetworkPolicy allows egress only to sandbox Pods, on any port.
    • Helm won't adopt resources created by kubectl apply -k sandbox-router/deploy/, so delete those first.

sandboxd

  • Directory watching (#1767): New server-streaming RPC watcher.v1.FileWatcherService/WatchDir on the gRPC port (9090). It streams CREATE, WRITE, REMOVE, RENAME, CHMOD, and ERROR events for a directory, optionally recursively, in batches of about 250 ms.
    • A RENAME event carries the old path. The new path arrives as a CREATE if it lands inside the watched tree.
    • If the watched directory is removed or renamed, the stream sends a final ERROR and closes.
    • Each sandboxd allows 64 concurrent watches.
    • Go SDK: Sandbox.WatchDir() and Watchers().WatchDir(). Python SDK: sandbox.watcher.watch(), sync and async (requires the grpc extra).
    • Older sandboxd versions return UNIMPLEMENTED.
  • Append mode for PUT /v1/files/{path} (#2077): With ?append=true, the request body is appended to the file instead of replacing it. The file and its parent directories are created if missing, and mode applies only when the append creates the file. Appends aren't atomic, and a retried append adds the body again. Older sandboxd versions ignore append and overwrite the file. The SDKs don't expose it yet.
  • Expected-kind check for GET and HEAD /v1/files/{path} (#2088): With type=file or type=directory, a path that exists but is the other kind returns 409 (CONFLICT) instead of the file bytes or directory listing. Any other type value returns 400. Older sandboxd versions ignore type. The SDKs don't send it yet.

SDKs

  • Python SDK:
    • New sandbox.commands.start() streams a command's output (#2054). It returns a ProcessHandle (AsyncProcessHandle on the async client). The handle's events() yields ProcessOutputEvent chunks and then a ProcessExitEvent, and signal() sends SIGINT, SIGTERM, or SIGKILL. Requires the grpc extra and SandboxdPodTunnelConnectionConfig or SandboxdInClusterConnectionConfig.
    • New sandbox.health() and sandbox.metadata() return SandboxHealth and SandboxMetadata from sandboxd (#1722).
    • New optional auth_token on SandboxdPodTunnelConnectionConfig and SandboxdInClusterConnectionConfig, for a sandboxd started with --auth-token-sha256 or --auth-token-file (#2024). The sync and async clients send it on REST requests and gRPC calls.
    • New SandboxFileConfig(max_download_size=...), passed as file_config= to SandboxClient or AsyncSandboxClient, caps files.read() and files.read_to() (#2041). The default, None, keeps downloads unlimited. Oversized downloads raise the new SandboxResponseTooLargeError.
    • Fixed a memory leak: long-lived SandboxClient and AsyncSandboxClient instances kept a reference to every Sandbox closed with sandbox.terminate() (#2085).
    • client.k8s_helper.wait_for_claim_ready() accepts opt-in require_current_generation=True, which ignores a Ready condition from before a claim spec update (#1872).
  • Go SDK (#2007): With RuntimeSandboxd over the default pod port-forward, Run and Commands().Start now return ErrNotReady (wrapping ErrPortForwardDied) after the tunnel dies, instead of a gRPC Unavailable error. The README's port-forward recovery recipe can then reconnect with Open().
  • TypeScript SDK (#2062): New SandboxClient.getSandboxByName() attaches to a core Sandbox without a SandboxClaim. It waits for Ready unless you pass waitForReady: false. The client doesn't track or delete these Sandboxes.
  • MCP server (#2040): New connection type sandboxd-in-cluster (K8S_SANDBOX_CONNECTION__TYPE) lets an in-cluster MCP server use sandboxd-based Sandboxes without the router or port-forwarding. Set K8S_SANDBOX_CONNECTION__MODE to in-cluster-service or in-cluster-pod-ip. The template's NetworkPolicy must allow the MCP server on ports 8080 and 9090.

Reinforcement Learning (examples/agent-sandbox-rl)

  • Fail fast on claims that can't start (#1791): SandboxFleet.acquire() now watches the claimed Pod and is on by default.
    • On InvalidImageName, ErrImageNeverPull, an OOM-killed container, or a Pod in phase Failed or Succeeded, it deletes the claim and raises SandboxStartError instead of waiting out ready_timeout (900s).
    • Retryable reasons such as ImagePullBackOff, CreateContainerConfigError, and CrashLoopBackOff fail only after FailFastPolicy.grace_s, and Unschedulable only after unschedulable_grace_s. Both are unset by default.
    • To turn it off, pass FleetConfig(fail_fast=FailFastPolicy(enabled=False)).
  • Liveness and cleanup (#1791, #1966):
    • SandboxHandle.is_alive() detects a deleted, finished, or replaced Pod, and exec() raises SandboxLostError when the Pod is gone.
    • reap_orphans(alive_run_ids) (CLI: --orphans --alive-run-ids) cleans up runs that aren't listed. It supports --min-age-s (default 300) and --dry-run.
    • teardown(verify=True) and verify_teardown() run the run-scoped reaper afterwards and report what's left (#1966).
  • External training loops (#1963): LookaheadWarmer keeps warm pools sized for the batches a trainer has in flight. add_tasks() extends a fleet's task plan in place, and wait_pools_ready() waits up to timeout without raising. All three work with AsyncSandboxFleet.
  • Pod templates (#1961):
    • New ResourceSpec.cpu_limit and memory_limit.
    • An extra_pod_spec container with image {task_image} runs the task image, for example as a grading sidecar.
    • TemplateSpec.extra_template_spec adds SandboxTemplate spec fields.
    • Rendered Pods now set automountServiceAccountToken: false and shareProcessNamespace: false explicitly. These were already the effective defaults.
  • Multi-container Pods (#1960): exec(container=), open_session(container=), and SandboxHandle.target_container choose the container. The default is the runtime container.
  • OpenHands (#1962): adapters.openhands.agent_server_template() returns a TemplateSpec for OpenHands agent-server pools. The server comes from the task image or is copied in from server_image=. Pods get /health startup and readiness probes, and a session key is required unless allow_unauthenticated=True.
  • Task sources (#1964): load_tasks, JsonlSource, and to_tasks read R2E-Gym and SWE-bench rows; override the field names with image_field= and id_field=. load_image_list and ImageListSource read image manifests from local paths or gs://.
  • Pre-pull (#1965): Images are pre-pulled by DaemonSets of prepull_images_per_pod images each (default 25), at most prepull_max_parallel at a time (default 4), using the template's tolerations and runtime class.
    • New optional prepull_delete_when_done and prepull_drop_caches.
    • Fixed a second run's pre-pull failing on the DaemonSet selector.
  • Testing (#1967): agent_sandbox_rl.testing.FakeFleet and FakeHandle provide an in-memory fleet for unit-testing training loops without a cluster.
  • Client setup (#2035): Fixed concurrent first use building more than one SandboxClient. Each cluster now builds one client from its own ApiClient. The package now requires k8s-agent-sandbox>=1.0.6.

Examples

  • DeerFlow (#2039): New examples/deer-flow-sandbox implements DeerFlow's sandbox provider on the Python SDK and sandboxd, so DeerFlow tool calls run in Sandbox Pods.
  • Deep Agents (#2064): New examples/deepagents runs a LangChain Deep Agent whose file and shell tools run in a Sandbox, using the clients/integrations/deepagents backend.
  • Agent Client Protocol (#2017): examples/agentclientprotocol accepts -addr kubeservice://<context>/<namespace>/<service>[:<port>]. It port-forwards to the Service through the API server in-process, with no kubectl or local port.
  • sandboxed-tools (#2019): Sessions now record their toolset, parent session, and creation time.
  • VS Code (#2069): examples/vscode-sandbox documents attaching desktop VS Code to the sandbox Pod.

Testing, CI & Tooling

  • Kata Containers (#2023, #2031): New test-e2e-kata-gcp periodic script runs the TestRuntimeClass* E2E tests with kata-qemu on a kOps/GCE cluster with nested virtualization. The shared kOps/GCE setup moved into dev/ci/shared/kops-gcp-lifecycle.sh. The Prow job isn't registered yet (kubernetes/test-infra#38129).
  • Peer-pods in E2E (#2029): The RuntimeClass E2E tests now support kata-remote (peer-pods). New SANDBOX_CONTROLLER_WORKERS sets the test controller's worker counts and is derived automatically for kata* runtime classes. SANDBOX_DEFAULT_TIMEOUT makes the 60s default timeout configurable.
  • Fewer slow E2Es (#1953, #2048): Replaced four shutdown-policy E2Es with unit tests, and trimmed the warm-pool rollout E2E to its Recreate case.
  • Tooling:
    • Go unit-test coverage profiles from make test-unit (#2045)
    • a logged, bounded Boskos heartbeat in resourcectl (#2047)
    • amd64-only kOps benchmark images via the new push-images --platform (#2032)
    • Helm lint follow-ups (#2028)
  • Dependencies: Built with Go 1.27.3 (#2037). controller-runtime v0.25.2 (#1859).
  • Docs and contributor guidance: Clearer docs-site sidebar hierarchy (#2022). AGENTS.md additions on writing style, the condition lifecycle, and test suites (#2036, #2081, #1969).

Installation

Standard Install (Core + Extensions)

Recommended for most users and GitOps engines (Argo CD, Config Sync, kustomize): 

kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/v1.0.9/sandbox-with-extensions.yaml

Selective Install

Install components separately: 

# Core only:
kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/v1.0.9/sandbox.yaml

# Extensions (opt-in):
kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/v1.0.9/extensions.yaml

Python SDK

pip install k8s-agent-sandbox==1.0.9

Contributors

We extend our sincere thanks to all contributors to this release:
@BootstrapperSBL, @HasonoCell, @LucasGois1, @Quiyyy, @RachelWanggg, @YoungJinJung, @abhayjoshi201, @aditya-shantanu, @briankhoi, @dongjiang1989, @ericcurtin, @esposem, @fponsse, @hakman, @janetkuo, @justinsb, @leomcl, @niting, @stream7, @tomergee, @vvoronko, @wjun29

New Contributors

Full Changelog: v1.0.8...v1.0.9

Check out latest releases or
releases around kubernetes-sigs/agent-sandbox v1.0.9

Don't miss a new agent-sandbox release

NewReleases is sending notifications on new releases.

Get notifications