SkyPilot v0.14.3
Get it now with:
uv pip install "skypilot>=0.14.3"What's Changed
- [Docs] Job groups: rework 'Dynamically attaching jobs to a job group' for readability by @ishankaul1 in #10764
- [Dashboard] Add plugin slots for the Infra cell and field by @SeungjinYang in #10797
- [Docs] README: refresh the News section by @hopechong in #10795
- [Jobs]
include_treeon the managed jobs queue: one fetch for a job and its dynamic tasks by @ishankaul1 in #10767 - [Kubernetes] Main containers being created after init containers finish are not a stall by @MadhavJivrajani in #10801
- [Server] Cap the storage the API server keeps file mounts on by @aylei in #10793
- [Core] Say when a queued launch was admitted by @DanielZhangQD in #10802
- [Metrics] Count WebSocket handshake attempts by @cg505 in #10688
- [Test] Stop shrinking test_api_server_start_stop below its declared size by @DanielZhangQD in #10805
- [Server] Spawn ssh-proxy subprocesses without forking the API server by @aylei in #10804
- [CI] Pin googleapis-common-protos<1.75.1 to match the protobuf 5.29.6 dev pin by @lloyd-brown in #10808
- [Test] Match managed-job GCP instances on the name prefix that survives truncation by @lloyd-brown in #10809
- [Dashboard] Prevent empty-state flashes on Jobs refresh and scope switches by @rohansonecha in #10810
- [Slurm] Support executable sudo allowlists for user impersonation by @kevinmingtarja in #10811
- [Server] Keep the metrics event loop free while the federation routes work by @DanielZhangQD in #10812
- [Dashboard] Gray dash and tooltip for a job with no name by @ishankaul1 in #10816
- [Slurm] Support reduced sudo permissions without container snapshots by @kevinmingtarja in #10820
- [Jobs] Keep a task's submission and start times across a parked restart by @aylei in #10788
- [Jobs] Finalize managed jobs atomically instead of in separate writes by @aylei in #10386
- [Dashboard] Pass the job group's aggregate status to the detail status badge slot by @aylei in #10829
- [Tests] Reset the SkyPilotContext ContextVar between unit tests by @aylei in #10830
- [Jobs] Count the launch errors the controller swallows by @DanielZhangQD in #10831
- [Jobs] Say why a managed job failed to submit, not just that it did by @DanielZhangQD in #10834
- [Slurm] Keep the full text of scontrol node attributes that contain spaces by @rohansonecha in #10839
- [Metrics] Federate LiteLLM proxy metrics by @rohansonecha in #10841
- [Jobs] Don't construct an exception per unselected column when building job rows by @zpoint in #10832
- [Slurm] Stop teardown from issuing redundant sudo
rmon the login node by @aylei in #10844 - [Jobs] Resume a pipeline at the pending task after a controller restart by @cg505 in #10078
- [Docs] Link docs and README logos to skypilot.ai by @hopechong in #10847
- [Core] Status refresh: do not mark a cluster UP when its handle has no cached IPs by @kevinmingtarja in #10140
- [Core] Status refresh: keep the ray health check for stopped-then-restarted clusters by @zpoint in #10850
- [Slurm] Preserve the latest cluster job's exit code on teardown by @kevinmingtarja in #10851
- [Jobs] Report managed job tasks that stalled before they ever ran by @DanielZhangQD in #10824
- [Kubernetes] Stop workload pods holding the provisioner's permissions by @DanielZhangQD in #10845
- [Test] Stop the read-only workspace tests leaking IS_SKYPILOT_SERVER into their worker by @DanielZhangQD in #10854
- [Azure] Support ACR authentication via the VM's managed identity by @ahluwalij in #10148
- [Docs] Add analytics for a better user experience by @hopechong in #10857
- [Test] Check TailLogs tests for leaked threads by identity, not count by @SeungjinYang in #10860
- [Server] Accept padded JWT segments from external auth proxies (PyJWT 2.14) and warn when the header cannot be decoded by @cg505 in #10858
- [Core] Make per-user SSH key bootstrap converge across processes sharing a DB by @cg505 in #10785
- [k8s] Fail fast when a pod disappears while waiting for it to be scheduled by @cg505 in #10800
- [Test] Make the local-disk path-resolution tests deterministic under xdist by @cg505 in #10864
- [Docs] Point Try SkyPilot Platform button to /demo by @hopechong in #10865
- [Test] Skip the fusermount-server fd leak test on remote servers by @zpoint in #10868
- [Nebius] Filter grpc.aio poller errors from the asyncio logger by @cg505 in #10862
- Support for Nebius Spot Pricing by @Jobarion in #10853
- [Jobs] Support multiple runtimes and runtime-owned recovery by @kevinmingtarja in #10836
- [Helm] Compute overview: include Slurm GPU allocation; show 0% when idle by @zpoint in #10867
- [CLI] Add --output id|json to sky jobs launch by @aylei in #10879
- [Jobs] Keep a per-task history of the scheduler allocations runtimes report by @aylei in #10873
- [Jobs] Managed job dependencies by @aylei in #10876
- [Examples] Add Halo training recipe by @advpropsys in #10835
- [Deps] Make Vast opt-in: drop it from the [all] extra by @Michaelvll in #10895
- [Dashboard] Pin shepherd.js to 13.0.3 by @Michaelvll in #10892
- [Examples] Use the official InvokeAI image for the Stable Diffusion example by @Michaelvll in #10898
- [Slurm] Expand Slurm hostlists in-tree by @Michaelvll in #10890
- [Jobs] Support runtime log readers by managed job ID by @kevinmingtarja in #10884
- [Slurm] Protect the requeue sbatch option by @rohansonecha in #10904
- [Dashboard] Allow customizing list-page filter bars by @stephenkevn in #10875
- [Examples] Drop the copied xformers patch from vicuna-llama-2; own fusermount usage text by @Michaelvll in #10894
- [Test] Make the dynamic job group smoke tests pass on a shared basic-auth API server by @ishankaul1 in #10859
- [Dashboard] Plugin slots on the workspace Allowed Users list by @lloyd-brown in #10902
- [Core] Handle raw PermissionError in child process walk by @amitgelber in #10856
- [CI Fix] Use a user venv in apiserver-start-stop setup on bare cloud VMs by @lloyd-brown in #10912
- [Core] Create the default user Python env on VMs without conda by @lloyd-brown in #10919
- [Core] Keep reading skylet tunnel output so port-forward never blocks by @kevinmingtarja in #10922
- [CI Fix] Make test_cli_output border check robust; skip test_batch_diffusion on Nebius by @lloyd-brown in #10920
- [CI Fix] Wait for pool shutdown in pool teardown smoke tests by @lloyd-brown in #10918
- [Core] Keep skylet tunnel bookkeeping per API server host by @kevinmingtarja in #10923
- [Nebius] Delete the cluster security group after autodown by @lloyd-brown in #10917
- [Nebius] Wait for workers to terminate on worker-only teardown by @lloyd-brown in #10921
- [API] Schedule the post-save
sky checkas its own request instead of blocking config save by @ishankaul1 in #10821 - [Docs] Fix favicon rendering with black corners by @hopechong in #10951
- [CI] Nightly: run the Kubernetes jobs-consolidation lane with gRPC enabled by @kevinmingtarja in #10933
- [API] Validate client user ids used as filesystem paths by @DanielZhangQD in #10897
- [Helm] Spread API server pods per ReplicaSet with matchLabelKeys by @aylei in #10938
- [Jobs] Treat a just-ended launch as activity in the unattended stall scan by @aylei in #10941
- [API] Fail a request whose result cannot be stored by @aylei in #10946
- [Test] Fix flaky test_pool_autoscaling_scale_up_to_max_then_down_to_zero by @DanielZhangQD in #10957
- [Server] Allow pinning executor pool and jobs controller counts by @aylei in #10916
- [API] Export the API server container's memory and processes by type; reap gauges of reused pids by @aylei in #10943
- [API] Contain task file-mount sources to the backend's user roots by @DanielZhangQD in #10899
- [Tests] Allow retry-log streaming to attach before task failure by @kevinmingtarja in #10936
- [API] Skip stored results when listing request status by @aylei in #10939
- [Examples] Pin huggingface-hub<1.0 in the HuggingFace GLUE examples by @zpoint in #10971
- [Jobs] Leave job file contents and task YAML out of queue reads that do not show them by @aylei in #10944
- [Core] Restore sky.provision log propagation after each provision by @DanielZhangQD in #10968
- [AWS] Share one botocore loader across sessions by @aylei in #10931
- [Core] End a followed job log tail when its kubectl exec session is gone by @aylei in #10958
- [Kubernetes] OCI network_tier=best: pick NCCL env per node shape by @DanielZhangQD in #10967
- [DB] Select psycopg2 explicitly for sync Postgres engines by @cg505 in #10975
- [API] Stamp the reason and verified account on rejected credentials by @DanielZhangQD in #10970
- [CLI] Keep
sky status -o jsonstdout parseable by @exdsgift in #10763 - [Jobs] Reset each managed job at most once per HA recovery pass, via compare-and-swap by @cg505 in #10327
- [API] Validate the client-supplied user id before it becomes a cluster owner by @Khas-Erdene-Tsogtsaikhan in #10437
- [Test] Shrink test_job_group_rl_architecture to 0.5 CPU / 2 GB per pod by @zpoint in #10907
- [Kubernetes] Assign hostNetwork Ray ports on the server and retire the ray-ports ConfigMap by @DanielZhangQD in #10852
- [Test] Give the host-network get_cluster_info test a real provider_config by @DanielZhangQD in #10981
- [Core] Build the cost report a chunk of history at a time by @aylei in #10949
- [Core] Merge Python-computed Kubernetes pod fields after template render by @kevinmingtarja in #10973
- [Jobs] Add jitter to the managed job status-check gap by @cg505 in #10913
- [Test] Add Nebius GPU spot e2e smoke test by @cg505 in #10863
- [Docs] Drop the "under active development" banner from the Slurm docs by @Michaelvll in #10955
- [Jobs] Skip local file-mount blob when recovery sources are cloud URLs by @hentt30 in #10977
- [Jobs] Scope shared bucket uploads by workspace by @hentt30 in #10959
- [API] Fall back to the API server when an SSH redirect target fails by @DanielZhangQD in #10871
- [Core] Add OTLP as an external logging store by @zpoint in #10908
- [Kubernetes] Fix relaunching a hostNetwork cluster whose Ray stopped by @DanielZhangQD in #10986
- [Jobs] Let managed jobs in non-default workspaces launch on a jobs controller VM by @zpoint in #10979
- [Jobs] Make the consolidation-mode recovery sweep targeted and rate-limited by @aylei in #10557
- [Kubernetes] Use the nvidia-rtx-pro-6000 GKE label for RTX PRO 6000 by @MohammadHijjawi97 in #10926
- [Core] Redact credentials from debug dumps by @aylei in #10985
- [Core] Return freed memory to the OS under jemalloc in release_memory by @aylei in #10942
- [Core] Bound the log tail reader on logs with few newlines by @aylei in #10945
- [RunPod] Migrate to REST API v2 (v1 retires Nov 15, 2026; GraphQL early 2027) by @ciprianf-flexion in #10846
- [Test] Smoke-test hostNetwork relaunch after Ray stops, on the queue that can run it by @DanielZhangQD in #10987
- [Core] Tidy comments and docstrings by @Michaelvll in #10989
- [Jobs] Check network connectivity once per controller process per status gap by @cg505 in #10982
- [Test] Trim the ray patches payload timestamp test docstring by @DanielZhangQD in #11000
- [Core] Report provisioning errors on an existing cluster instead of asserting INIT by @DanielZhangQD in #11002
- [Test] Pin the GKE test cluster's pod range outside 10.0.0.0/9 by @zpoint in #11007
- [Kubernetes] Add remote_identity: NONE for workload pods with no API identity by @DanielZhangQD in #11001
- [Jobs] Release a job group's launch slot once it starts running by @DanielZhangQD in #11026
- [DB] Use the SQLite lock timeout for the sync engine, bounded on the auth path by @ciprianf-flexion in #11006
- [Jobs] Reset the status-check window only while the job is alive by @DanielZhangQD in #11025
- [API] Stop honoring the controller-name check bypass from requests by @DanielZhangQD in #11027
- [API Server] Run the API server middlewares as pure ASGI by @aylei in #11029
- [Tests] Pin both launches in test_managed_jobs_api_access to Kubernetes by @zpoint in #11031
- [Core] Compute the Kubernetes node selector and spot toleration in Python by @kevinmingtarja in #10990
- [Dashboard] Use k / TB suffixes for large CPU and memory counts by @claude[bot] in #11044
- [Docker] Bump base image to Python 3.13 by @kevinmingtarja in #10105
- [Core] Scope cluster history and cluster events to the caller's readable workspaces by @trevanth24 in #11024
- [Jobs] Add job.wait_for_scheduling_timeout to cancel managed jobs that never start by @trevanth24 in #11004
- [Jobs] Stagger job controller starts by @aylei in #10790
- [API] Refuse controller-only launch fields from non-loopback callers by @DanielZhangQD in #11051
- [Nebius] Reuse SDK clients and close retired instances by @SalikovAlex in #11036
- [Core] Bound provisioning commands so a wedged transport cannot hang forever by @zpoint in #10781
- [Jobs] Retry emergency bookkeeping through a database outage; stop when the job record is gone by @cg505 in #10978
- [Dashboard] Plugin access check for opening a workspace's page by @lloyd-brown in #11045
- [Recipes] Support recipes managed by an external source by @lloyd-brown in #11043
- [Core] Retry the cluster health probe on all clouds by @nordmtr in #11016
- [Docs] Move slime example from Agents to Training by @ishankaul1 in #11042
- [Dashboard] Fit more job details above the fold on the job page by @stephenkevn in #10991
- [Dashboard] Lay out the Infra, Clusters and Jobs headers for narrow screens by @stephenkevn in #11060
- [Core] Ignore client overrides of workspace_config by @trevanth24 in #11048
- [Users] Scope the user list and service-account tokens to private workspaces by @trevanth24 in #11023
- [Core] Gate cluster log and job reads on the cluster's workspace by @trevanth24 in #11047
- [Test] Isolate test_cluster_read_permission from leaked config env by @DanielZhangQD in #11070
- [Core] Ignore client overrides of admin_policy and logs by @DanielZhangQD in #11069
- [Jobs] Print a finished job's logs when the followed tail never saw it RUNNING by @zpoint in #11056
- [Serve] Re-resolve the upload blob on every replica launch in consolidation mode by @cg505 in #11066
- [SSH Node Pools] Deploy hosts that need a sudo password, without exposing it by @rohansonecha in #11067
- [Docs] Put SSO in the auth doc title so it shows in the sidebar by @aylei in #11072
- [Jobs] Do not read the runtime observation cursor from the DB on every poll by @cg505 in #10983
- [DB] Treat asyncpg shutdown, resource and read-only errors as transient by @cg505 in #11057
- [Server] Add managed job submission plugin hook by @kevinmingtarja in #10796
- [Slurm] Pick the skylet keeper's node from squeue, like the provisioner by @ryxli in #11063
- [Test] Set logs.store on the API server in the log store smoke tests by @zpoint in #11081
- [Dashboard] Show the entrypoint and task YAML of cluster jobs by @zpoint in #11050
- [SSH Node Pools] Generate a random k3s join token per pool by @DanielZhangQD in #11079
- [API Server] Cut auth CPU for service-account Bearer requests by @aylei in #11074
- [Tests] Skip log store smoke tests under --dependency by @zpoint in #11083
- [Core] Stop rebuilding the cached wheel on every launch after a source mtime bump by @kevinmingtarja in #11082
- [Core] Only include SSH credentials in status for clusters the caller owns by @kevinmingtarja in #11085
- [Test] Raise the jobs-queue SLA in the scale test and fix false negatives in its output checks by @DanielZhangQD in #11091
- [Test] Read the jobs controller name from the status table in test_multi_tenant_managed_jobs by @zpoint in #11092
- [Test] Wait for the scale test's RDS instance to be deleted before removing its groups by @DanielZhangQD in #11096
- [Jobs] Emergency recovery keeps a running job on its cluster by @cg505 in #10963
- [API] Back off a log stream's status check while the request is parked by @cg505 in #11084
- [Catalog] Read AcceleratorName as a string so a catalog with no GPU rows can be filtered by @ishankaul1 in #11101
- [Verda] Keep the existing catalog when no GPU instance type is available by @ishankaul1 in #11102
- [Jobs] Propagate scheduler subprocess exit code in consolidation mode by @cg505 in #9646
- [Kubernetes] Map H100-MEGA to the nvidia-h100-mega-80gb GKE label by @MadhavJivrajani in #11104
- [API] Drop unused FastAPI prefix argument for FastAPI 0.145+ by @vnavkal in #11116
- [Core] Rename workspaces while they have active resources by @DanielZhangQD in #11055
- [Test] Restore the config env var after admin policy tests by @DanielZhangQD in #11119
- [Test] Pin fastapi<0.145 in the backward compatibility base env by @DanielZhangQD in #11120
- [Core] Stop shipping another server's or checkout's cached SkyPilot wheel by @DanielZhangQD in #11121
- [Dashboard] Make the viewer role selectable for users and service accounts by @kevinmingtarja in #11090
- fix(slurm): treat CONFIGURING allocations as INIT in query_instances by @ryxli in #11115
- fix(slurm): accept Slurm's full sbatch time grammar and check it at load by @ryxli in #11134
- fix(slurm): remove a stale global-scope container before every launch by @ryxli in #11135
- [Azure] Update A10 GRID driver for vGPU 18 by @ahluwalij in #10469
- [Jobs] Launch a pool job again when its controller restarts before a worker is picked by @aylei in #11138
- [Logs] Placement-aware logging agents and following running managed jobs from the log store by @aylei in #11076
- [Core] Upgrade the bundled Ray to 2.56.1 by @DanielZhangQD in #10585
- [Test] Give the ray-port provisioner test a region by @DanielZhangQD in #11146
- [Images] Bake VM images from a given SkyPilot commit; switch AWS/GCP to -ray256 images by @DanielZhangQD in #11126
- [Test] Treat the local-disk test roots as charged so they do not depend on the runner's /tmp mount by @ishankaul1 in #11151
- [Jobs] Retry a finished job's storage teardown before failing it by @MadhavJivrajani in #11150
- [Core] Let plugins register per-workspace config properties by @lloyd-brown in #11132
- [Kubernetes] Share aws eks get-token results across kubectl processes on the API server by @aylei in #11155
- [API Server] Resolve the active workspace for /validate by @trevanth24 in #11100
- [Kubernetes] Let a task's remote_identity: NONE beat per-context config by @DanielZhangQD in #11137
- [Kubernetes] Tell workers the head's Ray port when its GCS is not ready yet by @DanielZhangQD in #11161
- [k8s] Check a cluster's owner by its recorded context and cluster UID by @aylei in #11127
- [Jobs] Preserve any_of semantics when adding Kubernetes annotations by @dyld-w in #10490
- [Core] Let
sky api cancel <id>cancel another user's request for admins by @zpoint in #11093 - [Dashboard] Let a plugin table column sort by a value it computes by @zpoint in #11144
- [Jobs] Check workspace read access in /jobs/events by @DanielZhangQD in #11163
- [CI] Fail docker build instead of overwriting an existing image tag by @zpoint in #11164
- [Docker] Upgrade pip to clear critical certifi advisory in Trivy scan by @zpoint in #11167
- [API] Drop client-supplied DB pool and timeline env vars before the request env overlay by @DanielZhangQD in #11166
- [Core] Don't fail reads on a read-only workspace when its enabled-clouds cache is cold by @DanielZhangQD in #11162
- [API] Ignore a client-supplied ssh_proxy_command override by @DanielZhangQD in #11168
- [Jobs] Drain and bound the API server step-down on consolidation lock loss by @aylei in #11169
- [Examples] Add flying-roadster-rl: RL post-training an LLM pilot by @Michaelvll in #11171
- [Dashboard] Show a not-found page for unknown URLs by @stephenkevn in #11173
- [Jobs] Fix completed-task logs while a job group is active by @kevinmingtarja in #10826
- [Plugins] Add a hook that observes the effective request after admin policy by @zpoint in #11124
- [k8s] Cap the connections each Kubernetes API client keeps open by @kevinmingtarja in #11136
- [Test] Fix order-dependent flake in test_check_server_healthy_or_start_rechecks_status by @zpoint in #11175
- [Serve] Keep the autoscaler version and replica ids across controller recovery by @leshanbog in #10960
- [Kubernetes] Harden fusermount-server by @aylei in #11170
- [API Server] Find clusters in newly created workspaces when proxying SSH by @kevinmingtarja in #11178
- [Jobs] Show a pending managed job's submission time in the queue by @kevinmingtarja in #11177
- [Core] Limit infra errors and probes to the caller's allowed contexts and clusters by @DanielZhangQD in #11195
- [Jobs] Name the submission log by job id span so batch launches fit the file name limit by @DanielZhangQD in #11194
- [Core] Scope infra-read endpoints to the caller's workspaces by @DanielZhangQD in #11160
- [Test] Check only SkyPilot's own routes in the infra guard test by @DanielZhangQD in #11198
- [Slurm] Drop login shell banners from command output and allow non-GPU gres by @eugeneteoh in #11054
- [Jobs] Add a per-user limit on claimed jobs that are not running by @lloyd-brown in #11176
- [EFA] Skip warm nodes running a different accelerator when sizing EFA by @trevanth24 in #10492
- fix(api): bound SQLite request-retention cleanup by @vilmos-uagi in #11089
- [Azure] Bake Ray 2.56.1 into the default images (-ray256 gallery definitions) by @DanielZhangQD in #11196
- [Dashboard] Keep Duration next to Submitted in job details by @stephenkevn in #11205
- [Jobs] Count jobs against the controller job cap in cost units by @cg505 in #11208
- Fix AWS VPC failover across regions by @amitgelber in #11040
- [Serve] Preserve installed TLS credentials across updates by @amitgelber in #11041
- [Serve] Resume persisted replica teardown during recovery by @amitgelber in #10717
- [Storage] Bound controller staging buckets and preserve upload errors by @amitgelber in #10901
- [API Server] Add guide and tool to migrate an API server from SQLite to PostgreSQL by @amitgelber in #10988
- [Buildkite] Stop the pipeline generator reading --serve as --serve-consolidation by @zpoint in #11221
- [Jobs] Don't hold back recovered jobs with a live task under the per-user launch limit by @cg505 in #11219
- [Core] Merge the default user blocklist into a custom rbac.roles.user by @DanielZhangQD in #11222
- [AWS] Enable ap-south-2 and refetch stale zone mappings by @zpoint in #11207
- [Test] Dashboard e2e tests against a seeded API server, with twists by @zpoint in #11174
- [Dashboard] Let non-admin users create, rotate and delete their own service accounts by @lloyd-brown in #11235
- [Jobs] Make copying task logs into the controller log optional by @samuelv8 in #11212
- [Test] Dashboard e2e: fix flakes on busy runners (warm-up, server sizing, teardown, waits) by @zpoint in #11229
- [Core] Retry a stale or briefly missing request config file by @cg505 in #11224
- [Jobs] Restore a job's config file only when it is missing by @cg505 in #11234
- [Docs] Update SkyPilot Platform links and move Sandboxes under Jobs by @romilbhardwaj in #11236
- [CI] Pause the monthly auto-trigger for the release pipeline by @zpoint in #11240
- [Nebius] Readable, fail-fast sky check; progress-based launch timeout; SA setup docs by @Michaelvll in #11250
- [Kubernetes] Parse the kubeconfig again only when it changes by @DanielZhangQD in #11232
- [Core] Skip config YAML dumps that no handler would log by @DanielZhangQD in #11231
- [Kubernetes] Keep Kubernetes clients across a sync handler's config refresh while the kubeconfig is unchanged by @DanielZhangQD in #11237
- [Kubernetes] Keep recording a deleted pod's events past same-second repeats by @DanielZhangQD in #11241
- [Test] Purge failed pools in pool smoke test teardown by @aylei in #11255
- [Core] Record why a status refresh removed a cluster, and show it in a Details column by @DanielZhangQD in #11239
- [Core] Show what ended a cluster as its last event, not a later status refresh by @DanielZhangQD in #11253
- [Kubernetes] Retry a launch when its Kubernetes context is unreachable by @aylei in #11254
- [Kubernetes] Name the node and its state in abnormal-cluster events by @DanielZhangQD in #11252
- [CLI] Support --all for sky serve status by @lgqfhwy in #11256
- [Docs] Remove seed announcement banner and refresh README news by @hopechong in #11259
- [Dashboard] Show Infra with zone for terminated clusters by @bufocoder[bot] in #11260
- [Kubernetes] Retry pod creation while an admission webhook is unreachable, then report it by @kevinmingtarja in #11080
- [Test] Fail get_available_gpus when the infra is reported not enabled by @kevinmingtarja in #11263
- [Core] Remove ExtensionContext.register_rbac_rule, which is never enforced by @DanielZhangQD in #11265
- [Core] Make advisory and lease leader election exclude each other by @aylei in #10956
- [Core] Fix YAML loader fallback with LazyImport by @cg505 in #11262
- [Core] Floor ibm-vpc to wheel releases and sort the [all] extra by @zpoint in #11275
- [API] Let service accounts act on behalf of users by @aylei in #11268
- [Test] Deflake test_a_second_principal_does_not_reload_immediately by @MadhavJivrajani in #11278
- [Tests] Skip the webhook-unavailable smoke test against a remote API server by @rohansonecha in #11279
- [Server][Jobs] Do not start a local API server from inside the API server by @cg505 in #10571
- [CI] Notify downstream only after the nightly Docker image is pushed by @zpoint in #11276
- [Test] Dashboard e2e: bound wait_for_response; retry reading the config mid-rewrite by @zpoint in #11267
- [Storage] Support .skyignore negation (!) patterns by @Hawon-Oh in #9833
- [Core] Install the logging agent without sudo or apt, and keep the launch when it fails by @aylei in #11284
- [Kubernetes] Fail a launch whose pod spec the API server rejects as invalid by @cg505 in #11283
- [Dashboard] Keep unreachable and loading contexts from slowing the infra table by @rohansonecha in #11287
- [Dashboard] Show GKE TPU accelerators by generation by @rohansonecha in #11288
- [Jobs] Treat pool checkout timeouts as transient bookkeeping errors by @amitgelber in #11039
- [Core] Isolate PostgreSQL advisory locks from the state pool by @amitgelber in #11030
- [API Server] Let plugins run selected requests in the API server process by @aylei in #11291
- [Kubernetes] Support KAI Scheduler by @ryxli in #11053
- [Core] Remove the per-process advisory lock cap by @cg505 in #11307
- [Kubernetes] Set TPU_NAME on GKE TPU pods by @DanielZhangQD in #11299
- [Core] Keep node_names and other list fields in refreshed cluster records by @DanielZhangQD in #11300
- [API Server] Skip the users upsert for authenticated requests; re-read service-account tokens in the background by @aylei in #11292
- [Kubernetes] Parse the kubeconfig again only when it changes by @DanielZhangQD in #11286
- [Core] Reload the RBAC policy only when it changed by @DanielZhangQD in #11295
- [Kubernetes] Read realtime GPU availability from a node snapshot provider by @aylei in #11315
- [Kubernetes] Fail fast on contexts known to be unreachable by @aylei in #11316
- [API Server] Let plugins choose how a scheduled request executes by @aylei in #11318
- [Kubernetes] Move the Kubernetes API error classifiers into one module by @cg505 in #11304
- [Docs] Show the README logo that matches the GitHub theme by @stephenkevn in #11309
- [Kubernetes] Fix yapf formatting in kubernetes/utils.py by @cg505 in #11325
- [Jobs] Re-attach to a live launch request after a controller restart (continue the launch beside the initial launch) by @cg505 in #11280
- [API Server] Keep the shared log handler when /validate applies the client's env by @cg505 in #11327
- [Core] Reload the server config only when its source changed by @DanielZhangQD in #11297
- [Core] Write multi-row RBAC policy changes in one transaction by @DanielZhangQD in #11317
- [Core] Check workspace membership without the 1 h cache by @DanielZhangQD in #11319
- [Test] Stop test_backend_utils from leaking SKYPILOT_CONFIG by @DanielZhangQD in #11334
- [Server] Hide SSH key paths from non-admin SSH Node Pool reads by @DanielZhangQD in #11333
- [Kubernetes] Keep the requested chip count for GKE TPU label names by @rohansonecha in #11331
- [Core] Size parallel threads by the container's CPU limit by @samuelv8 in #11301
- [Kubernetes] Do not treat GKE TPU requests as GCP TPU VMs by @rohansonecha in #11338
- [API Server] Release executor memory only after RSS grows by a threshold by @aylei in #11335
- [Test] Fix flaky test_follow_tail_exits_when_stdin_closes by @zpoint in #11294
- [Jobs] Find the cluster without a workspace filter before downloading logs by @lloyd-brown in #11339
- [Kubernetes] Multi-host TPU slices on GKE by @rohansonecha in #11337
- [Docs] Add fault-tolerant PyTorch training example with torchft by @kevinmingtarja in #11344
- [Jobs] Check workspace read access for managed job logs by @DanielZhangQD in #11345
- [Kubernetes] Let sky check re-check only some contexts by @aylei in #11354
- [API Server] Build /api/status listings off the event loop by @aylei in #11348
- [Kubernetes] Return an empty list from realtime GPU availability when no GPUs are found by @aylei in #11358
- [API] Keep worker threads warm instead of starting a thread per task by @aylei in #11351
- Release 0.14.3 by @lloyd-brown in #11399
New Contributors
- @advpropsys made their first contribution in #10835
- @amitgelber made their first contribution in #10856
- @exdsgift made their first contribution in #10763
- @MohammadHijjawi97 made their first contribution in #10926
- @ciprianf-flexion made their first contribution in #10846
- @nordmtr made their first contribution in #11016
- @leshanbog made their first contribution in #10960
- @eugeneteoh made their first contribution in #11054
- @lgqfhwy made their first contribution in #11256
- @bufocoder[bot] made their first contribution in #11260
- @Hawon-Oh made their first contribution in #9833
Full Changelog: v0.14.2...v0.14.3