huggingface/huggingface_hub v1.31.0

[v1.31.0] Custom labels for Sandboxes, More resilient downloads and more

on GitHub

🏷️ Custom labels for Sandboxes

Dedicated sandboxes now accept custom labels, attached to the underlying Job. This is useful for cost attribution, bookkeeping, or finding and reconnecting to the sandboxes created by a given controller run. Labels use the same -l / --label KEY=VALUE syntax as hf jobs run. They are merged with the labels the SDK uses internally, and invalid or reserved labels are rejected before a billable Job is started. Pool-based sandboxes are unchanged: custom labels are only accepted for dedicated sandboxes.

>>> from huggingface_hub import Sandbox
>>> sandbox = Sandbox.create(image="python:3.12", labels={"controller-run": "run-42"})

hf sandbox create --label controller-run=run-42 --label team=data-infra

• [Sandbox] Support custom Job labels by @Wauplin in #4824

📥 More resilient downloads

A batch of fixes makes downloads more robust to unusual server responses, network hiccups and concurrent usage:

• A timeout while waiting for the response headers of a streamed download is now retried and resumed like a body read failure, instead of escaping http_get() while retries remain.

• Regular HTTP downloads no longer fail when the HEAD response has no Content-Length. The file size is validated against the GET response when available.

• snapshot_download now writes the refs/ cache file atomically, reusing the logic already used by hf_hub_download. This fixes a long-standing race when many concurrent snapshot_download calls target the same repo (seen in vLLM / llm-compressor).

• hf download --dry-run --local-dir ... no longer copies files from the Hub cache into the destination. On large files and slow disks this looked like a hang and could leave an incomplete file behind.

• [Download] Share retry handling for stream entry and body failures by @Wauplin in #4826

• [Download] Tolerate missing HEAD Content-Length by @Wauplin in #4805

• [Download] Write cache ref file atomically in snapshot_download by @Wauplin in #4829

• [Download] Prevent cached file copies during local-dir dry runs by @wakamex in #4817

🔌 httpx re-exported for library integrators

huggingface_hub now re-exports the HTTP library it uses as huggingface_hub.utils.httpx. Libraries built on top of huggingface_hub that need httpx types or exceptions (typically to catch errors) should import it from there rather than importing httpx directly. v1.x is built on httpx, and v2.x will move to its successor httpx2, so importing through huggingface_hub.utils keeps your code compatible with both. This is only for types and exceptions: to make requests to the Hub, keep using get_session().

from huggingface_hub.utils import httpx

try:
    ...
except httpx.HTTPError:
    ...

📚 Documentation: Utilities — The httpx module

• [httpx migration] Expose httpx as a huggingface_hub submodule by @Wauplin in #4803

🔒 Security

HfFileSystem.get() now validates remote filenames before writing anything locally. A server-side filename such as folder/..\..\outside.txt could previously escape the destination directory on Windows during a recursive download. The same check already protected hf_hub_download, snapshot_download and bucket sync. Unsafe filenames now raise ValueError on all platforms, including when downloading to an explicitly named file or a file object.

• [HfFileSystem] Reject unsafe paths before local downloads by @Wauplin in #4828

🖥️ CLI

• [CLI] Fix hf buckets rm --recursive deleting lexical siblings of the prefix: removing logs/ no longer touches logs_root.txt by @Wauplin in #4804
• [CLI] Don't let hf update self-upgrade a pip install on Windows: it now prints the pip install -U command to run instead of leaving a broken install by @Wauplin in #4823
• [CLI] Raise explicit error for shell-script extensions on Windows by @hanouticelina in #4846
• [CLI] Fix truncated command descriptions in the CLI reference by @hanouticelina in #4849

🐛 Bug and typo fixes

• [hf_api] Make CommitInfo copy-able and pickle-able by @Wauplin in #4822
• [LFS] Fix SliceFileObj.__iter__ yielding only the first 4MB chunk by @BenYang12 in #4844
• Fix dotenv parser truncating unquoted values containing # (used by hf jobs --env-file) by @BenYang12 in #4842
• Keep single-quoted values verbatim in load_dotenv by @AnishPatel526 in #4827

📖 Documentation

• [Docs] Clarify Jobs argument forwarding with -- and align examples by @davanstrien in #4809

🏗️ Internal

• Bump the actions group across 1 directory with 5 updates by @dependabot[bot] in #4798
• Post-release: bump version to 1.31.0.dev0 by @huggingface-hub-bot[bot] in #4806