huggingface/huggingface_hub v0.12.0 on GitHub

Spaces on steroids 🚀

Spaces support has been substantially enhanced. You can now:

Request hardware for your Space !
Configure your Space secrets
Get runtime information about your Space

# Assign hardware when creating the Space
api.create_repo(repo_id=repo_id, repo_type="space", space_sdk="gradio", space_hardware="cpu-upgrade")

# Configure some secrets
api.add_space_secret(repo_id=repo_id, key="HF_TOKEN", value="hf_api_***")

# Request hardware on the fly
api.request_space_hardware(repo_id=repo_id, hardware="t4-medium")

# Get Space runtime (state, hardware, sdk,...)
api.get_space_runtime(repo_id=repo_id)

Visit the docs for more details.

Manage space hardware and secrets programmatically by @Wauplin in #1243
Mock Space tests requiring GPUs by @Wauplin in #1263
Remove get_space_secrets endpoint by @Wauplin in #1264

And in bonus: Spaces now support Dockerfile natively !

api.create_repo(repo_id=repo_id, repo_type="space", space_sdk="docker")

Check out the docs for more details.

New features and improvements for `HfApi`

List all branches and tags from a repo

It's now possible to list branches/tags from a repo, getting exact ref and target_commit.
More details in the docs.

>>> api.list_repo_refs("bigcode/the-stack", repo_type='dataset')
GitRefs(
    branches=[
        GitRefInfo(name='main', ref='refs/heads/main', target_commit='18edc1591d9ce72aa82f56c4431b3c969b210ae3'),
        GitRefInfo(name='v1.1.a1', ref='refs/heads/v1.1.a1', target_commit='f9826b862d1567f3822d3d25649b0d6d22ace714')],
    converts=[],
    tags=[
        GitRefInfo(name='v1.0', ref='refs/tags/v1.0', target_commit='c37a8cd1e382064d8aced5e05543c5f7753834da')
    ]
)

List branches and tags from a repo by @Wauplin in #1276

Manage your "favorite repos" list

New endpoints to like a repo, unlike it and list_liked_repos.

Add method to list liked repos by @Wauplin in #1254
Fix list_liked_repos (only public likes are returned) by @Wauplin in #1273
Fix like test by @Wauplin (direct commit on main)

Get the "real" `repo_id` when creating a repo

When using create_repo, one can provide a simple repo_name without specifying a namespace (example: "my-cool-model" instead of "Wauplin/my-cool-model"). This was annoying for as one could easily know if a namespace has been added to the provided repo_id. To facilitate this, the return value of create_repo is now an instance of RepoUrl which contains information like the endpoint, namespace, repo_id and repo_type.

Extend create_repo return value by @Wauplin in #1268

Create branch from any revision

By default, new branches start from main HEAD. It's now possible to specify any branch, tag or commit id to start from.

Create branch from given revision by @Wauplin in #1293

Modelcards

Modelcards module is getting some adjustments to better integrate with the Hub. The scope of this work is larger than "just" huggingface_hub and resulted in the launch of the HF Model Card Guidebook to help operationalize model cards in the ML community.

Clarify what "More Resources" is by breaking it into its parts. by @meg-huggingface in #1238
Adding "summary" variables by @meg-huggingface in #1248
datasetcard_template: I think linking to a GH user does not make sense anymore now that dataset repos are fully on the Hub by @julien-c in #1257
Push empty model card and dataset card to the Hub by @Wauplin in #1261
[Dataset | Model card] When pushing to template repos, work on actual raw contents by @julien-c in #1282
Update REGEX_YAML_BLOCK by @julien-c in #1285
Make model use more visible by @osanseviero in #1286

Documentation updates

Quite some effort has been put into the documentation in the past few weeks:

a new landing page
a more detailed installation page
a new page explaining the difference between HTTP vs GIT approaches
revisited "Searching the Hub" page
small updates in the existing docs to add some examples and consistency

Revisit documentation part 1: divio-like landing page, installation page, configuration page by @Wauplin in #1233
Updates in docs by @Wauplin in #1235
Doc page: git vs http by @Wauplin in #1245
Minor suggestions post #1245 by @osanseviero in #1249
fix broken urls in docs by @Wauplin in #1253
[Minor doc tweak] *currently... by @julien-c in #1256
Update Repository docstring by @Wauplin in #1297
add link to create_repo by @Wauplin in #1299
Document .no_exist folder by @Wauplin in #1308
Revisit "search the hub" page (+HfApi reference page) by @Wauplin in #1303

Test suite for dependent libraries

huggingface_hub is getting more and more mature but you might still have some friction if you are maintainer of a library depending on huggingface_hub. To help detect breaking changes that would affect third-party libraries, we built a framework to run simple end-to-end tests in our CI. This is still preliminary work but the hope is make hfh ecosystem more and more robust over time. Check out our README for more details.

[RFC] Contrib test suite + tests for timm and sentence_transformers by @Wauplin in #1200

Faster download

Goal is to download files faster. First step has been to increase the chunk size by which the files are uploaded. Second step has been to add an optional Rust extension. This is not officially documented for now as we are internally assessing its benefits and limits. Installing and activating hf_transfer is purely optional.

Enabling hf_transfer use. by @Narsil in #1272
Increase the chunk size for faster download by @Narsil in #1267

Breaking changes

Repository "clone_from" feature do not create the remote repository if it doesn't exist on the Hub. Please use create_repo first before cloning it locally. The private init argument has also been removed as it was not needed anymore.
allow_regex and ignore_regex have been removed from snapshot_download in favor allow_patterns and ignore_patterns.
git-based implementation of push_to_hub_fastai has been removed in favor of the HTTP-based approach. Same for ModelHubMixin, PyTorchModelHubMixin, push_to_hub_keras and KerasModelHubMixin.
create_repo is now forced to use keyword-arguments. Same for metadata_eval_result.

Remove deprecated code in v0.12 by @Wauplin in #1196

QOL improvements

Not really some features, not really some fixes. But definitely a quality of life improvement for users 🙂

Progress bar when uploading LFS files by @Wauplin in #1266
tell user how to paste token on Windows by @fristhon in #1289
fixes 1290 - check for existing token during interpreter login by @lonnen in #1291
Store token in configurable HF_HOME (in HfFolder) by @Wauplin in #1236
Compute sha by chunks when uploading by @Wauplin in #1296

Internal

CI runs on Windows !

Add windows machine for CI by @Wauplin in #1112

Misc

Remove obsolete comment by @severo in #1265
fix: authorization header should never forwarded by @rtrompier in #1269
refacto Offline repository tests by @Wauplin (direct commit on main)
Revert "refacto Offline repository tests" by @Wauplin (direct commit on main)
1278 More robust temporary directory by @Wauplin in #1279
Refacto Repository tests by @Wauplin in #1277
skip flake8 B028 by @Wauplin (direct commit on main)

Bugfixes & small improvements

Fix metadata_update for verified evaluations by @lewtun in #1214
Accept pathlib.Path in upload_file by @Wauplin in #1247
Skip failing tests related to DatasetFilter by @Wauplin in #1251
fix tasks ids test by @Wauplin (direct commit on main)
Fix InferenceAPI on image task by @Wauplin in #1270
Update "move_repo" doc url + fix Bigcode test in CI by @Wauplin in #1292
Fix test_comment_discussion by @Wauplin in #1295
Handle hf:// urls + raise ValueError if repo type is unknown by @Wauplin in #1298
Make ModelSearchArguments and DatasetSearchArguments more robust by @Wauplin in #1300
Make tests more robust (no pre-existing repo on staging Hub) by @Wauplin in #1302
fix: use cache for snapshots even if refs does not exist by @brycedrennan @Wauplin in #1306
Fix caching error when offline and no refs dir by @Wauplin in #1307

huggingface/huggingface_hub v0.12.0 v0.12.0: Spaces on steroids 🚀, extended HfApi, documentation, modelcards and more on GitHub