⚡ New provider: Featherless.AI
Featherless AI is a serverless AI inference provider with unique model loading and GPU orchestration abilities that makes an exceptionally large catalog of models available for users. Providers often offer either a low cost of access to a limited set of models, or an unlimited range of models with users managing servers and the associated costs of operation. Featherless provides the best of both worlds offering unmatched model range and variety but with serverless pricing. Find the full list of supported models on the models page.
from huggingface_hub import InferenceClient
client = InferenceClient(provider="featherless-ai")
completion = client.chat.completions.create(
model="deepseek-ai/DeepSeek-R1-0528",
messages=[
{
"role": "user",
"content": "What is the capital of France?"
}
],
)
print(completion.choices[0].message)⚡ New provider: Groq
At the heart of Groq's technology is the Language Processing Unit (LPU™), a new type of end-to-end processing unit system that provides the fastest inference for computationally intensive applications with a sequential component, such as Large Language Models (LLMs). LPUs are designed to overcome the limitations of GPUs for inference, offering significantly lower latency and higher throughput. This makes them ideal for real-time AI applications.
Groq offers fast AI inference for openly-available models. They provide an API that allows developers to easily integrate these models into their applications. It offers an on-demand, pay-as-you-go model for accessing a wide range of openly-available LLMs.
from huggingface_hub import InferenceClient
client = InferenceClient(provider="groq")
completion = client.chat.completions.create(
model="meta-llama/Llama-4-Scout-17B-16E-Instruct",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Describe this image in one sentence."},
{
"type": "image_url",
"image_url": {"url": "https://vagabundler.com/wp-content/uploads/2019/06/P3160166-Copy.jpg"},
},
],
}
],
)
print(completion.choices[0].message)🤖 MCP and Tiny-agents
It is now possible to run tiny-agents using a local server e.g. llama.cpp. 100% local agents are right behind the corner!
- [MCP] Add local/remote endpoint inference support by @hanouticelina in #3121
Fixing some DX issues in the tiny-agents CLI.
- Fix
tiny-agentscli exit issues by @Wauplin in #3125 - [MCP] reinject JSON parse & runtime tool errors back into the chat history by @hanouticelina in #3137
📚 Documentation
New translation from the Hindi-speaking community, for the community!
- Added Hindi translation for git_vs_http.md in concepts section by @february-king in #3156
🛠️ Small fixes and maintenance
😌 QoL improvements
- Make hf-xet more silent by @Wauplin in #3124
- [HfApi] Collections in collections by @hanouticelina in #3120
- Fix inference search by @Wauplin in #3022
- [Inference Providers] Raise warning if provider's status is in error mode by @hanouticelina in #3141
🐛 Bug and typo fixes
- Fix snapshot_download on very large repo (>50k files) by @Wauplin in #3122
- fix tqdm_class argument of subclass of tqdm by @andyxning in #3111
- fix quality by @hanouticelina in #3128
- second example in oauth documentation by @thanosKivertzikidis in #3136
- fix table question answering by @hanouticelina in #3154
🏗️ internal
- Create claude.yml by @julien-c in #3118
- [Internal] prepare for 0.33.0 release by @hanouticelina in #3138
Significant community contributions
The following contributors have made significant changes to the library over the last release:
- @pohnean
- ✨ Support for Featherless.ai as inference provider (#3081)
- @february-king
- Added Hindi translation for git_vs_http.md in concepts section (#3156)