github huggingface/transformers v4.42.3
Patch release v4.42.3
on GitHub

2 days ago

Make sure we have attention softcapping for "eager" GEMMA2 model

After experimenting, we noticed that for the 27b model mostly, softcapping is a must. So adding it back (it should have been there, but an error on my side made it disappear) sorry all! 😭

  • Gemma capping is a must for big models (#31698)
Check out latest releases or
releases around huggingface/transformers v4.42.3

Don't miss a new transformers release

NewReleases is sending notifications on new releases.

Get notifications