March Release 🦥
Get the latest stable Unsloth via:
pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo
The March release should be stable - you can force the version via:
pip install "unsloth==2025.3.14" "unsloth_zoo==2025.3.12"
New Features
-
Read all details here: https://unsloth.ai/blog/gemma3
-
Gemma 3 1B, 4B, 12B and 27B finetuning all work now! Colab Notebook We fixed some issues which caused Gemma 3 training loss to be very high. This includes some tokenization issues so fine-tuning Gemma 3 will now work correctly if you use Unsloth.
-
We also encountered many infinite gradients during Gemma 3 (1B to 27B) finetuning. We found float16 mixed precision (Tesla T4, RTX 2080 series) to not function well, and we defaulted to float32 precision. Float16 also failed on A100, so this is a hardware agnostic issue. Bfloat16 is fine though! Unsloth auto selects the best data-type! You do not have to do anything! Colab Notebook to finetune Gemma 3
-
Preliminary support for full-finetuning and 8bit finetuning - set
full_finetuning = True
orload_in_8bit = True
Both will be optimized further in the future! A reminder you will need more powerful GPUs!
model, tokenizer = FastModel.from_pretrained(
model_name = "unsloth/gemma-3-4B-it",
max_seq_length = 2048, # Choose any for long context!
load_in_4bit = True, # 4 bit quantization to reduce memory
load_in_8bit = False, # [NEW!] A bit more accurate, uses 2x memory
full_finetuning = False, # [NEW!] We have full finetuning now!
# token = "hf_...", # use one if using gated models
)
- New Unsloth Auto Model support - nearly all models are now supported! We now supports vision and text models out of the box, without the need for custom implementations (and all are optimized!)
- Mixtral (yes finally!), Gemma 3, Granite 3.2, Cohere, OLMo, Reka, and generally any vision or language model! There might be some occasional models which don't work!
model, tokenizer = FastModel.from_pretrained(
model_name = "mistralai/Mixtral-8x7B-Instruct-v0.1",
)
- Windows support via pip install unsloth should function now! Utilizes https://pypi.org/project/triton-windows/ which provides a pip installable path for Triton. Use:
pip install unsloth
- Train on completions / responses only for vision models supported! Use it like below:
data_collator = UnslothVisionDataCollator(
model,
tokenizer,
train_on_responses_only = False,
instruction_part = "<|start_header_id|>user<|end_header_id|>\n\n",
response_part = "<|start_header_id|>assistant<|end_header_id|>\n\n",
)
SFTTrainer(..., data_collator = data_collator)
- Conversions to llama.cpp GGUFs for 16bit and 8bit now DO NOT need compiling! This solves many many issues, and this means no need to install GCC, Microsoft Visual Studio etc!
model.save_pretrained_merged("gemma-3-finetune", tokenizer)
model.save_pretrained_gguf(
"gemma-3-finetune",
quantization_type = "Q8_0", # For now only Q8_0, BF16, F16 supported
)
-
Vision models now auto resize images which stops OOMs and also allows truncating sequence lengths!
-
Many multiple optimizations in Unsloth allowing a further +10% less VRAM usage, and >10% speedup boost for 4bit (on top of our original 2x faster, 70% less memory usage). 8bit and full finetuning also benefit!
-
GRPO in Unsloth now allows non Unsloth uploaded models to be in 4bit as well - reduces VRAM usage a lot! (ie pretend your own finetune of Llama)
-
New training logs and infos - training parameter counts, total batch size
-
Vision models now also work for normal text training! This means non vision notebooks can work with vision models!
-
Complete gradient accumulation bug fix coverage for all models!
-
GRPO notebook for Gemma 3 coming soon with Hugging Face's reasoning course!
-
DoRA, Dropout, and other PEFT methods should just work!
Bug fixes
- Faster and less error prone streamlined finetuning experience! Apologies for the recent issues with constant releases and breaking breaks - the March release should be stable! Ie
pip install "unsloth==2025.3.14" "unsloth_zoo==2025.3.12"
- Pixtral and Llava finetuning are now fixed! In fact nearly all vision models are supported out of the box! Please update transformers for Pixtral:
pip install --no-deps git+https://github.com/huggingface/transformers.git
- Fixed all Colabs not working - cloud instances like Runpod should just work now!
- Fixed many many bugs - will reply to each issue with updates!
Other items
- GRPO Bug fixes by @danielhanchen in #1623
- Fixes Triton url in README.md by @DiogoNeves in #1607
- Update README.md by @shimmyshimmer in #1654
- Update README.md by @shimmyshimmer in #1688
- Fix bugs by @danielhanchen in #1701
- Fix bugs by @danielhanchen in #1706
- Memory efficient GRPO, DPO etc by @danielhanchen in #1716
- Add GRPO metrics by @danielhanchen in #1718
- llama-quantize on WINDOWS WSL error fix - edit save.py (gguf saving breaks) by @everythingisc00l in #1649
- Update rl_replacements.py by @SethHWeidman in #1754
- Update README.md by @danielhanchen in #1768
- fix an import error by @NinoRisteski in #1767
- Gemma Mask convert to float by @Erland366 in #1762
- [Windows Support] Add latest
xformers
wheels to pyproject.toml by @versipellis in #1753 - Memory Efficient GRPO by @danielhanchen in #1773
- Bug Fixes by @danielhanchen in #1774
- Export Model to ollama.com by @gjyotin305 in #1648
- Fix: GRPO with Mistral and importing by @oKatanaaa in #1831
- Fix key error in GRPOTrainer by @le-big-mac in #1818
- fixed syntax warnings by @KareemMusleh in #1522
- Direct windows support for unsloth by @adityaghai07 in #1841
- Fix Layernorm when num_cols not a power of 2 by @MekkCyber in #1867
- Added Python version warning to Windows Install Section by @areebuzair in #1872
- Update README.md by @shimmyshimmer in #1885
- Bug fixes by @danielhanchen in #1891
- Many bug fixes by @danielhanchen in #1900
- Logits fixes by @danielhanchen in #1916
- Bug fixes by @danielhanchen in #1920
- Bug fixes by @danielhanchen in #1951
- move use_modelscope to _utils by @KareemMusleh in #1938
- Don't use revision when loading model_config and is_peft=True by @wiwu2390 in #1949
- More syntax warnings by @KareemMusleh in #1944
- Gemma 3 by @danielhanchen in #1986
- Gemma 3 bug fixes by @danielhanchen in #2005
- Triton windows update by @Captain-T2004 in #1976
- Update RMS LayerNorm implementation, and list compr. change in chat templates by @NinoRisteski in #1974
- Gemma 3, bug fixes by @danielhanchen in #2014
New Contributors
- @DiogoNeves made their first contribution in #1607
- @everythingisc00l made their first contribution in #1649
- @SethHWeidman made their first contribution in #1754
- @versipellis made their first contribution in #1753
- @gjyotin305 made their first contribution in #1648
- @le-big-mac made their first contribution in #1818
- @MekkCyber made their first contribution in #1867
- @areebuzair made their first contribution in #1872
- @wiwu2390 made their first contribution in #1949
- @Captain-T2004 made their first contribution in #1976
Full Changelog: 2025-02...2025-03