github unslothai/unsloth 2025-03
Gemma 3
on GitHub

16 hours ago

March Release 🦥

Get the latest stable Unsloth via: 

pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

The March release should be stable - you can force the version via: 

pip install "unsloth==2025.3.14" "unsloth_zoo==2025.3.12"

New Features

  • Read all details here: https://unsloth.ai/blog/gemma3

  • Gemma 3 1B, 4B, 12B and 27B finetuning all work now! Colab Notebook We fixed some issues which caused Gemma 3 training loss to be very high. This includes some tokenization issues so fine-tuning Gemma 3 will now work correctly if you use Unsloth.
    image

  • We also encountered many infinite gradients during Gemma 3 (1B to 27B) finetuning. We found float16 mixed precision (Tesla T4, RTX 2080 series) to not function well, and we defaulted to float32 precision. Float16 also failed on A100, so this is a hardware agnostic issue. Bfloat16 is fine though! Unsloth auto selects the best data-type! You do not have to do anything! Colab Notebook to finetune Gemma 3

  • Preliminary support for full-finetuning and 8bit finetuning - set full_finetuning = True or load_in_8bit = True Both will be optimized further in the future! A reminder you will need more powerful GPUs! 

model, tokenizer = FastModel.from_pretrained(
    model_name = "unsloth/gemma-3-4B-it",
    max_seq_length = 2048, # Choose any for long context!
    load_in_4bit = True,  # 4 bit quantization to reduce memory
    load_in_8bit = False, # [NEW!] A bit more accurate, uses 2x memory
    full_finetuning = False, # [NEW!] We have full finetuning now!
    # token = "hf_...", # use one if using gated models
)
  • New Unsloth Auto Model support - nearly all models are now supported! We now supports vision and text models out of the box, without the need for custom implementations (and all are optimized!)
  • Mixtral (yes finally!), Gemma 3, Granite 3.2, Cohere, OLMo, Reka, and generally any vision or language model! There might be some occasional models which don't work! 
model, tokenizer = FastModel.from_pretrained(
    model_name = "mistralai/Mixtral-8x7B-Instruct-v0.1",
)
pip install unsloth
  • Train on completions / responses only for vision models supported! Use it like below: 
data_collator = UnslothVisionDataCollator(
    model,
    tokenizer,
    train_on_responses_only = False,
    instruction_part = "<|start_header_id|>user<|end_header_id|>\n\n",
    response_part = "<|start_header_id|>assistant<|end_header_id|>\n\n",
)
SFTTrainer(..., data_collator = data_collator)
  • Conversions to llama.cpp GGUFs for 16bit and 8bit now DO NOT need compiling! This solves many many issues, and this means no need to install GCC, Microsoft Visual Studio etc! 
model.save_pretrained_merged("gemma-3-finetune", tokenizer)
model.save_pretrained_gguf(
    "gemma-3-finetune",
    quantization_type = "Q8_0", # For now only Q8_0, BF16, F16 supported
)

  • Vision models now auto resize images which stops OOMs and also allows truncating sequence lengths!

  • Many multiple optimizations in Unsloth allowing a further +10% less VRAM usage, and >10% speedup boost for 4bit (on top of our original 2x faster, 70% less memory usage). 8bit and full finetuning also benefit!

  • GRPO in Unsloth now allows non Unsloth uploaded models to be in 4bit as well - reduces VRAM usage a lot! (ie pretend your own finetune of Llama)

  • New training logs and infos - training parameter counts, total batch size
    image

  • Vision models now also work for normal text training! This means non vision notebooks can work with vision models!

  • Complete gradient accumulation bug fix coverage for all models!

  • GRPO notebook for Gemma 3 coming soon with Hugging Face's reasoning course!

  • DoRA, Dropout, and other PEFT methods should just work!

Bug fixes

  • Faster and less error prone streamlined finetuning experience! Apologies for the recent issues with constant releases and breaking breaks - the March release should be stable! Ie pip install "unsloth==2025.3.14" "unsloth_zoo==2025.3.12"
  • Pixtral and Llava finetuning are now fixed! In fact nearly all vision models are supported out of the box! Please update transformers for Pixtral: pip install --no-deps git+https://github.com/huggingface/transformers.git
  • Fixed all Colabs not working - cloud instances like Runpod should just work now!
  • Fixed many many bugs - will reply to each issue with updates!

Other items

New Contributors

Full Changelog: 2025-02...2025-03

Check out latest releases or
releases around unslothai/unsloth 2025-03

Don't miss a new unsloth release

NewReleases is sending notifications on new releases.

Get notifications