hiyouga/LlamaFactory v0.2.0

v0.2.0: Web UI Refactor, LongLoRA

on GitHub

New features

• Support LongLoRA for the LLaMA models
• Support training the Qwen-14B and InternLM-20B models
• Support training state recovery for the all-in-one Web UI
• Support Ascend NPU by @statelesshz in #975
• Integrate MMLU, C-Eval and CMMLU benchmarks

Modifications

• Rename repository to LLaMA Factory (former LLaMA Efficient Tuning)
• Use the cutoff_len argument instead of max_source_length and max_target_length #944
• Add a train_on_prompt option #1184

Bug fix

• Fix numeric error caused by the layer norm dtype in 84b7486 [1]
• Fix bugs in PPO Trainer by @mmbwf in #900
• Fix #424 #762 #814 #887 #913 #1000 #1026 #1032 #1064 #1068 #1074 #1086 #1097 #1176 #1177 #1190 #1191

[1] huggingface/transformers#25598 (comment)