hiyouga/LLaMA-Factory v0.1.7

v0.1.7: Script Preview and RoPE Scaling

on GitHub

New features

• Preview training script in Web UI by @codemayq in #479 #511
• Support resuming from checkpoints by @niuba in #434 (transformers>=4.31.0 required)
• Two RoPE scaling methods: linear and NTK-aware scaling for LLaMA models (transformers>=4.31.0 required)
• Support training the ChatGLM2-6B model
• Support PPO training in bfloat16 data type #551

Bug fix

• Unusual output of quantized models #278 #391
• Runtime error in distributed DPO training #480
• Unexpected truncation in generation #532
• Dataset streaming error in pre-training #548 #549
• Tensor shape mismatch in PPO training using ChatGLM2 #527 #528
• #475 #476 #478 #481 #494 #551