v0.4.0: peft integration

Apply RLHF and fine-tune your favorite large model on consumer GPU using peft and trl ! Share also easily your trained RLHF adapters on the Hub with few lines of code

With this integration you can train gpt-neo-x (20B parameter model - 40GB in bfloat16) on a 24GB consumer GPU!

What's Changed

• Allow running evaluate-toxicity with cpu by @jordimas in #195
• [core] Fix quality issue by @younesbelkada in #197
• Add 1.12.1 torch compatibility in sum method by @PanchenkoYehor in #190
• peft integration by @edbeeching in #163
• [core] Update dependency by @younesbelkada in #206

New Contributors

• @PanchenkoYehor made their first contribution in #190

Full Changelog: v0.3.1...v0.4.0