This release comes with many changes for the R models:
- ReLU activations replaced with SiLU activations. This change is mostly inconsequential but the SiLU seems to generalise a little better.
- Small dataset adjustments, has more data now and a higher anime to fanart ratio. The new chroma model was trained purely on fanart.
- Higher initial learning rate, which was manually halved a few times.
- Training was done mostly in mixed precision, with only a few epochs done with full fp32 operations before exporting.
These new models were retrained from scratch and they show better reconstruction quality, on top of also being more stable on noisy content.
R8F64_DS has been deprecated. While the DS variants make sense when doing real-time inference of blurry/noisy web content, this doesn't translate that well into the use-cases the R models are generally aimed at.
There are no changes to the C models. I've tried retraining them from scratch with the changes mentioned above, but the results weren't positive.