Feb 20, 2023
- Add 320x320
convnext_large_mlp.clip_laion2b_ft_320
andconvnext_lage_mlp.clip_laion2b_ft_soup_320
CLIP image tower weights for features & fine-tune - 0.8.13dev0 pypi release for latest changes w/ move to huggingface org
Feb 16, 2023
safetensor
checkpoint support added- Add ideas from 'Scaling Vision Transformers to 22 B. Params' (https://arxiv.org/abs/2302.05442) -- qk norm, RmsNorm, parallel block
- Add F.scaled_dot_product_attention support (PyTorch 2.0 only) to
vit_*
,vit_relpos*
,coatnet
/maxxvit
(to start) - Lion optimizer (w/ multi-tensor option) added (https://arxiv.org/abs/2302.06675)