We still had some bugs 🐛 in 0.19.1 some bugs, notably:
SDXL (Kohya-style) LoRA
The official SD-XL 1.0 LoRA (Kohya-styled) is now supported thanks to #4287. You can try it as follows:
from diffusers import DiffusionPipeline
import torch
pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16)
pipe.load_lora_weights("stabilityai/stable-diffusion-xl-base-1.0", weight_name="sd_xl_offset_example-lora_1.0.safetensors")
pipe.to(torch_dtype=torch.float16)
pipe.to("cuda")
prompt = "beautiful scenery nature glass bottle landscape, purple galaxy bottle"
negative_prompt = "text, watermark"
image = pipe(prompt, negative_prompt=negative_prompt, num_inference_steps=25).images[0]
In addition, a couple more SDXL LoRAs are now supported:
(SDXL 0.9:)
- https://civitai.com/models/22279?modelVersionId=118556
- https://civitai.com/models/104515/sdxlor30costumesrevue-starlight-saijoclaudine-lora
- https://civitai.com/models/108448/daiton-sdxl-test
- https://filebin.net/2ntfqqnapiu9q3zx/pixelbuildings128-v1.safetensors
To know more details and the known limitations, please check out the documentation.
Thanks to @isidentical for their sincere help in the PR.
Batched inference
@bghira found that for SDXL Img2Img batched inference led to weird artifacts. That is fixed in: #4327.
Downloads
Under some circumstances SD-XL 1.0 can download ONNX weights which is corrected in #4338.
Improved SDXL behavior
#4346 allows the user to disable the watermarker under certain circumstances to improve the usability of SDXL.
All commits:
- [SDXL Refiner] Fix refiner forward pass for batched input by @patrickvonplaten in #4327
- [ONNX] Don't download ONNX model by default by @patrickvonplaten in #4338
- [SDXL] Make watermarker optional under certain circumstances to improve usability of SDXL 1.0 by @patrickvonplaten in #4346
- [Feat] Support SDXL Kohya-style LoRA by @sayakpaul in #4287