license: apache-2.0
language:
- en
library_name: diffusers
pipeline_tag: text-to-image
tags:
- stable-diffusion
- stable-diffusion-xl
- sdxl
- SD-XL 0.9
- tutorial
- NathMath
sdxl-0.9-usage
This repo is a tutorial intended to help beginners use the new released model, stable-diffusion-xl-0.9 in ComfyUI, with both the base and refiner models together to achieve a magnificent quality of image generation.
With usable demo interfaces for ComfyUI to use the models (see below)!
After test, it is also useful on SDXL-1.0!
Usage
Here is a full tutorial to use stable-diffusion-xl-0.9 FROM ZERO!
- Go to Github and find the latest version of ComfyUI, which is a cutting-edge GUI interface for you to flexibly use all kinds of diffusion models.
- Note that in the link below, it provides both a standalone zip (includes a prepared python environment), and a source version.
- We highly recommend the first approach because we are all lazy buddies.
- Link: https://github.com/comfyanonymous/ComfyUI
- Extract all files into your disk if you have chosen the standalone version of this ComfyUI, or install all the dependences to create a essential python environment.
- Note that files extracted may not necessarilty be like the image shown below.
- Get into the folder or the environment you have just created when you have finished the last step.
- Next, what you can find includes those two ".bat" files named "run_cpu.bat" and "run_nvidia_gpu.bat" respectively.
- Then, choose the suitable option and click it, which means, to use CUDA (if you have NVDIA GPUs) or not, and double click to run the script.
- Note that with the application of xformers, you do not need to load all of your models simultaneously, but those gargantuan weights are going to be temporarily stored in RAM and move back and force when interfereing.
- Download the latest version of stable-diffusion-xl-base-0.9 and stable-diffusion-xl-refiner-0.9 from the official repo on huggingaface.
- It is appreciated that your availability of the model has become unprecedentedly effortless now, so hurry up and get on.
- Note that for generation use, you do not need to download all model components, but only the two "safetensors" from the repo, which are respectively "sd_xl_base_0.9.safetensors" and "sd_xl_refiner_0.9.safetensors".
- Link: https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9
- Link: https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-0.9
- Move your "base" and "refiner" models into the ComfyUI's subfolder, like "...\ComfyUI\models\checkpoints", where "..." means your root folder of your ComfyUI environment.
- Click the file folder of THIS REPO and find some files named "sdxl-0.9-...-.json", which is a demo environment I have created for testing and using the stable-diffusion-xl-0.9 in ComfyUI.
- Note that the 3-digit number in the file name represents the version of my demo, and always, you should download the latest one owning the biggest number.
- Link: https://huggingface.co/NathMath/sdxl-0.9-usage/tree/main
When you have loaded into the ComfyUI, it will display this default interface shown below:
Then, find out the "load" button on a floating panel, click, and choose the ".json" file download from THIS REPO, it will quickly load a friendly interface to use the stable-diffusion-xl-0.9 series.
Things will become easy soon.
As labels of all components are shown explicitly in interface, you can find enterpoints and panels by their names.
Then, load "base" and "refiner" models, set relevant parameters, and type your favorite prompts to start your exciting journey.
Note that it will automatically save TWO images of EACH queue, respectively before applying with the refiner, and after.
- Congratulations, you have sucessfully learned how to use sdxl with FULL MODELS.
Tips
Positive prompts and Negative prompts in both input box are ought to be identical, but you can freely try whatever you want.
The output quality of the stable-diffusion-xl-0.9 models highly depends on your prompts and parameter settings, so try to adjust them to achieve a better quality. Note that these pretrained models (have not been fine tuned yet) need to be fed with enough details about your PAINTING STYLE, so try to provide sufficient information to ensure the output quality.
Although ComfyUI's model moving technique (move models to VRAM when using and move back to RAM after) can significantly reduce the required VRAM use, please note, possibly, if your models are combined with FP16 and FP32, there could be a round-off error when converting data between these two formats. The main reason why it happens may mainly due to the internal codes, and because of this, you'd better RESTART the interface to reload models from your disk every 200 times, for example, to maintain precise weights and eusure best image quality.
Different from previous diffusion models, this stable-diffusion-xl-0.9 is naturally designed for high resolution image generation, so in order to achieve a decent quality, please use a relatively higher resolution, for recommendation, 1024 * 1024.
This is not the final version of this repo. I will frequently update demos for further applications and advanced functions, so if you are interested, please like it.
Need to be updated...
Q&A
- Please feel free to leave any questions in the community of this repo, and I will soon answer them if I am able to solve.