Stable Diffusion Beginners Guide
In the ever-evolving landscape of artificial intelligence, the world of image generation has reached new heights with the advent of Stable Diffusion. In this guide, we will embark on a journey to unravel the power of Stable Diffusion, a tool that allows you to transform your ideas into captivating digital art.
While Stable Diffusion requires a bit more technical know-how to use, it has more options for image customization than other AI image generators.
Join us as we explore the fundamentals and techniques to harness the potential of this AI image generator, empowering you to bring your artistic visions to life with unparalleled precision and style.
What is Stable Diffusion?
Stable Diffusion AI is a latent diffusion model for generating AI images. These images can vary in type depending on the model/checkpoint used to create them.
How to Use Stable Diffusion
There are two primary ways to use Stable Diffusion, run it locally on your computer/on Google Colab or use an online service. This guide will mostly cover using it locally/on Google Colab as that offers the most customization and many online services are paid, whereas using it on your computer is free!
Note: While it is free to run locally, there are some hardware requirements if you want to run it efficiently. If you don’t have a GPU or limited processing power, then maybe a paid online service may be better.
Download Locally/Google Colab
So how do you run Stable Diffusion on your computer? The simplest way is to download and run the AUTOMATIC1111 GUI. Automatic1111 offers a ton of options on their GUI and has a large community of support for any issues you may encounter.
Alternatively, you can run it on Google Colab. While Google Colab is free to use, you may find yourself wanting more resources, in which case they have a Pro package to purchase.
Use the tutorial below for setting up on Google Colab!
Online Generators
Thankfully there are multiple online services that offer Stable Diffusion as an image generator! Depending on which online service you use, the capabilities and customization will change.
The following are a few free and paid options:
Is Stable Diffusion Free?
YES! Stable Diffusion is open source and can be downloaded and used by anyone! Some services offered online may be paid for as they provide the resources and platforms to use Stable Diffusion. But, if you want an all-free version then you can run it locally or in Google Colab.
What Can Stable Diffusion Do?
Stable Diffusion can do much more than simple image generation from text and has multiple customization options.
Generate Images From Text
To start, Stable Diffusion can generate images from a text prompt. Generally, there is a text prompt and a negative prompt for things you don’t want to include in the image.
Generate Images From Other Images
You can also generate images from other images. This is a handy way of editing images to look different, whether that be turning them into a different art style or using them as a reference for something else.
Create Videos
There are a few ways to generate videos using Stable Diffusion. Creating videos with AI image generators is still a difficult task and is being enhanced every day.
To start, you can use a Stable Diffusion extension like Deforum which can create a video from a text prompt.
The second option is to edit an existing video with Stable Diffusion. This is a common technique for stylizing videos.
Prompt Creation For Stable Diffusion
Creating a solid prompt for Stable Diffusion is a bit beyond the scope of this beginner’s guide, but here are a few things to get you started.
Using descriptive words to focus on the most important details of what you want is incredibly important. Specify the number of characters, style of the image, colors, lighting, and any small details that may be important to what you’re trying to create.
The best tip I can offer is to search sites like PromptHero and other AI image aggregators to see what prompts are working for others.
Parameters And How To Use Them
There are multiple parameters that Stable Diffusion uses to generate images. Below are a few suggestions on how to use them.
Sampling Method – Stable Diffusion comes with numerous sampling methods. While Euler A and DPM++ 2M Karras work great for many of my images, others may be better for the ones you make. It’s best to start with the defaults and commonly used ones but feel free to branch out and try other methods.
Sampling Steps – How many times the model samples. 20 – 30 is the common amount to use. Increase if the image is too blurry.
Image Size – The size of the image plays a big role in how long it takes to generate the image. The bigger the image, the longer it takes. Remember that images made at smaller scales can be upscaled later. I usually start at 512 x 512 to test prompts and upscale later.
CFG Value – This determines how closely Stable Diffusion follows your prompt. The typical setting is around 7 as you want the model to have some creativity.
Seed Value – The initial seed for the image. -1 generates a random seed. If you want to experiment with prompts then use the same seed (any number).
Finding Models/Checkpoints For Stable Diffusion
Stable Diffusion uses checkpoints, which are models to add weight toward generating images of a certain genre. There are tons of different checkpoints to choose from and you can even make your own!
I’d recommend searching through HuggingFace and CivitAI to find checkpoints!
What Are LORAs And How To Use Them
LORA stands for Low-Rank Adaptation. They’re used to fine-tune Stable Diffusion models toward specific genres, characters, or any small details.
They can be accessed in the GUI above the prompt parameter settings.