- Quick installation
- Provided commands
- Using the bot software
- Additional configuration
- Special notes
- Known bugs
This is the code of the first version of my Discord bot software named "DeguDiffusion", that allows you to run your own AI-powered images generation bot on Discord using a local installation of StableDiffusion.
The software is available under MIT license.
The concept behind it is simple :
- You create a bot account, and add its token to the
.env
file asDISCORD_TOKEN
. - The software will use the bot account to connect to Discord and register image
generation commands (
/degudiffusion
notably) on the servers it has been invited to. - Users on the servers use these commands to send image generation requests to your server.
- Your server generate the images, using a local installation of HuggingFaces StableDiffusion, and send back the result through Discord.
The software is meant to run on a machine equipped with an NVIDIA GPU.
This has not been tested at all with AMD or Intel GPU.
A Job Queue system is provided, allowing you to queue generations requests and treat them once previous ones finished.
Each request job generates a specific thread, unless its size is lower than a certain amount, in which cases messages are sent directly on the same channel.
This software won't receive major updates, I'm just uploading this version since it's working, and want to keep the working version somewhere before I break it and remake a more versatile one.
Setup a .env
file with at least two entries :
# Make sure your bot has sufficient rights and privileges.
# Write your Discord bot token after the '='. No quotes needed.
DISCORD_TOKEN=
# Make sure you accepted the license on
# https://huggingface.co/CompVis/stable-diffusion-v1-4
# Write your Huggingfaces token after the '='. No quotes needed.
HUGGINGFACES_TOKEN=
Then run STARTBOT.bat
(or STARTBOT.sh
on Linux).
/degudiffusion
Summons a form, where you can setup the generation
Right-click on a Bot message > Apps > Repeat Diffusion
Summons a form, reusing the settings provided in the message.
Fails if the message right-clicked contained no diffusion information.
Right-click on a generated image message > Apps > Check Degu PNG Metadata
Sends back the metadata of a generated PNG, if it still exist on the server.
Note : This doesn't try to download the PNG !
This only reads it from the server harddrive, if it's still present.Also, this uses ephemeral messages ("Only you can see this message" messages).
Sometimes these messages cannot be seen by the client, when sent inside threads, due to some obscure Discord client bug.
In this case, just try again.
This is mainly designed to run on a simple Windows PC, but it also been tested on Linux.
- Python 3 (Tested with Python 3.10)
- A NVIDIA Graphics Card (CUDA)
- A Discord bot account for a
DISCORD_TOKEN
- A HuggingFaces account for a
HUGGINGFACES_TOKEN
If you're using Docker, check https://hub.docker.com/r/vrvoyage/degudiffusion
Setup a .env
file with at least two entries :
# Make sure your bot has sufficient rights and privileges.
# Write your Discord bot token after the '='. No quotes needed.
DISCORD_TOKEN=
# Make sure you accepted the license on
# https://huggingface.co/CompVis/stable-diffusion-v1-4
# Write your Huggingfaces token after the '='. No quotes needed.
HUGGINGFACES_TOKEN=
Check
.env.sample
for more configuration directives.
Read below if you want to know how to get these tokens.
Double-click on STARTBOT.bat and wait for "StableDiffusion ready to go".
Run STARTBOT.sh (./STARTBOT.sh
) and wait for "StableDiffusion ready to go"
Use the following setup :
version: "3.9"
services:
degu:
image: vrvoyage/degudiffusion:1.1
build: .
env_file: .env
environment:
- STABLEDIFFUSION_CACHE_DIR=stablediffusion_cache
- IMAGES_OUTPUT_DIRECTORY=generated # Can be commented if SAVE_IMAGES_TO_DISK is set to false
# - SAVE_IMAGES_TO_DISK=false # If you uncomment this, you can comment the first volume
volumes:
- ./generated:/app/generated # Only required if SAVE_IMAGES_TO_DISK=false isn't set
- ./cache:/app/stablediffusion_cache # Related to STABLEDIFFUSION_CACHE_DIR
- ./config:/app/config # Can be mounted in Read-Only if a 'replacers.json' file is already present
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
The Docker-image is available here : https://hub.docker.com/r/vrvoyage/degudiffusion
The build image is available here : https://github.com/vr-voyage/degu-diffusion-docker
On Windows, make sure your docker installation supports CUDA.
On Linux, make sure you have
nvidia-container-runtime
installed and setup in yourdocker/daemon.json
configuration file (Generally in/etc
).
pip install -r requirements.txt
python degu_diffusion_v0.py
If you want to test StableDiffusion itself alone,
run TEST_STABLEDIFFUSION.bat
or TEST_STABLEDIFFUSION.sh
.
Using default settings, this should output 8 images in the output
folder (generated/
by default).
If the test fails, check the HUGGINGFACES_TOKEN
you put in the .env
file.
Also pay attention to every line output on the terminal, some of them might
provide clear explanations about what's going on.
You need to be registered on HuggingFaces.
Then go to "Access Token" from your User Profile and generate or copy your token
in the .env
file after HUGGINGFACES_TOKEN=
(No need to add quotes).
By default, you also need to accept the licence of CompVis/stable-diffusion-v1-4
A bot is just a special Discord account, who needs a TOKEN to login to Discord, instead of a login/password.
So, the DISCORD_TOKEN
is used by this software to actually login to Discord
as the Bot.
You can run a bot while being also being connected to Discord with another account, on the same machine, and the bot was actually only tested in that kind of environment.
- Go to the Discord Developer portal.
- Create a "New application", by clicking the upper right button near your Profile icon.
- Setup the name then make sure you're currently editing your new application.
- In Bot (Left panel), in "Build-A-Bot", click on "Add Bot" and Confirm.
- On Oauth2 General (Left panel), select :
- AUTHORIZATION METHOD
In-app Authorization - SCOPES
bot
application.commands
- BOT PERMISSIONS
- Read Messages / View Channels
- Send Messages
- Create Public Threads
- Send Messages in Threads
- Attach Files
- AUTHORIZATION METHOD
- On OAuth2 URL Generator (Left panel) :
- Select the same SCOPES (
bot
andapplication.commands
) and PERMISSIONS. - Copy the generated URL at the bottom.
- Select the same SCOPES (
- Enter this URL in your browser to add the generated bot to one of your server.
You can also send this link to people who'd like to invite the bot to their server.
- In Bot, again, click on 'Reset Token' and copy it after
DISCORD_TOKEN=
in the.env
file (No need to use quotes).
If the permissions were wrong :
Set the permissions again on both panels
Open the new URL in your browser and invite the Bot again on the same server.
If you don't know it, click on "Reset Token" in the "Bot" section of your application. You can view your application settings on the Discord Developer Portal.
Once generated, copy the token as DISCORD_TOKEN
in the .env
file.
Add the following line to your .env
file to switch to Float16 mode
and reduce the VRAM usage by half (roughly) :
STABLEDIFFUSION_MODE=fp16
Add the following lines to your .env
file, to use the
Waifu Diffusion model instead :
STABLEDIFFUSION_MODEL_NAME=hakurei/waifu-diffusion
Here's the list of optional environment variables you
can define to configure the bot.
When not defined, their Default value will be used.
-
IMAGES_OUTPUT_DIRECTORY
Define where you want to store the generated pictures.
Default :generated
Spaces are allowed. No need to use quotes.
The directory will be created if it doesn't exist.
Example :IMAGES_OUTPUT_DIRECTORY=another folder
When running the bot, this setting is ignored when
SAVE_IMAGES_TO_DISK
is set tofalse
. -
SAVE_IMAGES_TO_DISK
Define whether generated files are saved on the disk or not.
Default :true
Example :SAVE_IMAGES_TO_DISK=false
This setting is ignored when testing Stable Diffusion alone.
-
STABLEDIFFUSION_MODEL_NAME
Determine the HuggingFaces model used byStableDiffusionPipeline
.
Default :CompVis/stable-diffusion-v1-4
Example :STABLEDIFFUSION_MODEL_NAME=hakurei/waifu-diffusion
-
STABLEDIFFUSION_MODE
Allows you to select between different StableDiffusion modes.
Default :fp32
Currently only fp16 and fp32 are supported.
VRAM usage is lower in fp16, so if you're low on VRAM, set this tofp16
.
Example :STABLEDIFFUSION_MODE=fp16
-
MAX_IMAGES_PER_JOB
Maximum number of images to output per job request.
Default :64
That means that the NUMBER OF IMAGES typed in/degudiffusion
form will be clamped to that maximum value.
Example :MAX_IMAGES_PER_JOB=8
-
MAX_INFERENCES_PER_IMAGE
Maximum number of inferences steps per image.
Default :120
This clamps the INFERENCES number typed in/degudiffusion
form to that maximum value.
Example :MAX_INFERENCES_PER_IMAGE=30
-
MAX_GUIDANCE_SCALE_PER_IMAGE
Maximal guidance scale allowed.
Default :20
This clamps the GUIDANCE SCALE number typed in/degudiffusion
form will be clamped to that maximum value.
Example :MAX_GUIDANCE_SCALE_PER_IMAGE=7.5
-
IMAGES_WIDTH
andIMAGES_HEIGHT
The width and height of generated images.
Default :512
Be EXTREMELY careful with this one, VRAM usage grows dramatically when using higher values.
I highly recommend to switch to fp16 when using more than 512x512.
Going below 512 in any direction will generally lead to garbage results.
Example :
IMAGES_WIDTH=768
IMAGES_HEIGHT=768
-
MAX_IMAGES_BEFORE_THREAD
The number of images after which the bot will automatically create a thread.
Default :2
That means that if you set it to 5 :
When requesting up to 5 images per job, the bot will output everything on the channel from where the job request was done.
When requesting 6 images or more, the bot will create a thread and send the results inside this thread.
Example :MAX_IMAGES_BEFORE_THREAD=5
-
COMPACT_RESPONSES
When set toTrue
ortrue
, the job response will only include the pictures, without any further details (like the Seed, Actual Prompt.).
Default :false
Example :COMPACT_RESPONSES=True
You can still use "Check Degu PNG Metadata" when using compact responses.
-
DEFAULT_IMAGES_PER_JOB
The default NUMBER OF IMAGES used in/degudiffusion
form.
Default :8
Example :DEFAULT_IMAGES_PER_JOB=3
-
DEFAULT_PROMPT
The default PROMPT used in/degudiffusion
form.
Default :Degu enjoys its morning coffee by {random_artists}, {random_tags}
Example :DEFAULT_PROMPT=A Nendoroid of a Chipmunk by {random_artists}, {lyuma_cheatcodes}
-
DEFAULT_SEED
The default SEED used in/degudiffusion
form.
Default to an empty value.
Note that you don't have to type a SEED value, in Degu Diffusion.
When no seed is provided, a random seed is generated for you.
Example :DEFAULT_SEED=-1
-
DEFAULT_INFERENCES_STEPS
The default number of INFERENCES used in/degudiffusion
form.
Default :60
Example :DEFAULT_INFERENCES_STEPS=30
-
DEFAULT_GUIDANCE_SCALE
The default GUIDANCE SCALE used in/degudiffusion
form.
Default :7.5
Example :DEFAULT_GUIDANCE_SCALE=15
-
SEED_MINUS_ONE_IS_RANDOM
Determine if -1 should be interpreted as a random seed or an actual seed value.
Default : True
By default, now,-1
is treated as a random value, since many users are used to type-1
to get a random seed.
Note that you don't have to type a SEED value, in Degu Diffusion.
When no seed is provided, a random seed is generated for you.
Example :SEED_MINUS_ONE_IS_RANDOM=false
-
STABLEDIFFUSION_LOCAL_ONLY
ForceStableDiffusionPipeline
to use predownloaded local files only, and avoid connecting to the internet.
Default :false
Example :STABLEDIFFUSION_LOCAL_ONLY=true
When set to true,
HUGGINGFACES_TOKEN
is not required anymore. -
STABLEDIFFUSION_CACHE_DIR
Determine whereStableDiffusionPipeline
download its files to.
Empty by Default
This is mainly used for Docker setups.
When not set, or set to an empty string,StableDiffusionPipeline
will determine where to download its files.
Example :STABLEDIFFUSION_CACHE_DIR=sd_cache
The directory will be created if it doesn't exist.
-
TORCH_DEVICE
Determine the PyTorch device used. Default to "cuda".
Default :cuda
Any value other thatcuda
is untested.
Example :TORCH_DEVICE=rocm
-
FORM_NUMBER_OF_IMAGES_INPUT_MAX
Maximum number of characters allowed for NUMBER OF IMAGES in /degudiffusion form.
Default :4
Example :FORM_NUMBER_OF_IMAGES_INPUT_MAX=3
-
FORM_PROMPT_INPUT_MAX
Maximum number of characters allowed for PROMPT in /degudiffusion form.
Default :500
Example :FORM_PROMPT_INPUT_MAX=100
Note that with the default setup, StableDiffusion will parse up to 77 tokens and ignore the rest.
-
FORM_SEED_INPUT_MAX
Maximum number of characters allowed for SEED in /degudiffusion form.
Default :38
Example :FORM_SEED_INPUT_MAX=200
-
FORM_INFERENCES_INPUT_MAX
Maximum number of characters allowed for INFERENCES in /degudiffusion form.
Default :3
Example :FORM_INFERENCES_INPUT_MAX=2
-
FORM_GUIDANCE_SCALE_INPUT_MAX
Maximum number of characters allowed for GUIDANCE SCALE in /degudiffusion form.
Default :6
Example :FORM_GUIDANCE_SCALE_INPUT_MAX=3
Be careful, conversion from float to string adds at least one decimal.
This conversion might lead to errors when using theDEFAULT_GUIDANCE_SCALE
, while preparing the form.
For example, ifDEFAULT_GUIDANCE_SCALE=7
then the displayed value will be7.0
, and will take 3 characters.
IfFORM_GUIDANCE_SCALE_INPUT_MAX
is set to2
characters, the form will become unuseable.
{random_artists}
Adds 1 to 5 random artists names{random_tags}
Adds 0 to 4 random tags{lyuma_cheatcodes}
Try it !
You can add more by editing replacers.json
If the file config/denied_expressions.txt exists, each line of these files
is interpreted as a Regular Expression.
When a user request an image generation, if the prompt matches any of these
regular expressions, the generation request is denied.
This file is optional and is not created by default.
There are various online tools to build Regular Expressions, like, for example, https://regex101.com/ .
Using the following config/denied_expressions.txt
Table
If the prompt contains the characters "table", the generation request will be denied.
Meaning that the prompt :
- "Hamsters playing roulette around a tABlE" will be denied.
- "Horses in their Stables" will be denied.
Using the following config/denied_expressions.txt
\bTable
Sofa\b
chair
Prompt will be denied if any of thees conditions is true :
- The prompt contains any word starting with "table"
- The prompt contains any word ending with "sofa"
- The prompt contains the characters "chair"
Meaning that the promt :
- "A pic-nic table" will be denied.
- "People playing Tabletop RPG" will be denied.
- "An Unstable portal" will be accepted.
- "A photo of a bed-sofa" will be denied.
- "A world Sofar" will be accepted.
- "Reunion of Armchair general" will be denied.
The regular expressions are compiled with the re
Python
standard library, using only one flag : re.IGNORECASE
.
Here are two screenshots with :
If you're not too familiar with StableDiffusion, remember that this will eat your VRAM for breakfast.
Add STABLEDIFFUSION_MODE=fp16
to your .env
if you want to run in
FP16 mode, reducing the amount of VRAM used.
While this can reduce the amount of VRAM used by 1.5 ~ 2 times,
remember that it's still GPU intensive and VRAM heavy.
- The 'Job done' message is sent just before the last result, due to timing issues.
- Invoking
/degudiffusion
from a Discord thread by cause errors, since the software will try to create a thread inside a thread, which is not supported by Discord.