A friendly guide to local AI image gen with Stable Diffusion and Automatic1111

A picture is worth a 1,000 words... or was that a 1,000 TOPS

Integrating Automatic1111 into Open WebUI

If you followed our recent tutorial on building a ChatGPT-style dashboard for retrieval augmentation, you may be interested to know that Automatic1111 can be integrated directly into Open WebUI, allowing you to generate images directly from the chat interface.

To get started, you'll need to launch the Automatic1111 SD Web UI using the --api --listen tags.

./webui.sh --api --listen

Once connected in the Open WebUI dashboard, you can start generating images directly from your LLM chats

From there, log into the Open WebUI dashboard and open up the Admin Settings panel. Select "Images" from the sidebar and set "Automatic1111" as the generation engine. Finally set the Automatic Base URL to:

http://127.0.0.1:7860

Note: If you are running Automatic1111 on a different machine, you'll want to change 127.0.0.1 to that machine's IP address or hostname. Again, please be mindful of security and any firewalls in the way.

If everything is configured correctly, you should be able to select your preferred model, image size, and the number of sampling steps that should be used.

Turns out that AI chatbots are pretty good at generating AI image prompts, go figure

To test it out, start a new chat and ask your large language model of choice to generate an image prompt for you. We recommend starting your prompt with "generate a short two-sentence image prompt for..."

Once it responds, click the image button below which should send it to SD Web UI for processing. After a few moments an image should appear in the chat.

Closing thoughts

Like most AI technologies out there today, image generation is controversial. In the right hands it can be a powerful creative tool. Unfortunately in the wrong ones it also has the potential to displace artists, or worse, be used to create disinformation, propaganda, and other harmful or inappropriate content. We've documented some of this technology here not as an endorsement but to help dispel the hype and show our readers how this stuff works, and how others are using it, whether one personally approves of its use or not.

Efforts are being made to make these models safer and less biased, but we're still very much in the early days of the technology and there are sure to be growing pains. Who can forget when Google had to pull its image-gen feature from Gemini after it started creating ethnically diverse images in entirely inappropriate historical contexts?

There's also the issue of how these models are created in the first place. We focused heavily on Stable Diffusion in this piece because its creator made the models available to the public under an incredibly permissive license. However, it seems that in creating these models Stability AI may have violated the copyrights of artists whose work appeared on the web.

The model builder's use of copyrighted materials is now facing down multiple copyright infringement cases brought by Getty and other artists, who claim their works were used without permission to train its signature model. What this could mean for Stability AI and the broader industry remains to be seen.

In any case, if you do choose to embrace generative AI art, we encourage you to do so in the most respectful and responsible manner possible.

The Register aims to bring you more AI content soon, so be sure to share your burning questions in the comments section. And, if you haven't already, be sure to check our other local AI guides. ®

Editor's Note: Nvidia provided The Register with an RTX 6000 Ada Generation graphics card to support this story and others like it. Nvidia had no input as to the contents of this article.