In our previous guide, we demonstrated how easy it is to set up ComfyUI using Docker. Building on that success, today we're going to explore an additional step: integrating ComfyUI with Open WebUI. By connecting these two platforms, you'll be able to harness the power of Ollama's prompt generation capabilities directly through ComfyUI's API. Don't let the technical details intimidate you. This integration is surprisingly straightforward.
Jeremy
If you haven't already set up Ollama and Open WebUI, please refer to our in-depth tutorial that will walk you through the process of getting both applications up and running.
Jeremy
How it Works
Before we get started, let's make sure you're on the same page about how this works. Using ComfyUI with Open WebUI is a bit different from other chat options you may have used before, it requires a slightly more involved process to create images. Ollama will need to generate prompts for us first, and then do the actual image generation work by clicking the image icon that appears after connecting ComfyUI. Here is an example of how it looks.
If you're fine with having to go the extra mile, this does work really well. I made a few default prompt suggestions to make it quicker. This way I can just change key words in the suggested prompts as needed.
Connect ComfyUI to Open WebUI
- Setup your ComfyUI workflow in the manner of which you want it to perform through Open WebUI. Be sure to choose the Model and adjust the steps and other settings.
- Export the Workflow as the Export (API) option and save it.
- Go to your Open WebUI settings and in the Admin Panel, click Settings and browse to the Images section. Here you will enable image generation and image prompt generation. Choose ComfyUI as the Engine then add the ComfyUI base URL. Now upload the workflow file. You won't have to add an API key so ignore that option.
- Towards the bottom under ComfyUI Workflow Nodes, you need to match the Node IDs in the workflow you downloaded to the corresponding Node names in the list. And this is the most confusing part of the process but you can look in the workflow file and see where the IDs are.
- Scroll down and choose the model and adjust the image size. Keep in mind, the larger the image, the longer it will take to generate.
- Click save and test it out.
Remember, you have to tell Ollama through Open WebUI to create the prompt for the image you want to generate then click the little image icon under the output. It works best if you tell it to be concise and only respond with the prompt.
You'll see it change from an icon to 3 little animated dots letting you know it's working on creating the image. Let's see what I got!
Not bad, but not exactly what I was after. We can simply click the regenerate button to generate a new prompt yielding different results!
So, after generating a new prompt with the Regenerate button, you'll need to click the Generate Image button again to see your creation take shape. It's a simple process. If you want to try something different, you can easily regen the prompt and start over again.
Final Notes and Thoughts
Using Open WebUI to make an image takes a few extra steps. You have to tell Ollama what prompt to use first, and then click a button to see the result. It's not the most direct way to create images, but it can be fun to use!
That being said, it's still a great way to experiment with different prompts and see what kind of interesting images you can come up with. And who knows, you might stumble upon something really cool that you wouldn't have found otherwise.
Jeremy
Discussion