$devvkit learn --librarie comfyui-guide

ComfyUI Guide

[ai][image-generation][stable-diffusion][workflow]

AI / LLM Tools

Install

git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
pip install -r requirements.txt
python main.py
# Or download portable package from comfy.org

ComfyUI is a node-based graph interface for Stable Diffusion. Each node is an operation (text encoding, KSampler, VAE decode, upscale, masking), connected by wires into a visual pipeline. This gives you complete control over the generation process — unlike AUTOMATIC1111's linear interface.

Drag and drop a workflow image into ComfyUI to load it — workflows are embedded in the generated PNG metadata. The default "efficient" workflow includes checkpoint loading, positive/negative prompt, empty latent, KSampler, VAE decode, and preview.

ComfyUI supports all SD models: SD1.5, SDXL, SD3, FLUX, Stable Video Diffusion, and Stable Audio. Install custom nodes from the Manager for ControlNet, IP-Adapter, animatediff, and more. The API mode (`--listen`) enables remote generation from scripts.

Setup

Launch ComfyUI— Start the interface.

git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
pip install -r requirements.txt
python main.py
# Opens: http://127.0.0.1:8188

# With GPU:
python main.py --force-fp16  # NVIDIA
python main.py --force-fp16 --use-pytorch-cross-attention  # Faster attention

Add models— Place models in directories.

# Checkpoints → ComfyUI/models/checkpoints/
# LoRAs → ComfyUI/models/loras/
# VAEs → ComfyUI/models/vae/
# ControlNet → ComfyUI/models/controlnet/
# Upscale models → ComfyUI/models/upscale_models/

# Download SDXL (example):
wget -P ComfyUI/models/checkpoints/ https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors

Workflow Basics

Workflow: basic generate— Minimal text-to-image.

# Right-click → Add Node → search for:
# 1. Load Checkpoint (select your model)
# 2. CLIP Text Encode (positive prompt)
# 3. CLIP Text Encode (negative prompt)
# 4. Empty Latent Image (width/height)
# 5. KSampler (steps: 20, cfg: 7, sampler: euler)
# 6. VAE Decode
# 7. Save Image

# Connect: checkpoint → model/CLIP/VAE to each node

Workflow: img2img— Generate from image.

# Add: Load Image node
# Replace: Empty Latent with VAE Encode
# Hook: Load Image → VAE Encode (pixels→latent)
# VAE Encode → KSampler (latent_image input)
# Lower denoising (0.3-0.6) for img2img

Custom Nodes

Install custom nodes— Use ComfyUI Manager.

# Clone Manager:
git clone https://github.com/ltdrdata/ComfyUI-Manager.git \
  ComfyUI/custom_nodes/ComfyUI-Manager

# Restart ComfyUI, click "Manager" button
# Install:
# - ControlNet
# - IPAdapter
# - AnimateDiff
# - WAS Node Suite
# - Efficiency Nodes

API Mode

API mode— Generate from code.

import json
import requests

# Get default workflow from ComfyUI: Settings → Save (API Format)
with open('workflow_api.json') as f:
    workflow = json.load(f)

# Change prompt node
workflow['6']['inputs']['text'] = 'a beautiful landscape'

response = requests.post(
    'http://127.0.0.1:8188/api/prompt',
    json={'prompt': workflow}
)
print(response.json())

Tips

Batch generation— Generate multiple images.

# Use "Latent Batch" node or:
# Workflow with "Empty Latent" → repeat with different seeds
# Use "Primitive" node for seed, hook to KSampler seed input
# Then iterate seeds:

# Or in CLI:
python main.py --quick-test-for "a cat"  # Test generation

# For batching via API, send multiple prompts
# ComfyUI queues them automatically

Upscale workflow— High-res output.

# After VAE Decode, add:
# 1. Upscale Image (by 2x or 4x)
# 2. If needed: use ControlNet tile for upscale

# Ultimate SD Upscale node (from Manager):
# Splits image into tiles, upscales each,
# blends edges — results > original resolution

# Alternative: generate at higher resolution
# directly (SDXL native: 1024x1024)