timpietruskyblibla/runpod-worker-comfy repository overview

⁠runpod-worker-comfy

• Quickstart⁠
• Features⁠
• Config⁠
  • Upload image to AWS S3⁠
• Use the Docker image on RunPod⁠
  • Create your template (optional)⁠
  • Create your endpoint⁠
  • GPU recommendations⁠
• API specification⁠
  • JSON Request Body⁠
  • Fields⁠
    • "input.images"⁠
• Interact with your RunPod API⁠
  • Health status⁠
  • Generate an image⁠
    • Example request for SDXL with cURL⁠
• How to get the workflow from ComfyUI?⁠
• Bring Your Own Models and Nodes⁠
  • Network Volume⁠
  • Custom Docker Image⁠
    • Adding Custom Models⁠
    • Adding Custom Nodes⁠
    • Building the Image⁠
• Local testing⁠
  • Setup⁠
    • Setup for Windows⁠
  • Testing the RunPod handler⁠
  • Local API⁠
    • Access the local Worker API⁠
    • Access local ComfyUI⁠
• Automatically deploy to Docker hub with GitHub Actions⁠
• Acknowledgments⁠

⁠Quickstart

• 🐳 Choose one of the five available images for your serverless endpoint:
  • timpietruskyblibla/runpod-worker-comfy:3.6.0-base: doesn't contain anything, just a clean ComfyUI
  • timpietruskyblibla/runpod-worker-comfy:3.6.0-flux1-schnell: contains the checkpoint, text encoders and VAE for FLUX.1 schnell⁠
  • timpietruskyblibla/runpod-worker-comfy:3.6.0-flux1-dev: contains the checkpoint, text encoders and VAE for FLUX.1 dev⁠
  • timpietruskyblibla/runpod-worker-comfy:3.6.0-sdxl: contains the checkpoint and VAE for Stable Diffusion XL⁠
  • timpietruskyblibla/runpod-worker-comfy:3.6.0-sd3: contains the checkpoint for Stable Diffusion 3 medium⁠
• ℹ️ Use the Docker image on RunPod⁠
• 🧪 Pick an example workflow⁠ & send it to your deployed endpoint⁠

⁠Features

• Run any ComfyUI⁠ workflow to generate an image
• Provide input images as base64-encoded string
• The generated image is either:
  • Returned as base64-encoded string (default)
  • Uploaded to AWS S3 (if AWS S3 is configured⁠)
• There are a few different Docker images to choose from:
  • timpietruskyblibla/runpod-worker-comfy:3.6.0-flux1-schnell: contains the flux1-schnell.safetensors⁠ checkpoint, the clip_l.safetensors⁠ + t5xxl_fp8_e4m3fn.safetensors⁠ text encoders and ae.safetensors⁠ VAE for FLUX.1-schnell
  • timpietruskyblibla/runpod-worker-comfy:3.6.0-flux1-dev: contains the flux1-dev.safetensors⁠ checkpoint, the clip_l.safetensors⁠ + t5xxl_fp8_e4m3fn.safetensors⁠ text encoders and ae.safetensors⁠ VAE for FLUX.1-dev
  • timpietruskyblibla/runpod-worker-comfy:3.6.0-sdxl: contains the checkpoints and VAE for Stable Diffusion XL
    • Checkpoint: sd_xl_base_1.0.safetensors⁠
    • VAEs:
      • sdxl_vae.safetensors⁠
      • sdxl-vae-fp16-fix⁠
  • timpietruskyblibla/runpod-worker-comfy:3.6.0-sd3: contains the sd3_medium_incl_clips_t5xxlfp8.safetensors⁠ checkpoint for Stable Diffusion 3 medium
• Bring your own models⁠
• Based on Ubuntu + NVIDIA CUDA⁠

⁠Config

⁠Upload image to AWS S3

This is only needed if you want to upload the generated picture to AWS S3. If you don't configure this, your image will be exported as base64-encoded string.

• Create a bucket in region of your choice in AWS S3 (BUCKET_ENDPOINT_URL)
• Create an IAM that has access rights to AWS S3
• Create an Access-Key (BUCKET_ACCESS_KEY_ID & BUCKET_SECRET_ACCESS_KEY) for that IAM
• Configure these environment variables for your RunPod worker:

⁠Use the Docker image on RunPod

⁠Create your template (optional)

• Create a new template⁠ by clicking on New Template
• In the dialog, configure:
  • Template Name: runpod-worker-comfy (it can be anything you want)
  • Template Type: serverless (change template type to "serverless")
  • Container Image: <dockerhub_username>/<repository_name>:tag, in this case: timpietruskyblibla/runpod-worker-comfy:3.6.0-sd3 (or -base for a clean image or -sdxl for Stable Diffusion XL or -flex1-schnell for FLUX.1 schnell)
  • Container Registry Credentials: You can leave everything as it is, as this repo is public
  • Container Disk: 20 GB
  • (optional) Environment Variables: Configure S3⁠
    • Note: You can also not configure it, the images will then stay in the worker. In order to have them stored permanently, we have to add the network volume⁠
• Click on Save Template

⁠Create your endpoint

• Navigate to Serverless > Endpoints⁠ and click on New Endpoint

• In the dialog, configure:

  • Endpoint Name: comfy
  • Worker configuration: Select a GPU that can run the model you have chosen (see GPU recommendations⁠)
  • Active Workers: 0 (whatever makes sense for you)
  • Max Workers: 3 (whatever makes sense for you)
  • GPUs/Worker: 1
  • Idle Timeout: 5 (you can leave the default)
  • Flash Boot: enabled (doesn't cost more, but provides faster boot of our worker, which is good)
  • Select Template: runpod-worker-comfy (or whatever name you gave your template)
  • (optional) Advanced: If you are using a Network Volume, select it under Select Network Volume. Otherwise leave the defaults.

• Click deploy

• Your endpoint will be created, you can click on it to see the dashboard

⁠GPU recommendations

⁠API specification

The following describes which fields exist when doing requests to the API. We only describe the fields that are sent via input as those are needed by the worker itself. For a full list of fields, please take a look at the official documentation⁠.

⁠JSON Request Body

{
  "input": {
    "workflow": {},
    "images": [
      {
        "name": "example_image_name.png",
        "image": "base64_encoded_string"
      }
    ]
  }
}

⁠Fields

⁠"input.images"

An array of images, where each image should have a different name.

🚨 The request body for a RunPod endpoint is 10 MB for /run and 20 MB for /runsync, so make sure that your input images are not super huge as this will be blocked by RunPod otherwise, see the official documentation⁠

⁠Interact with your RunPod API

1. Generate an API Key:

   • In the User Settings⁠, click on API Keys and then on the API Key button.
   • Save the generated key somewhere safe, as you will not be able to see it again when you navigate away from the page.

2. Use the API Key:

   • Use cURL or any other tool to access the API using the API key and your Endpoint ID:
     • Replace <api_key> with your key.

3. Use your Endpoint:

   • Replace <endpoint_id> with the ID of the endpoint⁠. (You can find the endpoint ID by clicking on your endpoint; it is written underneath the name of the endpoint at the top and also part of the URLs shown at the bottom of the first box.)

⁠Health status

curl -H "Authorization: Bearer <api_key>" https://api.runpod.ai/v2/<endpoint_id>/health

⁠Generate an image

You can either create a new job async by using /run or a sync by using /runsync. The example here is using a sync job and waits until the response is delivered.

The API expects a JSON in this form⁠, where workflow is the workflow from ComfyUI, exported as JSON⁠ and images is optional.

Please also take a look at the test_input.json⁠ to see how the API input should look like.

⁠Example request for SDXL with cURL

curl -X POST -H "Authorization: Bearer <api_key>" -H "Content-Type: application/json" -d '{"input":{"workflow":{"3":{"inputs":{"seed":1337,"steps":20,"cfg":8,"sampler_name":"euler","scheduler":"normal","denoise":1,"model":["4",0],"positive":["6",0],"negative":["7",0],"latent_image":["5",0]},"class_type":"KSampler"},"4":{"inputs":{"ckpt_name":"sd_xl_base_1.0.safetensors"},"class_type":"CheckpointLoaderSimple"},"5":{"inputs":{"width":512,"height":512,"batch_size":1},"class_type":"EmptyLatentImage"},"6":{"inputs":{"text":"beautiful scenery nature glass bottle landscape, purple galaxy bottle,","clip":["4",1]},"class_type":"CLIPTextEncode"},"7":{"inputs":{"text":"text, watermark","clip":["4",1]},"class_type":"CLIPTextEncode"},"8":{"inputs":{"samples":["3",0],"vae":["4",2]},"class_type":"VAEDecode"},"9":{"inputs":{"filename_prefix":"ComfyUI","images":["8",0]},"class_type":"SaveImage"}}}}' https://api.runpod.ai/v2/<endpoint_id>/runsync

Example response with AWS S3 bucket configuration

{
  "delayTime": 2188,
  "executionTime": 2297,
  "id": "sync-c0cd1eb2-068f-4ecf-a99a-55770fc77391-e1",
  "output": {
    "message": "https://bucket.s3.region.amazonaws.com/10-23/sync-c0cd1eb2-068f-4ecf-a99a-55770fc77391-e1/c67ad621.png",
    "status": "success"
  },
  "status": "COMPLETED"
}

Example response as base64-encoded image

{
  "delayTime": 2188,
  "executionTime": 2297,
  "id": "sync-c0cd1eb2-068f-4ecf-a99a-55770fc77391-e1",
  "output": { "message": "base64encodedimage", "status": "success" },
  "status": "COMPLETED"
}

⁠How to get the workflow from ComfyUI?

• Open ComfyUI in the browser
• Open the Settings (gear icon in the top right of the menu)
• In the dialog that appears configure:
  • Enable Dev mode Options: enable
  • Close the Settings
• In the menu, click on the Save (API Format) button, which will download a file named workflow_api.json

You can now take the content of this file and put it into your workflow when interacting with the API.

⁠Bring Your Own Models and Nodes

⁠Network Volume

Using a Network Volume allows you to store and access custom models:

1. Create a Network Volume:

   • Follow the RunPod Network Volumes guide⁠ to create a volume.

2. Populate the Volume:

   • Create a temporary GPU instance:
     • Navigate to Manage > Storage, click Deploy under the volume, and deploy any GPU or CPU instance.
     • Navigate to Manage > Pods. Under the new pod, click Connect to open a shell (either via Jupyter notebook or SSH).
   • Populate the volume with your models:

     cd /workspace
     for i in checkpoints clip clip_vision configs controlnet embeddings loras upscale_models vae; do mkdir -p models/$i; done
     wget -O models/checkpoints/sd_xl_turbo_1.0_fp16.safetensors https://huggingface.co/stabilityai/sdxl-turbo/resolve/main/sd_xl_turbo_1.0_fp16.safetensors
     

     Copy

3. Delete the Temporary GPU Instance:

   • Once populated, terminate the temporary GPU instance⁠.

4. Configure Your Endpoint:

   • Use the Network Volume in your endpoint configuration:
     • Either create a new endpoint or update an existing one.
     • In the endpoint configuration, under Advanced > Select Network Volume, select your Network Volume.

Note: The folders in the Network Volume are automatically available to ComfyUI when the network volume is configured and attached.

⁠Custom Docker Image

If you prefer to include your models and custom nodes directly in the Docker image, follow these steps:

1. Fork the Repository:
   • Fork this repository to your own GitHub account.

⁠Adding Custom Models

To include additional models in your Docker image, edit the Dockerfile and add the download commands:

RUN wget -O models/checkpoints/sd_xl_base_1.0.safetensors https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors

⁠Adding Custom Nodes

To include custom nodes in your Docker image:

1. Export a snapshot from ComfyUI Manager⁠ that includes all your desired custom nodes

   1. Open "Manager > Snapshot Manager"
   2. Create a new snapshot by clicking on "Save snapshot"
   3. Get the *_snapshot.json from your ComfyUI: ComfyUI/custom_nodes/ComfyUI-Manager/snapshots

2. Save the snapshot file in the root directory of the project

3. The snapshot will be automatically restored during the Docker build process, see Building the Image⁠

⁠Building the Image

Build your customized Docker image locally:

# Build the base image
docker build -t <your_dockerhub_username>/runpod-worker-comfy:dev-base --target base --platform linux/amd64 .

# Build the SDXL image
docker build --build-arg MODEL_TYPE=sdxl -t <your_dockerhub_username>/runpod-worker-comfy:dev-sdxl --platform linux/amd64 .

# Build the SD3 image
docker build --build-arg MODEL_TYPE=sd3 --build-arg HUGGINGFACE_ACCESS_TOKEN=<your-huggingface-token> -t <your_dockerhub_username>/runpod-worker-comfy:dev-sd3 --platform linux/amd64 .

⁠Local testing

Both tests will use the data from test_input.json⁠, so make your changes in there to test this properly.

⁠Setup

1. Make sure you have Python >= 3.10
2. Create a virtual environment:

   python -m venv venv
   

   Copy
3. Activate the virtual environment:
   • Windows:

     .\venv\Scripts\activate
     

     Copy
   • Mac / Linux:

     source ./venv/bin/activate
     

     Copy
4. Install the dependencies:

   pip install -r requirements.txt
   

   Copy

⁠Setup for Windows

1. Install WSL2 and a Linux distro (like Ubuntu) following this guide⁠. You can skip the "Install and use a GUI package" part.
2. After installing Ubuntu, open the terminal and log in:

   wsl -d Ubuntu
   

   Copy
3. Update the packages:

   sudo apt update
   

   Copy
4. Install Docker in Ubuntu:
   • Follow the official Docker installation guide⁠.
   • Install docker-compose:

     sudo apt-get install docker-compose
     

     Copy
   • Install the NVIDIA Toolkit in Ubuntu: Follow this guide⁠ and create the nvidia runtime.
5. Enable GPU acceleration on Ubuntu on WSL2: Follow this guide⁠.
   • If you already have your GPU driver installed on Windows, you can skip the "Install the appropriate Windows vGPU driver for WSL" step.
6. Add your user to the docker group to use Docker without sudo:

   sudo usermod -aG docker $USER
   

   Copy

Once these steps are completed, you can either run the Docker image directly on Windows using Docker Desktop or switch to Ubuntu in the terminal to run the Docker image via WSL

wsl -d Ubuntu

⁠Testing the RunPod handler

• Run all tests: python -m unittest discover
• If you want to run a specific test: python -m unittest tests.test_rp_handler.TestRunpodWorkerComfy.test_bucket_endpoint_not_configured

You can also start the handler itself to have the local server running: python src/rp_handler.py To get this to work you will also need to start "ComfyUI", otherwise the handler will not work.

⁠Local API

For enhanced local development, you can start an API server that simulates the RunPod worker environment. This feature is particularly useful for debugging and testing your integrations locally.

Set the SERVE_API_LOCALLY environment variable to true to activate the local API server when running your Docker container. This is already the default value in the docker-compose.yml, so you can get it running by executing:

docker-compose up

⁠Access the local Worker API

• With the local API server running, it's accessible at: localhost:8000⁠
• When you open this in your browser, you can also see the API documentation and can interact with the API directly

⁠Access local ComfyUI

• With the local API server running, you can access ComfyUI at: localhost:8188⁠

⁠Automatically deploy to Docker hub with GitHub Actions

The repo contains two workflows that publish the image to Docker hub using GitHub Actions:

• dev.yml⁠: Creates the image and pushes it to Docker hub with the dev tag on every push to the main branch
• release.yml⁠: Creates the image and pushes it to Docker hub with the latest and the release tag. It will only be triggered when you create a release on GitHub

If you want to use this, you should add these secrets to your repository: