bennykok/comfydeploy-runpod-worker repository overview

⁠runpod-worker-comfy

• Quickstart⁠
• Features⁠
• Config⁠
  • Upload image to AWS S3⁠
• Use the Docker image on RunPod⁠
• API specification⁠
  • JSON Request Body⁠
  • Fields⁠
    • "input.images"⁠
• Interact with your RunPod API⁠
  • Health status⁠
  • Generate an image⁠
    • Example request with cURL⁠
• How to get the workflow from ComfyUI?⁠
• Build the image⁠
• Local testing⁠
  • Setup⁠
    • Setup for Windows⁠
  • Test: handler⁠
  • Test: docker image⁠
• Automatically deploy to Docker hub with Github Actions⁠
• Acknowledgments⁠

⁠Quickstart

• 🐳 Use the latest image for your worker: timpietruskyblibla/runpod-worker-comfy:latest⁠
• ⚙️ Set the environment variables⁠
• ℹ️ Use the Docker image on RunPod⁠

⁠Features

• Run any ComfyUI⁠ workflow to generate an image
• Provide input images as base64-encoded string
• The generated image is either:
  • Returned as base64-encoded string (default)
  • Uploaded to AWS S3 (if AWS S3 is configured⁠)
• Build-in checkpoint:
  • sd_xl_base_1.0.safetensors⁠
• Build-in VAE:
  • sdxl_vae.safetensors⁠
  • sdxl-vae-fp16-fix⁠
• Build-in LoRA:
  • xl_more_art-full_v1.safetensors⁠ (Enhancer)
• Based on Ubuntu + NVIDIA CUDA⁠

⁠Config

⁠Upload image to AWS S3

This is only needed if you want to upload the generated picture to AWS S3. If you don't configure this, your image will be exported as base64-encoded string.

• Create a bucket in region of your choice in AWS S3 (BUCKET_ENDPOINT_URL)
• Create an IAM that has access rights to AWS S3
• Create an Access-Key (BUCKET_ACCESS_KEY_ID & BUCKET_SECRET_ACCESS_KEY) for that IAM
• Configure these environment variables for your RunPod worker:

⁠Use the Docker image on RunPod

• Create a new template⁠ by clicking on New Template
• In the dialog, configure:
  • Template Name: runpod-worker-comfy (it can be anything you want)
  • Container Image: <dockerhub_username>/<repository_name>:tag, in this case: timpietruskyblibla/runpod-worker-comfy:latest (or dev if you want to have the development release)
  • Container Registry Credentials: You can leave everything as it is, as this repo is public
  • Container Disk: 20 GB
  • Enviroment Variables: Configure S3⁠
    • Note: You can also not configure it, the images will then stay in the worker. In order to have them stored permanently, we have to add the network volume⁠
• Click on Save Template
• Navigate to Serverless > Endpoints⁠ and click on New Endpoint
• In the dialog, configure:
  • Endpoint Name: comfy
  • Select Template: runpow-worker-comfy (or whatever name you gave your template)
  • Active Workers: 0 (whatever makes sense for you)
  • Max Workers: 3 (whatever makes sense for you)
  • Idle Timeout: 5 (you can leave the default)
  • Flash Boot: enabled (doesn't cost more, but provides faster boot of our worker, which is good)
  • Advanced: Leave the defaults
  • Select a GPU that has some availability
  • GPUs/Worker: 1
• Click deploy
• Your endpoint will be created, you can click on it to see the dashboard

⁠API specification

The following describes which fields exist when doing requests to the API. We only describe the fields that are sent via input as those are needed by the worker itself. For a full list of fields, please take a look at the official documentation⁠.

⁠JSON Request Body

{
  "input": {
    "workflow": {},
    "images": [
      {
        "name": "example_image_name.png",
        "image": "base64_encoded_string"
      }
    ]
  }
}

⁠Fields

⁠"input.images"

An array of images, where each image should have a different name.

🚨 The request body for a RunPod endpoint is 10 MB for /run and 20 MB for /runsync, so make sure that your input images are not super huge as this will be blocked by RunPod otherwise, see the official documentation⁠

⁠Interact with your RunPod API

• In the User Settings⁠ click on API Keys and then on the API Key button
• Save the generated key somewhere, as you will not be able to see it again when you navigate away from the page
• Use cURL or any other tool to access the API using the API key and your Endpoint-ID:
  • Replace <api_key> with your key
  • Replace <endpoint_id> with the ID of the endpoint, you find that when you click on your endpoint, it's part of the URLs shown at the bottom of the first box

⁠Health status

curl -H "Authorization: Bearer <api_key>" https://api.runpod.ai/v2/<endpoint_id>/health

⁠Generate an image

You can either create a new job async by using /run or a sync by using runsync. The example here is using a sync job and waits until the response is delivered.

The API expects a JSON in this form⁠, where workflow is the workflow from ComfyUI, exported as JSON⁠ and images is optional.

Please also take a look at the test_input.json⁠ to see how the API input should look like.

⁠Example request with cURL

curl -X POST -H "Authorization: Bearer <api_key>" -H "Content-Type: application/json" -d '{"input":{"workflow":{"3":{"inputs":{"seed":1337,"steps":20,"cfg":8,"sampler_name":"euler","scheduler":"normal","denoise":1,"model":["4",0],"positive":["6",0],"negative":["7",0],"latent_image":["5",0]},"class_type":"KSampler"},"4":{"inputs":{"ckpt_name":"sd_xl_base_1.0.safetensors"},"class_type":"CheckpointLoaderSimple"},"5":{"inputs":{"width":512,"height":512,"batch_size":1},"class_type":"EmptyLatentImage"},"6":{"inputs":{"text":"beautiful scenery nature glass bottle landscape, purple galaxy bottle,","clip":["4",1]},"class_type":"CLIPTextEncode"},"7":{"inputs":{"text":"text, watermark","clip":["4",1]},"class_type":"CLIPTextEncode"},"8":{"inputs":{"samples":["3",0],"vae":["4",2]},"class_type":"VAEDecode"},"9":{"inputs":{"filename_prefix":"ComfyUI","images":["8",0]},"class_type":"SaveImage"}}}}' https://api.runpod.ai/v2/<endpoint_id>/runsync

# Response with AWS S3 bucket configuration
# {"delayTime":2188,"executionTime":2297,"id":"sync-c0cd1eb2-068f-4ecf-a99a-55770fc77391-e1","output":{"message":"https://bucket.s3.region.amazonaws.com/10-23/sync-c0cd1eb2-068f-4ecf-a99a-55770fc77391-e1/c67ad621.png","status":"success"},"status":"COMPLETED"}

# Response as base64-encoded image
# {"delayTime":2188,"executionTime":2297,"id":"sync-c0cd1eb2-068f-4ecf-a99a-55770fc77391-e1","output":{"message":"base64encodedimage","status":"success"},"status":"COMPLETED"}

⁠How to get the workflow from ComfyUI?

• Open ComfyUI in the browser
• Open the Settings (gear icon in the top right of the menu)
• In the dialog that appears configure:
  • Enable Dev mode Options: enable
  • Close the Settings
• In the menu, click on the Save (API Format) button, which will download a file named workflow_api.json

You can now take the content of this file and put it into your workflow when interacting with the API.

⁠Build the image

You can build the image locally: docker build -t timpietruskyblibla/runpod-worker-comfy:dev --platform linux/amd64 .

🚨 It's important to specify the --platform linux/amd64, otherwise you will get an error on RunPod, see #13⁠

⁠Local testing

Both tests will use the data from test_input.json⁠, so make your changes in there to test this properly.

⁠Setup

• Make sure you have Python >= 3.10
• Create a virtual environment: python -m venv venv
• Activate the virtual environment: .\venv\Scripts\activate (Windows) or source ./venv/bin/activate (Mac / Linux)
• Install the dependencies: pip install -r requirements.txt

⁠Setup for Windows

Note: Our hope was that we can use this Docker Image with Docker Desktop on Windows. But regardless what we did, it was not possible. So we decided to use Ubuntu as part of WSL (Windows Subsystem for Linux) inside of Windows. This works without any problems, but only if you don't run Docker on Windows itself.

To run the Docker image on Windows, we need to have WSL2 and a Linux distro (like Ubuntu) installed on Windows.

• Follow the guide on how to get WSL2 and Linux installed in Windows⁠ to install Ubuntu
  • You can skip the "Install and use a GUI package" part as we don't need a GUI

• When Ubuntu is installed, you have to login to Ubuntu in the terminal: wsl -d Ubuntu
• Update the packages: sudo apt update
• Install Docker in Ubuntu⁠ & then install docker-compose sudo apt-get install docker-compose
• Install the NVIDIA Toolkit in Ubuntu⁠ and create the nvidia runtime

• Enable GPU acceleration on Ubuntu on WSL2 to use NVIDIA CUDA⁠

  • For the step "Install the appropriate Windows vGPU driver for WSL": If you already have your GPU driver installed on Windows, you can skip this

• Add your user to the docker group, so that you can use Docker without sudo: sudo usermod -aG docker $USER

⁠Test: handler

• Run all tests: python -m unittest discover
• If you want to run a specific test: python -m unittest tests.test_rp_handler.TestRunpodWorkerComfy.test_bucket_endpoint_not_configured

You can also start the handler itself to have the local server running: python src/rp_handler.py To get this to work you will also need to start "ComfyUI", otherwise the handler will not work.

⁠Test: docker image

• If you want to run the Docker container, you can use: docker-compose up

⁠Automatically deploy to Docker hub with Github Actions

The repo contains two workflows that publish the image to Docker hub using Github Actions:

• docker-dev.yml⁠: Creates the image and pushes it to Docker hub with the dev tag on every push to the main branch
• docker-release.yml⁠: Creates the image and pushes it to Docker hub with the latest and the release tag. It will only be triggered when you create a release on GitHub

If you want to use this, you should add these secrets to your repository:

⁠Acknowledgments

• Thanks to all contributors⁠ for your awesome work
• Thanks to Justin Merrell⁠ from RunPod for worker-1111⁠, which was used to get inspired on how to create this worker
• Thanks to Ashley Kleynhans⁠ for runpod-worker-a1111⁠, which was used to get inspired on how to create this worker
• Thanks to comfyanonymous⁠ for creating ComfyUI⁠, which provides such an awesome API to interact with Stable Diffusion