SKILL.md - ComfyUI Controller

Skill Description

Advanced ComfyUI batch video and image generation system with LTX-2.3 support, browser automation, workflow execution, and model download capabilities.

Features

• 🎬 Batch Video Generation: Generate 10-100 videos in batch
• 🖼️ LTX2 Model Support: Multi-version LTX2 video generator (v1/v2/v3)
• 🌐 Browser Automation: Browser-based video generation
• ⚡ Workflow Execution: Run ComfyUI workflows via HTTP API
• ⬇️ Model Download: Auto-download model weights from URLs
• 🚀 Turbo Z Generation: Fast image generation
• 🎵 Batch Scene Generation: Music video multi-scene rendering
• 🔧 Workflow Management: Automatic workflow execution

Installation

pip install -r requirements.txt

System Requirements:

• Python 3.8+
• ComfyUI installed at ~/ComfyUI
• GPU ≥24GB VRAM (recommended for LTX2)
• playwright, requests

Usage

Batch Generate 10 Videos

python auto_generate_10_videos.py

Batch Generate 100 Xianxia News

python auto_run_100_xiuxian.py

LTX2 Video Generation (Recommended)

python generate_xiuxian_ltx2_v2.py

Recommended Settings:

• Steps: 15 (best balance, ~57 min)
• Frames: 72 (3s test) or 480 (20s final)
• Mode: I2V + LoRA

Run ComfyUI Workflow

# Edit workflow JSON first, then run
~/ComfyUI/venv/bin/python scripts/comfyui_run.py \
  --workflow assets/tmp-workflow.json

Download Model Weights

# Download from URL
echo "https://example.com/model.safetensors" | \
~/ComfyUI/venv/bin/python scripts/download_weights.py \
  --base ~/ComfyUI

# Specify subfolder
~/ComfyUI/venv/bin/python scripts/download_weights.py \
  --base ~/ComfyUI --subfolder loras \
  https://example.com/lora.safetensors

Browser Video Generation

python browser_video_gen.py

Turbo Z Image Generation

python z_turbo_generate.py

Batch Scene Generation (Music Video)

Use scripts/batch_scenes.js for automation:

// Run in browser console
const scenes = [
  {
    name: "scene_01",
    prompt: "A lonely girl running through rain at night",
    image: "unified_ref.png",
    steps: 15,
    frames: 72
  }
];

for (const scene of scenes) {
  await comfyui_batch.configureScene(scene);
  // Click Run button
}

File Structure

comfyui-controller/
├── auto_generate_10_videos.py      # Batch generate 10 videos
├── auto_run_100_xiuxian.py         # Auto run 100 xianxia news
├── batch_xiuxian_videos.py         # Batch process xianxia videos
├── browser_video_gen.py            # Browser video generator
├── generate_xiuxian_ltx2_v2.py     # LTX2 v2 generator
├── z_turbo_generate.py             # Turbo Z generator
├── comfyui_smart_controller.py     # Smart controller main
├── scripts/
│   ├── comfyui_run.py              # Workflow execution
│   ├── download_weights.py         # Model weight downloader
│   └── batch_scenes.js             # Batch scene automation
├── assets/
│   ├── default-workflow.json       # Default workflow
│   └── tmp-workflow.json           # Temporary workflow
└── LTX2_VIDEO_GUIDE.md             # LTX2 best practices

Performance Reference

Best Practices

1. 15 steps is the sweet spot - I2V converges at 15-20 steps
2. Unified reference image - Same input for all scenes
3. Reload workflow every time - Avoid VAEDecode failures
4. Never reload during execution - Current run will fail
5. Frame selection - 72 frames (3s) for testing, 480 frames (20s) for final

Troubleshooting

Q: ComfyUI server not reachable? A: Check if ~/ComfyUI/main.py exists and server is running on 127.0.0.1:8188

Q: VAEDecode validation failed? A: Reload entire workflow, wait for models to fully load

Q: Output video not saved? A: Check for VAEDecode errors in log, then re-run

Q: How to download models? A: Use scripts/download_weights.py with URLs

License

MIT License

Author

yun520-1

GitHub

https://github.com/yun520-1/ComfyUI-Controller-Little-bug