andyxtreme/autoshorts repository overview

⁠AutoShorts – Extended Edition (Blackwell / Web UI / Dead-Air Removal)

This is an extended variant of AutoShorts⁠ by Divyaprakash Dhurandhar. It keeps the original pipeline and adds the fixes and features needed to run it on current hardware and to configure it without editing files.

AutoShorts analyzes your gameplay videos to identify the most engaging moments—action sequences, funny fails, or highlight achievements—then automatically crops, renders, and adds subtitles or AI voiceovers to create ready-to-upload short-form content.

⁠What's different in this edition

⁠Support

If this edition helps you, I'd appreciate a small donation – thank you! ☕

Please also consider supporting Divyaprakash Dhurandhar, who wrote the original AutoShorts this edition builds on – see Credits⁠.

⁠🎬 Example Output

Here are some shorts automatically generated from gameplay footage:

sample 1	sample 2	sample 3	sample 4

⁠🎥 Showcase: Multi-Language & Style Generation

AutoShorts automatically adapts its editing style, captions, and voiceover personality based on the content and target language. Here are some examples generated entirely by the pipeline:

⁠✨ Features

⁠🎯 AI-Powered Scene Analysis

• Multi-Provider Support: Choose between OpenAI (GPT-5-mini, GPT-4o) or Google Gemini for scene analysis, or run in local mode with heuristic scoring (no API needed)
• Gemini Deep Analysis Mode 🧠: Upload full video to Gemini for context-aware scene detection — the AI sees the whole game, not just short clips
• 7 Semantic Types (all analyzed automatically):
  • action — Combat, kills, intense gameplay, close calls
  • funny — Fails, glitches, unexpected humor, comedic timing
  • clutch — 1vX situations, comebacks, last-second wins
  • wtf — Unexpected events, "wait what?" moments, random chaos
  • epic_fail — Embarrassing deaths, tragic blunders, game-losing mistakes
  • hype — Celebrations, "LET'S GO" energy, peak excitement
  • skill — Trick shots, IQ plays, advanced mechanics, impressive techniques

⁠🎙️ Subtitle Generation

• Speech Mode: Uses OpenAI Whisper to transcribe voice/commentary
• AI Captions Mode: AI-generated contextual captions for gameplay without voice
• Caption Styles:
  • Classic: gaming, dramatic, funny, minimal
  • GenZ Mode ✨: genz - Slang-heavy reactions ("bruh 💀", "no cap", "finna")
  • Story Modes ✨: Narrative-style captions
    • story_news - Professional esports broadcaster
    • story_roast - Sarcastic roasting commentary
    • story_creepypasta - Horror/tension narrative
    • story_dramatic - Epic cinematic narration
  • auto - Auto-match style to detected semantic type
• PyCaps Integration: Multiple visual templates including hype, retro-gaming, neo-minimal
• AI Enhancement: Semantic tagging and emoji suggestions (e.g., "HEADSHOT! 💀🔥")

⁠🔊 AI Voiceover (Qwen3-TTS)

• Voice Design Engine: Powered by Qwen3-TTS 1.7B-VoiceDesign for creating unique voices from natural language descriptions
• Dynamic Voice Generation: AI automatically generates voice persona based on caption style + caption content
• Style-Adaptive Voices: Each caption style has a unique voice preset:
  • GenZ → Casual energetic voice with modern slang
  • Story News → Professional broadcaster
  • Story Roast → Sarcastic playful narrator
  • Story Creepypasta → Deep ominous voice with tension
  • Story Dramatic → Epic movie-trailer narrator
• Natural Language Instructions: Define voice characteristics via text prompts without needing reference audio
• Ultra-Low Latency: Local inference with FlashAttention 2 optimization
• Multilingual Support: Native support for 10+ languages including English, Chinese, Japanese, Korean
• Smart Mixing: Automatic ducking of game audio when voiceover plays

⁠⚡ GPU-Accelerated Pipeline

• Scene Detection: Custom implementation using decord + PyTorch on GPU
• Audio Analysis: torchaudio on GPU for fast RMS and spectral flux calculation
• Video Analysis: GPU streaming via decord for stable motion estimation
• Image Processing: cupy (CUDA-accelerated NumPy) for blur and transforms
• Rendering: PyTorch + NVENC hardware encoder for ultra-fast rendering

⁠📐 Smart Video Processing

• Scenes ranked by combined action score (audio/motion weighting configurable)
• Dead-air removal: pauses without speech or motion are cut out of the clip
• Configurable aspect ratio (default 9:16 for TikTok/Shorts/Reels)
• Smart cropping with optional blurred background for non-vertical footage
• Retry logic during rendering to avoid spurious failures

⁠🛡️ Robust Fallback System

AutoShorts is designed to work even when optimal components fail:

⁠📋 Requirements

⁠Hardware

• NVIDIA GPU with CUDA support (6GB+ VRAM recommended for Qwen3-TTS 1.7B)
• NVIDIA Drivers and System RAM (16GB+ recommended)

⁠Software

• Docker with Compose
• NVIDIA Container Toolkit⁠
• NVIDIA driver with CUDA 12.8 support (required for Blackwell / RTX 50xx)

Everything else — Python, FFmpeg 4.4.2, the CUDA toolchain, the CUDA-enabled decord build and the fonts the caption renderer needs — is inside the image.

⁠🚀 Installation

AutoShorts runs in Docker. The image carries a verified CUDA 12.8 stack, the CUDA-enabled decord build and the fonts the caption renderer needs — all of which are tedious to get right on bare metal.

The image is published on Docker Hub as andyxtreme/autoshorts⁠ (linux/amd64 — CUDA and NVENC rule out ARM).

⁠A — Use the prebuilt image

You only need docker-compose.yml⁠, no source code. Put it in a folder, create an empty settings file, and start:

touch .env            # PowerShell: New-Item -ItemType File .env
docker compose up -d

The web UI is then available at http://<HOST-IP>:8501. gameplay/ and generated/ are created next to the compose file on first start.

⁠B — Build from source

Clone this repository, then build the image under the name the compose file expects:

git clone https://github.com/andyxtreme/autoshorts.git
cd autoshorts
docker build -t andyxtreme/autoshorts:latest .
docker compose up -d

Since the image then already exists locally, Compose uses your build instead of pulling. The build compiles decord against CUDA in a separate stage, so expect it to take a while on the first run.

⁠Running once without the UI

docker compose run --rm -e MODE=batch autoshorts

Or without Compose:

docker run --rm --gpus all --shm-size=8g \
    -e MODE=batch \
    -v $(pwd)/gameplay:/app/gameplay \
    -v $(pwd)/generated:/app/generated \
    -v $(pwd)/.env:/app/.env \
    andyxtreme/autoshorts:latest

⁠⚙️ Configuration

Copy .env.example to .env and configure:

cp .env.example .env

⁠Key Configuration Options

See .env.example for the complete list with detailed descriptions.

⁠Clip Selection

Which moments become clips. Applied before anything is rendered.

⁠Clip Length

⁠Dead Air Removal

Cuts stretches out of a rendered clip where nothing is said and nothing happens, then stitches the rest back together. Runs after rendering and before subtitles, so captions are transcribed from the final timeline.

⁠Caption Layout

⁠How the settings interact

Four pairs are easy to confuse because they sound similar but act at different stages of the pipeline.

ACTION_W_VIDEO vs. SILENCE_MOTION_KEEP — both weigh motion, but at opposite ends. The action weights decide where in the source clips are looked for, before rendering. The motion-keep threshold decides what survives inside a clip, after rendering. Gameplay-heavy shorts want a high ACTION_W_VIDEO and a low SILENCE_MOTION_KEEP; commentary-driven shorts want the opposite.

MIN_SHORT_LENGTH vs. SILENCE_MIN_RESULT — the first governs how much raw material is selected, the second how short the finished clip may end up. They are deliberately separate: select generously, cut tightly. Lowering MIN_SHORT_LENGTH to allow shorter finals would also make the pipeline accept thinner source windows, and it filters scenes as well.

SUBTITLE_MAX_LINES vs. SUBTITLE_OVERFLOW — the line limit alone is only a target. With the default exceed_lines, a generous SUBTITLE_MAX_CHARS still spills onto extra lines. Set SUBTITLE_OVERFLOW=exceed_width for a hard line limit.

MAX_SHORT_LENGTH vs. the detected scene — the maximum is an upper bound, not a target. If scene detection finds a 27-second scene, that caps the window regardless of a higher MAX_SHORT_LENGTH.

⁠📖 Usage

1. Place source videos in the gameplay/ directory — either by copying them in directly, or by uploading them in the dashboard. Files copied in from outside appear in the queue automatically.

2. Start the run with Start Processing on the Generate page, or in a terminal:

   docker compose run --rm -e MODE=batch autoshorts
   

   Copy

3. Generated clips are saved to generated/<source video name>/

⁠🧭 Dashboard (Streamlit UI)

The dashboard is the container's default entrypoint and is served on http://<HOST-IP>:8501. Every setting documented above is editable there, and it also manages the input queue, starts jobs and previews the results.

About	Generate	Browse

Features	Settings	Roadmap

⁠Output Structure

One folder per source video, so clips from different recordings do not interleave:

generated/
└── video_name/
    ├── scene-0.mp4            # Rendered short clip
    ├── scene-0.words.json     # Whisper word-level timings
    ├── scene-0_sub.json       # Caption layout data
    ├── scene-0.ffmpeg.log     # Render log
    ├── scene-1.mp4
    └── ...

Rerunning a video clears its folder first, so you always get one consistent set of clips rather than a mix of runs. Only files the pipeline writes (scene-*) are removed; anything else you keep in that folder is left alone.

⁠🧪 Development

⁠Linting

pip install ruff
ruff check .

⁠Running Tests

pytest -q

⁠Debug Variables

For faster iteration during development, you can skip expensive steps using these environment variables in your .env:

Example workflow for testing subtitles only:

# In .env
DEBUG_SKIP_ANALYSIS=1
DEBUG_SKIP_RENDER=1
DEBUG_RENDERED_CLIPS="generated/test_clip.mp4:action"

⁠🔧 Troubleshooting

⁠🤝 Contributing & Roadmap

We love contributions! Whether you're fixing a bug, adding a feature, or improving documentation:

• Check out our Contributing Guide⁠ to get started.
• See the Roadmap⁠ for our future plans (YOLO Auto-Zoom, Next-Gen TTS, etc.).

⁠🙏 Acknowledgments

This project builds upon the excellent work of:

• artryazanov/shorts-maker-gpu⁠ — Heuristics-based shorts maker
• Binary-Bytes/Auto-YouTube-Shorts-Maker⁠ — Original concept and inspiration

⁠📄 Credits & License

• Original project: AutoShorts⁠ by Divyaprakash Dhurandhar – the scene analysis, GPU rendering, caption and TTS pipeline are his work.
• This edition adds GPU compatibility fixes, dead-air removal, a configurable caption layout and a web UI. See What's different in this edition⁠.
• License: MIT – see LICENSE⁠. Copyright remains with the original author.

Support the original author: