jncchds/abeat repository overview

⁠ABeat by CHDS

Automatic Beat Saber map generator: audio file in, playable map (all difficulties, walls, bombs, lights) out. Songs come from an uploaded file or a YouTube / YouTube Music link (only use audio you have the rights to), or from an existing Beat Saver map (zip or BeatSaver link), which is kept as a human version to compare with. A web UI lets you listen, inspect and tune the generator, keeps every generation as a version and compares any two (or a generation against an imported human map) lane by lane; the same server runs locally or in Docker.

audio ─▶ analysis worker (Python) ─▶ analysis.json + song.egg + cover.jpg
           beats/downbeats, sections, onsets per layer, energy
                       │
                       ▼
         Abeat.Core (C#): rhythm selection ─▶ flow planner (beam search) ─▶ walls + lights
                       │                                                     │
                       ▼                                                     ▼
              Abeat.Cli / Abeat.Web                        Info.dat (v2.1) + *.dat (v3.3) + zip

Manual⁠: everything ABeat does, every setting, the CLI and the Docker setup.

⁠Quick start

⁠Desktop app

Download the app for your system from Releases⁠: a portable .exe for Windows (x64), an .AppImage for Linux (x86_64) and a .dmg for macOS (Apple Silicon). It is a normal window with nothing to install; it keeps its folders next to itself (data/, models/, runtime/) and installs the analysis runtime on first start, as the Docker image does. Details in the manual⁠.

⁠Docker (web UI)

docker compose up -d             # then open http://localhost:8080

One small image (~420 MB) for every machine. On first start it installs the analysis runtime with the PyTorch build for your hardware (CPU, NVIDIA CUDA, AMD ROCm or Intel XPU; detected, or ABEAT_ACCEL) into bind-mounted folders: ./docker-data/data (songs, maps: back this up), ./docker-data/models (downloaded models) and ./docker-data/runtime (Python environment, ArcViewer). GPUs: docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d (or .rocm.yml, .intel.yml). Optional features: ABEAT_EXTRAS=lyrics,roformer. Details in the manual⁠. Prebuilt images (amd64, arm64) are on Docker Hub as jncchds/abeat⁠: set image: jncchds/abeat:latest in docker-compose.yml and run docker compose up -d --no-build to skip building.

⁠Local

Requirements: .NET 10 SDK, Node.js 22 (or 20.19+) and uv⁠ (it installs its own Python).

cd analysis && uv sync --extra ml --extra cpu && cd ..   # torch flavor: cpu, cuda, cuda12, rocm or xpu; add --extra lyrics / roformer
(cd src/abeat-ui && npm install && npm run build)   # builds the UI into src/Abeat.Web/wwwroot
dotnet run --project src/Abeat.Web             # web UI on http://localhost:5080
# or the CLI:
dotnet run --project src/Abeat.Cli -- generate "Artist - Song.mp3" -d all
dotnet run --project src/Abeat.Cli -- generate "https://music.youtube.com/watch?v=..." -d all
dotnet run --project src/Abeat.Cli -- check path/to/any/map.zip    # flow report for any map
dotnet run --project src/Abeat.Cli -- movement work/beatsaver      # hand movement per difficulty

Web data defaults to ~/.local/share/abeat (set ABEAT_DATA to change). Songs are only analysed on upload; set ABEAT_AUTO_GENERATE=true to also generate a map right after each analysis. ABEAT_MARK_AI=true adds " (AI)" to the level author of generated maps.

scripts/fetch-arcviewer.sh downloads ArcViewer⁠ (GPL-3.0, ~80 MB) so the web app can open maps in it from the same server (the container fetches it on first start). Without it the ArcViewer button uses the public site.

Folders: the Songs page shows folders first, then songs that are in no folder; folders nest two levels. Drag a song onto a folder or a breadcrumb, or pick its folder on the song page. Deleting a folder keeps its songs. The filter searches all folders and lists matching folders and songs.

Playlists: add versions from a song page ("＋ playlist"), then download the playlist zip and import it in BSManager (Maps → Import; the .bplist inside goes to Playlists → Import). BSManager's one-click links only work for maps published on BeatSaver.

⁠How it works

Analysis (analysis/, Python): beat tracking with beat_this⁠ (or librosa), a constant-BPM grid fit (or, for drifting live recordings, a tempo map written as v3 BPM events), phase refinement against a sharp attack envelope (spectral-flux envelopes lag ~50 ms, which players feel), downbeats, energy curve, novelty-based sections with labels for repeated parts, and onsets per layer (Demucs⁠ stems, the default, or frequency bands with --no-stems). The audio is padded so grid beat 0 is at t = 0 and written as song.egg.

Vocal onsets (per song, re-analysis reuses the stems): spectral flux; sung notes (default), flux onsets kept only where a pitched voice follows (CREPE); or lyric syllables, the lyrics forced-aligned to the vocal stem (MMS aligner) and split into syllables at their vowels. Paste the lyrics on the song page (repeats written out), or let Whisper transcribe them (uv sync --extra ml --extra lyrics); the transcription can be loaded into the lyrics box to correct it.

Drums, notes and stems: drum-stem hits are labelled kick / snare / hat, so hats count less when choosing notes and the drummer plays the lights. --pitched notes transcribes the other and bass stems with basic-pitch⁠ (note lengths drive arcs). --roformer takes the vocal stem from BS-RoFormer (python-audio-separator⁠, extra roformer): cleaner vocals, but slow without a GPU. With lyrics, sections whose words repeat (a returning chorus) share a label even when the arrangement changes.

Lead melody (with stems): one melody line through the song, whether a singer, a violin or a synth carries it. Stems can't tell (a violin or synth lead often lands in the "vocal" stem), so CREPE⁠ pitch on the vocal and other stems decides which one leads, moment by moment, or that nothing does (drum breaks, pads); basic-pitch gives that stem's notes. The line becomes the melody layer: its notes are favoured when choosing notes, and they drive arcs, angle leans and the melody lights. The timeline shows it tinted by the stem carrying it.

Rhythm selection (RhythmSelector): onsets are grouped into 1/12-beat grid slots (sixteenths, and triplets only for songs with a triplet feel), scored by layer weight × strength × metric position × energy, and picked bar by bar to hit a notes-per-second target that follows section energy. Notes are not snapped: each lands at the detected time of its sound (corrected for the stem's detection lag). Strong, isolated hits become doubles, spread over the sections (calm parts get nearly as many as loud ones, as in curated maps). Before a drop the map pauses for a beat or two and lands on a double. Taps: tap along with the song (Tap beats, two runs) to mark where you want notes; each run's latency is removed, taps snap onto the song's onsets, and tapped moments get notes first (tap priority), with a double where both runs tapped. Taps appended later get their own latency. A song with a human map can turn one of its difficulties into taps (Use as taps, exact timing) to remake that map or build easier levels on its rhythm. Fit to taps (Speed and density settings, only when pressed) loosens Expert+'s limits so every tap gets its note and moves the other difficulties by the same ratios. Vocal, bass and other stems use spectral-flux onsets (one per sung syllable or note) rather than energy rises, which fired on consonants and breaths.

Flow planning (FlowPlanner + SwingCostModel): beam search over both sabers' states (position, last swing direction, parity). Each candidate cut is scored for:

• physical flow: parity resets, angle change vs. a clean reversal, saber travel, swing speed, arms crossing, vision blocks, over-extension
• figures and moves: only figures (hand + cell + cut direction), moves (a hand's figure to its next one), stacks (one swing through 2-3 lined-up notes) and double shapes that curated mappers commonly use on that difficulty, plus streaks: runs of 3-16 notes of one hand that recur in human maps. Each hand continues known streaks about as often as human maps do, so overlapping ones chain into long human-made runs; two-hand streaks (4 notes in a row across both hands, like left-right-left-right, doubles included) are matched the same way. Learned from 89 curated BeatSaver maps and 85 official maps (OST 4-8 and music packs). A hand crosses to the other side of the grid only where a known move takes it; the Vocabulary page shows every one of them drawn on the grid; note height never follows the audio, so easier levels keep notes lower like human maps do (top row: ~18 % of notes on Easy, ~26 % on Expert+)
• musical fit: accents prefer big vertical swings; hand roles as players tap along: one hand leads the drops, the other the rest, and the support hand takes the bar downbeats. Where a section has a melody or vocal line to follow (20-80 % of its notes), the leading hand becomes the melody hand and plays the melody, the other hand plays the beat; the two swap where the lead swaps (flow weight melody hand, 0 = off)
• dynamics: loud moments pull to the outer columns and bigger moves, soft ones stay small and central
• bursts: one hand may flick back and forth faster than its usual limit (down to the difficulty's burst gap, up to 4 notes in a row, 6 on Expert+) when each flick is a clean reversal in place, as curated maps do in drum fills; set the burst gap to the hand gap to turn them off
• variety: stagnation penalty and per-phrase target cells keyed by section label, so repeated choruses get recognisably similar patterns

Expression (Expression): the melody notes held longest over the gap to the same hand's next note (0.3-2 s later) get an arc to it, both hands at once on a double, on about as many gaps as curated mappers use (6-9 % by difficulty). A long melody or vocal note the hand has nothing after (it sounds on and the hand cuts again only later) gets a held arc instead: it follows the sound to its end and stops there in a dot without a note, next to the head or one cell towards the hand's next note (a few per map, about 8 % of the arcs, as in curated maps); notes in a rising or falling melody line get a small cut-angle offset that leans with it; notes followed by a roll, flam or stutter too fast for single notes become chains (Hard and up, at most one per 30-120 s, as rare as in curated maps). All of them leave hands, cells and directions alone, so flow is unaffected. Each can be switched off in the settings.

Walls (WallGenerator): dodge walls in note-free gaps, side walls in calm sections (only while notes are played) and short rhythm walls in the outer lanes (full height where free, long enough to make the controller buzz) on the strongest kicks and snares of loud sections, as most curated maps have; full-width crouch walls before drops are optional (off by default). No wall ever holds a note or bomb.

Pattern memory: when a part of the song comes back, the map is planned a second time with a bonus for playing it the way its first occurrence was played, so a returning chorus brings its patterns back (repetition weight).

Environments and lights: every environment of moddable Beat Saber, from The First to The Prodigy. --environment auto (the default) picks one that suits the song: a four-on-the-floor dance track lands in EDM-style stages, a band with a backbeat in the rock ones, dark songs in darker places. Classic environments get classic lights (section palettes, note flashes, drum-driven lasers and rings, plus their extra lasers and moving set pieces); Weave and later get a v3 group lightshow laid out from how curated maps use that environment's light groups, with lights that turn and move: they drift in quiet parts, swing on the bars when it gets loud, fold together over a build-up and burst open in white on the drop, and one group follows the melody; runways and floors that mappers keep lit glow between flashes. Colours come from the cover art (sabers, walls, and lights in their own hues; the boost lights that flash up on drops wear the note colours); after the last note the lights dim with the outro instead of cutting out; Detect copies them into the settings, where any of them can be changed by hand. Click the cover on a song page to replace it with your own image. --modes onesaber,90,360 adds One Saber (planned for one saber over the whole grid) and 90°/360° (the Standard notes with lane rotations).

All weights are in FlowWeights and editable in the UI or a settings file (abeat settings).

Evaluation (FlowAnalyzer): the same physical cost model scores any map, including human-made ones (abeat check), so the weights can be calibrated against maps people like.

Hand movement (MovementAnalyzer, abeat movement): for every pair of consecutive swings of one saber it measures how far the swing turns away from a clean back-and-forth and how far the saber tip travels between them, and turns that into strain, the effective swings per second the move asks for (turns past 45° and travel past one cell count extra). A map's 90th-percentile strain tells which difficulty its movement plays like, calibrated on curated maps; the report cards show it as "plays like …" next to the flow score. The generator matches the human mix of turns, travel and strain for each difficulty, so Expert+ maps no longer move like a human Hard.

⁠Benchmark vs human maps

abeat fetch-maps downloads top-rated curated BeatSaver maps (no mods); abeat bench re-maps each map's own song and compares. Current results on 18 songs / 62 difficulties with Demucs stems (2026-10-04):

The environment layouts (scripts/environment_prior.py: which light groups, event types, axes, ranges, poses, mirror partners, how often each group is switched on and how the basic light types are used next to the groups that curated maps use in each environment, read from BeatSaver without downloading the audio) are embedded as environment-prior.json; rerun the script when Beat Saber adds environments.

The style prior (scripts/style_prior.py: cell and direction shares, figure and double-shape vocabularies per difficulty) is learned from the curators' playlist Curators' Favorites of 2025⁠ plus map 37114:

dotnet run --project src/Abeat.Cli -- fetch-maps --playlist 1116456 -o work/beatsaver-curated
dotnet run --project src/Abeat.Cli -- fetch-maps --map 37114 -o work/beatsaver-curated
python3 scripts/style_prior.py work/beatsaver-curated

⁠Layout

⁠Status / next steps

• Calibrate FlowWeights and the flow score against curated BeatSaver maps (abeat check).
• Learned rhythm selection (which onsets humans map) trained on BeatSaver maps, exported to ONNX.
• Smarter bomb patterns.
• Optional Chroma (custom colours, gradients, lighting) and Noodle Extensions (custom note/wall paths) output, declared as map requirements/suggestions.

⁠Credits

ABeat is made by CHDS, designed and built together with Claude (Anthropic's AI assistant, via Claude Code), which co-wrote the analysis worker, generator, web UI and Docker setup.