aclmb/stemgen
Utility to generate STEM file from stereo tracks using AI magic
Image
Machine learning & AI
0
8.4K
aclmb/stemgen repository overview
Stemgen
Warning
**Breaking change are coming in version 0.5.0!** Stemgen is being rewritten in Rust and will leverage the ONNX version of Demucs. See [#30](https://github.com/acolombier/stemgen/issues/30) if you encounter any issues!
Note
Stemgen currently doesn't have a stable release. Please use carefully!
Stemgen is a library and tool that can be used to generate NI stem files from most audio files. It is inspired from the the tool of the same name Stemgen. Here is how it compares:
- Transparent: no binaries or opaque processes required the generation. Everything is available open source, from the demucsing, till the STEM metadata generation.
- More flexible: support virtually any input format and codec and allows full customisation of the stem metadata.
- Dynamic: easy to ship or run on the go with Docker, or to script to generate many STEM at once.
- Producer friendly: offer a multi platform, open source altrernative to the NI's SteamCreator and allow stem creation from STEM tracks.
Under the hood, it uses:
- Facebook's demucs to split the signal into multiple audio stream
- Torch to generate the audio container with all the stream
- Some Taglib sources to generate the STEM metadata
- Taglib to manage the traditional audio metadata
Usage
stemgen generate [GENERATE OPTIONS, COMMON OPTIONS] FILES... OUTPUT
Generate a NI STEM file out of an audio stereo file.
FILES path(s) to a file supported by the FFmpeg codec available on your
machine
OUTPUT path to an existing directory where to store the generated STEM
file(s)
stemgen create [GENERATE OPTIONS, COMMON OPTIONS] OUTPUT
Create a NI STEM file out of existing stem tracks.
OUTPUT path to the generated STEM file
Options for "genetate":
--model <model_name> Demucs model.
--device <cpu or cuda> Device for the demucs model inference
--ext TEXT Extension for the STEM file
--repo DIRECTORY The local directory to use to fetch models
for demucs.
--model TEXT The model to use with demucs. Use --list-
models to list the supported models. Default
to htdemucs fine-trained
--shifts INTEGER Number of random shifts for equivariant
stabilization to use for demucs. Increase
separation time but improves quality for
Demucs. 10 was used in the original paper.
--overlap FLOAT Overlap between the splits to use for
demucs.
--jobs INTEGER The number of jobs to use for demucs.
--list-models List detected and supported models usable by
demucs and exit
Options for "create":
--mastered FILE Source file for the pre-mastered track
[required]
--drum FILE Source file for the drum stem (the first
one) [required]
--bass FILE Source file for the bass stem (the second
one) [required]
--other FILE Source file for the other stem (the third
one) [required]
--vocal FILE Source file for the vocal stem (the fourth
and last one) [required]
--copy-id3tags-from-mastered Copy all ID3 tags from the mastered track
Common options:
--force Proceed even if the output file already
exists
--verbose Display verbose information which may be
useful for debugging
--use-alac / --use-aac The codec to use for the stem stream stored
in the output MP4.
--drum-stem-label <label> Custom label for the drum stem (the first
one)
--drum-stem-color <hex-color> Custom color for the drum stem (the first
one)
--bass-stem-label <label> Custom label for the bass stem (the second
one)
--bass-stem-color <hex-color> Custom color for the bass stem (the second
one)
--other-stem-label <label> Custom label for the other stem (the third
one)
--other-stem-color <hex-color> Custom color for the other stem (the third
one)
--vocal-stem-label <label> Custom label for the vocal stem (the fourth
and last one)
--vocal-stem-color <hex-color> Custom color for the vocal stem (the fourth
and last one)
--version Display the stemgen version and exit
--help Show this message and exit.
Example
Generating a STEM track from a Stereo MP3
-
Simple usage
stemgen generate "Artist - Title.mp3" . -
Using
htdemucs_ftfor better result, but more memory usage (see the benchmark section)stemgen generate "Artist - Title.mp3" . --model htdemucs_ft
Create a STEM track from pre-splitted STEM tracks
-
Simple usage
stemgen create --mastered "Pre-mastered mix.mp3" \ --drum "drum part.mp3" \ --bass "bass part.mp3" \ --other "other part.mp3" \ --vocal "vocal part.mp3" \ "Artist - Title.stem.mp4" -
Customize the STEM metadata
stemgen create \ --mastered "Pre-mastered mix.mp3" \ --drum "Kick part.mp3" \ --bass "SubBass part.mp3" \ --other "synth part.mp3" \ --vocal "Voices part.mp3" \ --drum-stem-label "Kick" \ --drum-stem-color "#37e4d0" \ --bass-stem-label "SubBass" \ --bass-stem-color "#656bba" \ --other-stem-label "Synths" \ --other-stem-color "#52d034" \ --vocal-stem-label "Voices" \ --vocal-stem-color "#daae2a" \ "Artist - Title.stem.mp4"
Note on STEM customisation
NI recommends using the following labels for the stem:
- Acid
- Atmos
- Bass
- Bassline
- Chords
- Clap
- Comp
- Donk
- Drone
- Drums
- FX
- Guitar
- HiHat
- Hits
- Hook
- Kick
- Lead
- Loop
- Melody
- Noise
- Pads
- Reece
- SFX
- Snare
- Stabs
- SubBass
- Synths
- Toms
- Tops
- Vocals
- Voices
Memory Benchmark
Benchmarks are performed with a 3m30s song with CUDA, running on the following machine spec:
12th Gen Intel(R) Core(TM) i7-12700H
64 GB RAM
NVIDIA GeForce RTX 3050
Samsung 980 PRO SSD
| Model | Memory usage peak | Real time |
|---|---|---|
htdemucs (default) | 1.8 GB | 32.637s |
htdemucs_ft | 3.3 GB | 1m6.427s |
License
Stemgen is released under a MIT license.
Tag summary
Content type
Image
Digest
sha256:cd81cf2c3…
Size
50.7 MB
Last updated
2 months ago
docker pull aclmb/stemgen:main