Four-stem audio separation: why Huge-SCNet fits the MSS target

Four stems are not another name for backing tracks

Standard four-stem separation usually means vocals, drums, bass, and other. Vocals are the singing track, drums are the drum kit, bass is the bass part, and other holds the remaining instruments: guitar, piano, strings, synths, brass, effects, and more. The backing track heard after muting vocals is really drums, bass, and other summed together.

This target comes from the main datasets and evaluation tradition in music source separation. MUSDB18 and MUSDB18-HQ have long shaped model training and evaluation around these four stems. When the community says 4-stem, it usually means this output target.

The distinction changes the tool choice. If the goal is a clean vocal or a no-vocal instrumental, a two-stem vocal separator is more direct. If the goal is drum practice, bass analysis, sampling, remixing, or editable material cleanup, four stems are the more useful target. Other is not a weaker backing track; it is the hardest instrument basket in the system.

Four stages from U-Net to SCNet

Music source separation is the attempt to recover several sources from one mixed waveform across time and frequency. Earlier models could produce listenable outputs, but the artifacts were familiar: vocal leakage, drum bleed, muddy bass, smeared high frequencies, or a torn phase quality in dense arrangements.

This history explains why model selection cannot be reduced to vocal quality. Four-stem separation must also preserve drum transients, bass fundamentals and harmonics, vocal boundaries, and the high-frequency details inside other.

The timing is source-scoped as well: the SCNet paper appeared publicly in January 2024, while Huge-SCNet-4stems V1.2 belongs to the later public-checkpoint context. That is why this timeline uses 2024-2026 instead of implying that SCNet itself only appeared after 2025.

Mel-Band RoFormer: vocal strength and four-stem limits

Mel-Band RoFormer was proposed by ByteDance's Speech, Audio, and Music Intelligence (SAMI) team. The paper authors are Ju-Chiang Wang, Wei-Tsung Lu, and Minz Won, with the title page affiliation listed as SAMI, ByteDance. Community pages sometimes summarize the line as ByteDance AI Labs, but the most precise source wording is the paper authors and SAMI, ByteDance.

The model is attractive because it shapes frequency bands closer to human hearing. The ear is more sensitive to low and mid frequencies, where vocal fundamentals, consonants, breath, and many pitch cues live. Mel-band projection gives this range a finer representation, helping the model follow leads, backing vocals, slides, vibrato, and reverb tails.

The RoFormer family uses rotary position encoding, or RoPE, on both the time and subband sequences. This makes attention scores depend on relative position. Slides, sustains, and repeated phrases are useful structural intuitions for that property, but the paper directly tests only the controlled with-RoPE versus without-RoPE models; it does not report separate slide or long-harmony metrics.

Signal path defined in the paper

BS-RoFormer first converts the mixture waveform into a complex spectrogram. Its band projection and hierarchical Transformer estimate a complex mask, that mask is applied to the mixture spectrum, and inverse STFT reconstructs the target waveform. Mel-RoFormer changes the projection inside f_θ: overlapping mel bands replace the heuristic, non-overlapping split. It does not replace the entire reconstruction path.

That is why Mel-Band RoFormer is a strong route for vocal/accompaniment, lead/backing-vocal, and clean vocal extraction work. The problem is that four-stem separation is not only about vocals. It also has to separate drums, bass, and other, and other often contains the most difficult high-frequency instrumental detail.

After the papers: how RoFormer became an open tool ecosystem

BS-RoFormer appeared in September 2023 with two kinds of evidence in one paper. The larger L=12 system, trained on MUSDB18HQ plus 500 in-house songs, documented SAMI-ByteDance's first-place SDX23 submission. Smaller MUSDB18HQ-only models supplied controlled ablations for RoPE, band splitting, and overlap-and-average reconstruction. The paper published architecture, training conditions, and result tables. It did not attach the original SDX23 checkpoint or link a ByteDance/SAMI training and inference repository.

Mel-Band RoFormer followed in October 2023. It kept the RoPE Transformers, inner-/inter-band sequence modeling, complex masks, and waveform reconstruction, but replaced the hand-designed non-overlapping split with 60 overlapping mel bands. The one-month interval matters: this was a public revision to the frequency projection prompted by the earlier paper's own limitation, not a second official checkpoint package released alongside the text.

Executable code accumulated outside the papers. The lucidrains repository turned the diagrams and equations into PyTorch model classes and openly invited others to reproduce the separator. Stereo handling, multiple output stems, and the Mel-Band variant made the structure reusable, but code alone could not reconstruct the in-house songs, optimizer state, training duration, or final competition parameters.

Music-Source-Separation-Training then gave that reusable structure a common workshop: dataset loaders, augmentation, training, validation, checkpoint resume, chunked inference, overlap, and ensembling. Its README credits lucidrains for recreating the RoFormer models from the papers and accepts community model submissions. One architecture could now be trained for vocal/instrumental separation, four stems, drum sub-stems, or other specialist targets. There was no single canonical community weight.

Kimberley Jensen's vocal model made the process concrete. The repository says it was trained on more data with ZFTurbo's code, uses an eight-second chunk by default, and exposes num_overlap to reduce stitching artifacts at the cost of longer inference. It also credits contributors to the dataset. Those details explain why a downloadable checkpoint has a character that an architecture name alone cannot describe.

The current MVSep page attributes the first high-quality public weights to Kimberley Jensen, then describes later modification and continued training of the open weights. Bas Curtiz, unwa, becruily, gabox, and other editions appear on the same page with different fullness, bleed, noise, and SDR tradeoffs. This is a branching weight genealogy: the models share Mel-Band RoFormer, but not their data, objective, fine-tuning history, or final sound.

One architecture name can now refer to a competition system, a paper ablation, a vocal checkpoint, a four-stem checkpoint, or a platform ensemble. Accurate copy has to expose three things: who proposed the architecture, where the current weights came from, and which inference recipe delivers the result. Without all three, phrases such as “official ByteDance model” or “Mel-Band four-stem competition score” quietly splice together different artifacts from different moments in this history.

Controlled comparison: Mel does not replace BS across every stem

The cleanest evidence is the 2023 Mel-Band paper's controlled comparison, not a mixture of MVSep pages and unrelated community checkpoints. Every row below uses MUSDB18HQ without extra songs. Within each depth, the main change is the projection: BS-RoFormer's heuristic, non-overlapping bands are replaced by 60 overlapping mel bands.

The result is a tradeoff, not a sweep. At L=6, vocals rise from 10.78 to 11.21 dB and drums from 9.61 to 9.91 dB, while bass falls from 11.43 to 9.64 dB and other slips from 7.86 to 7.81 dB. At L=9, vocals gain 0.58 dB and other gains 0.13 dB, drums loses 0.32 dB, and bass is not reported because training progressed too slowly.

The paper links the bass result to frequency mapping. BS-RoFormer deliberately allocates fine bands at low frequencies, while the mel projection is more helpful for vocals and some non-bass targets. The useful engineering conclusion is therefore task-specific: mel projection is a strong vocal choice, but psychoacoustic motivation alone does not prove four-stem dominance.

SCNet and the four-stem target

SCNet takes a more information-oriented view of the spectrum. The paper authors include Weinan Tong, Jiaxu Zhu, Jun Chen, Shiyin Kang, Tao Jiang, Yang Li, Zhiyong Wu, and Helen Meng; its title page lists Shenzhen International Graduate School, Tsinghua University, Skywork AI PTE. LTD., Peng Cheng Lab, and The Chinese University of Hong Kong among the affiliations. Architecturally, SCNet explicitly splits the mixture spectrogram into subbands and uses sparse compression to handle the different information density of those bands. In simpler terms, it does not put nearly all capacity into the vocal range. It pushes the model to focus on time-frequency regions that actually carry useful instrument signal.

That matters for four stems. Bass sits low but carries important harmonics. Drums combine very short transients with cymbals and room sound. Other may contain guitar, piano, strings, synths, and effects at the same time. Subband modeling gives different frequency regions more targeted treatment, while sparse compression keeps attention away from blank or low-information regions.

The SCNet paper describes the full system as an audio encoder, a separation network based on dual-path RNN, and an audio decoder. The encoder maps the mixture into frequency-domain subband representations, the separation network models structure across time and frequency, and the decoder reconstructs the target stems.

Huge-SCNet-4stems V1.2 belongs to this route. Its output contract is a balanced set of vocals, drums, bass, and other files that can continue into remixing or transcription, rather than a vocal-only optimization target.

It is also important to separate the SCNet paper from the Huge-SCNet-4stems V1.2 checkpoint. The paper provides the sparse-compression architecture, subband design, and MUSDB18-HQ research result; the downloadable artifact is the public Hugging Face checkpoint huge_scnet_4stems_v1.2.ckpt. That community checkpoint cannot inherit the paper's 9.0 dB MUSDB18-HQ result because the data and release provenance differ.

The code boundary should be kept separate too. The paper authors publish thuhcsi/SCNet as the official implementation repository, but the repository says training code is still to be completed and points readers to HTDemucs, Music-Source-Separation-Training, and SCNet-PyTorch as interim training frameworks. Huge-SCNet-4stems V1.2 should therefore be described as a public four-stem checkpoint choice on the SCNet route, not as the official model package from the SCNet paper.

MVSep Ensemble as a platform workflow

MVSep Ensemble 2025.06.30 represents a platform-level multi-model workflow. The public algorithm page lists more than one model family. The vocal path may use UVR-MDX-NET, Demucs, MDX23C, VitLarge23, BS Roformer, Mel Roformer, SCNet XL, and other routes, while bass, drums, and other can call different Demucs or related models.

The strength of this kind of ensemble comes from specialization: assign vocal-heavy material to vocal specialists, low-frequency and transient-heavy material to stronger instrumental separators, then fuse the results. It can be a quality-ceiling reference, but it is not one checkpoint and not a standalone model a user can reproduce by downloading a single file.

MVSep lists the 2025.06.30 ensemble at 13.67 average SDR. That number is useful as a hosted-platform ceiling, but it should not be placed in the same row as a standalone checkpoint. It reflects a server-side workflow that can combine vocal, instrumental, bass, drums, other, and additional instrument-specific paths before producing the final result.

For that reason, discussion of a downloadable single four-stem model must keep platform ensembles outside the single-model category. Otherwise the comparison shifts from model architecture to hosted workflow.

Public four-stem comparison

The table below uses figures from the cited public model and benchmark pages. The scores help locate each model under the same four-stem target. They should not be read as an absolute verdict for every song, genre, or listening case.

The source scope is intentionally narrow: the Huge-SCNet-4stems V1.2 10.02 dB row comes from the public HF / MVSep reference; SCNet XL IHF, BS Roformer, and HTDemucs4 use public ZFTurbo / MVSep Multisong rows. This is not a mixed table of SCNet paper MUSDB18-HQ results, Mel-Band vocal pages, and MVSep Ensemble hosted-workflow scores.

How to read the scores

Higher SDR usually means less distortion against the reference stem. But average SDR cannot replace listening, and it cannot flatten every stem into one answer. One model may be strong on bass and drums while another keeps vocals cleaner. Other is especially difficult because it contains many instruments.

The value of Huge-SCNet-4stems V1.2 is its clear artifact boundary: one public checkpoint name, standard four-stem output, and traceable public reference figures. SCNet XL IHF remains an important comparison. MVSep Ensemble remains a hosted quality-ceiling reference. They answer different questions.

Users should start with the target. Use vocal separation for the cleanest vocal. Use four-stem separation for editable vocals, drums, bass, and other. Use six-stem separation when guitar and piano need their own files.

Output boundary: four-stem balance versus six-stem detail

Six-stem separation pushes other further into guitar, piano, and remaining other. It is more sensitive to guitar strums, piano attacks, keyboard texture, and high-frequency harmonics. A BS Roformer SW six-stem checkpoint therefore represents a different target and training contract from Huge-SCNet-4stems V1.2.

Those claims belong to separate layers: RoFormer describes an architecture family, Mel-Band targets vocal representation, SCNet balances four stems, and MVSep ensembles combine hosted components. Each answers a different question about architecture, output target, public availability, workflow form, or inference contract.

Conclusion: choose the architecture, not the buzzword

Four-stem separation is not a renamed backing track and not a place to paste the most popular vocal model onto every instrument. It asks the model to balance vocals, drums, bass, and other, especially drum transients, bass lows, instrumental high harmonics, and the mixed instrument basket inside other.

Huge-SCNet-4stems V1.2 occupies a clear position around that standard four-stem target. Mel-Band RoFormer remains important for understanding vocal separation, while MVSep Ensemble remains useful as a hosted quality reference. Each belongs to a different layer of the research and deployment history.