Beat This!: recovering beats and downbeats without a DBN
Multi-resolution spectra, RoFormer sequence modelling and shift-tolerant objectives, with beat timing kept separate from note transcription.
MUSIC INFORMATION RETRIEVAL / TECHNICAL NOTE
Beat tracking asks when the pulse occurs. Note transcription asks which instrument played which pitch. Beat This! solves the first problem and folds much of the structure formerly delegated to dynamic Bayesian post-processing into the learned sequence model.
Shift-tolerant supervision
Human beat annotations do not agree at sample precision. With target impulses y_t and frame probabilities p_t, ordinary framewise BCE punishes a musically valid prediction displaced by only a few frames. A compact view of the shift-tolerant objective is:
L_shift = min_{δ∈[-k,k]} BCE(p_t, y_{t+δ})
The tolerance belongs to training, not a post-hoc snap-to-grid operation. Beat and downbeat heads share the sequence representation, but the rarer downbeats still require their own decision path.
Moving the DBN out of the main path
Classical systems use a DBN to impose tempo continuity and meter transitions. Beat This! shows that the main model can produce stable beat and downbeat sequences without that dataset-tuned stage. Second-level beat events, downbeats and interval-derived BPM remain three distinct outputs.
Reproducible acceptance
The official final0.ckpt is pinned by SHA-256 and tasks run offline. Because the final models saw several public beat datasets during training, scores on those sets can be optimistic. Acceptance therefore includes a real complete song and exports the raw timing table for DAW grids, slicing and downstream MIDI quantisation.
How transcription methods evolved
Beat tracking traditionally predicts activations with a neural network and lets a dynamic Bayesian network choose the sequence. Beat This uses stronger sequence modeling and shift-tolerant supervision to produce stable beats and downbeats directly, moving the DBN out of the default path.
Removing post-processing does not remove musical priors. Meter, tempo range and annotation style remain in the data. ISMIR results must be read by dataset and metric; rubato, drumless passages and compound meter remain long-tail cases.
Shift-tolerant loss handles annotation boundaries
L_shift=min_{|δ|≤Δ} BCE(ŷ_t,y_{t+δ})δsmall permitted frame shiftΔannotation tolerance windowBCEbinary cross-entropy on activations
Human beat annotations can differ by a few frames. Shift-tolerant supervision searches a small alignment window, reducing penalties for annotation jitter without allowing arbitrary song-level drift.
Beat, downbeat, BPM and MIDI notes are different outputs. Overlay predictions on waveform and bar grid to inspect half/double tempo, pickups and drumless passages.
Paper, code and weights can be followed through Beat This source and weights / ISMIR 2024 paper.
Compare only after event definitions match
Each row retains its source architecture, metric or task definition; results without a shared protocol remain separate.
| Model | Architecture or evidence | Scope |
|---|---|---|
| Beat This without DBN | end-to-end beats and downbeats | default path and long-span stability |
| Beat This with optional postprocess | special tempo constraints | record parameters to avoid hidden changes |
| activation plus DBN | explicit tempo/transition prior | robust but can lock onto the wrong tempo |
Aural review of note events
Overlay predictions on waveform and bar grid to inspect half/double tempo, pickups, tempo changes and drumless passages. Mean F-measure can hide a whole song shifted by one beat.
Shift-tolerant loss gives sensible gradients near an annotation boundary; it does not grant arbitrary timestamp drift.
Musical information MIDI does not preserve
Beat times are not a complete score. Segmentation, groove, swing ratio and meter changes need higher-level analysis.
Datasets use different windows, continuity rules and downbeat weighting, so rankings must retain the protocol name.
Papers and public files for Beat This source and weights
Model names, numbers and limitations trace to the papers, repositories or model cards below.