Specialist audio-to-MIDI: the boundaries of GAPS, Filo, VioPTT and SwiftF0

AUTOMATIC MUSIC TRANSCRIPTION

GAPS, FiloSax, FiloBass, VioPTT and SwiftF0 all emit MIDI, but their input distributions, labels and matching tolerances differ. Route the instrument first; compare only inside the corresponding dataset.

Event F1

After matching notes inside an onset tolerance, P=TP/(TP+FP), R=TP/(TP+FN) and F1=2PR/(P+R). Offset-aware matching is stricter, so every F1 needs its tolerance and matching definition.

Three modelling families

GAPS converts clean or separated solo guitar into note events and MIDI, without string positions, frets, continuous F0 or Pitch Bend. Adjust MIDI velocity by comparing with the original recording. FiloSax targets monophonic jazz saxophone solos and exports discrete pitch, onset, and offset events as MIDI. It does not output continuous pitch, slide, or vibrato tracks. FiloBass targets monophonic jazz double bass (upright bass), not general or electric bass. Use a clean or separated stem; kick and low-frequency bleed can create false notes. VioPTT outputs violin note events and one of four technique labels—détaché, flageolet, spiccato, pizzicato—or none. MIDI velocity is fixed, not measured performance dynamics. SwiftF0 tracks monophonic continuous F0; TelkNet segments that contour into notes and exports MIDI, frame-level pitch CSV, and a plot. It is not a polyphonic full-mix transcriber.

Processing chain

Route clean or separated solo guitar, jazz saxophone, jazz double bass, solo violin, or monophonic melody recordings to the matching specialist model.

How transcription methods evolved

Audio-to-MIDI has no single model that covers every instrument and metric. GAPS targets guitar; FiloSax, FiloBass and VioPTT exploit instrument-specific range and articulation; SwiftF0 is closer to general pitch tracking. Shared file formats do not make their training distributions equivalent.

The field moved from framewise pitch detection to joint onset, offset, pitch and velocity prediction, then toward sequence decoding of complete note events. Progress is visible in repeated notes, sustain, slides and vibrato—not just a larger F1. Paper scores remain incompatible when tolerance windows and matching rules differ.

**GAPS paper dataset-creation flowchart**YouTube audio and guitar scores pass through alignment, manual verification, realignment, and transcription filtering before forming the specialist dataset.Official source: Figure 1 / figs/GAPS_flowchart.png

Event transcription must score both onset and offset

F1=2PR/(P + R), P=TP/(TP + FP), R=TP/(TP + FN)

TPevents matching that paper's tolerance
FPextra predicted events
FNmissed events

Papers define true positives differently: some require onset tolerance only; others also require offset, pitch or technique. An F1 value is not comparable once its matching rule is removed.

GAPS, FiloSax, FiloBass, VioPTT and SwiftF0 target different instruments and annotation levels. Correct routing matters more than combining maxima from unrelated datasets.

Paper, code and weights can be followed through Unified specialist runtime / GAPS paper / VioPTT.

**GAPS paper fretboard-distribution heatmap**The long-tail distribution of string-fret combinations shows why guitar transcription cannot be reduced to one aggregate event F1.Official source: figs/fret_heatmap.png

Compare only after event definitions match

Each row retains its source architecture, metric or task definition; results without a shared protocol remain separate.

Model	Native or derived output	Exact boundary
GAPS	MIDI · pitch/onset/offset	GAPS converts clean or separated solo guitar into note events and MIDI, without string positions, frets, continuous F0 or Pitch Bend. Adjust MIDI velocity by comparing with the original recording.
FiloSax	MIDI · pitch/onset/offset	FiloSax targets monophonic jazz saxophone solos and exports discrete pitch, onset, and offset events as MIDI. It does not output continuous pitch, slide, or vibrato tracks.
FiloBass	MIDI · pitch/onset/offset	FiloBass targets monophonic jazz double bass (upright bass), not general or electric bass. Use a clean or separated stem; kick and low-frequency bleed can create false notes.
VioPTT	MIDI + technique CSV	VioPTT outputs violin note events and one of four technique labels—détaché, flageolet, spiccato, pizzicato—or none. MIDI velocity is fixed, not measured performance dynamics.
SwiftF0	F0 CSV → note segmentation → MIDI	SwiftF0 tracks monophonic continuous F0; TelkNet segments that contour into notes and exports MIDI, frame-level pitch CSV, and a plot. It is not a polyphonic full-mix transcriber.

Aural review of note events

Piano needs sustain and repeated-note checks; saxophone and violin need slides, ornaments and vibrato; bass is easily confused with kick drums and low harmonics. A synthesizer can hide timing errors, so inspect the piano roll against the waveform.

Event F1 changes with the tolerance window. A 50 ms onset match, strict onset match, offset-aware note score and frame F1 answer different questions. The tool records model, threshold and quantization so results can be recomputed.

Musical information MIDI does not preserve

MIDI stores symbolic events, not the original timbre, room or touch. Instrument-identity errors in a mix propagate into arrangement and notation.

Without a shared dataset and matching tolerance rules, source-specific metrics remain separate rather than producing a synthetic cross-paper winner.

Papers and public files for Unified specialist runtime

Model names, numbers and limitations trace to the papers, repositories or model cards below.