dots.tts: continuous acoustic latents, flow matching and 48 kHz autoregressive speech
A technical walkthrough of AudioVAE, semantic re-encoding, the Qwen backbone and AR flow matching in SOAR and MeanFlow profiles.
SPEECH SYNTHESIS
dots.tts does not reduce speech to discrete audio tokens. AudioVAE supplies continuous latents, a semantic encoder compresses generated patches for the language backbone, and a flow head predicts the next acoustic trajectory.
Conditional flow matching
The model learns L=E||v_θ(x_t,t,h,s)-u_t||², conditioned on language hidden state h and speaker information s. SOAR adds self-corrective alignment; MF distils the trajectory to four evaluations.
SOAR and MeanFlow trade different errors
The official Seed-TTS-Eval table gives SOAR 79.2 average SIM across the three published splits and the four-step MeanFlow model 78.2. SOAR is self-corrective aligned; MeanFlow distils a longer teacher trajectory into a four-step student, changing both speed and acoustic failure modes.
Reference audio and random voices
The official usage notes recommend a short, clean reference whose transcript matches the recording. Roughly ten seconds is normally sufficient; longer prompts do not automatically improve cloning. Random sampling without a reference is not a persistent character identity.
How acoustic representations evolved
dots.tts conditions continuous flow matching on text, speaker reference and acoustic context. dots.tts-soar adjusts training and weights for open-domain robustness and inference behavior. LongCat-AudioDiT also uses continuous generation, but it is not the same task or protocol.
Speech synthesis moved from acoustic features plus vocoders through autoregressive codec tokens to diffusion and flow matching. The important change is the trajectory and conditioning, not a simple old-to-new label. A short reference carries timbre, accent, microphone and room together.
Autoregressive flow matching on continuous acoustic latents
p(z₁:T|text)=∏_t pθ(z_t|z_<t,text), z_t∈R^dz_tcontinuous acoustic latentz_<tprevious acoustic contexttextnormalized text and controls
AudioVAE compresses 48 kHz waveforms into continuous latents, semantic re-encoding restores content cues, the Qwen2.5 backbone models context, and a flow head restores acoustics. Continuous z is not a discrete codec token.
SOAR at 32 steps and MF at four steps expose a quality-latency tradeoff. Similarity, WER, first-audio latency and full real-time factor must remain separate.
Paper, code and weights can be followed through dots.tts / dots.tts-soar / Technical report.
What each speech metric answers
Each row retains its source architecture, metric or task definition; results without a shared protocol remain separate.
| Model | Architecture or evidence | Scope |
|---|---|---|
| dots.tts | base continuous-flow checkpoint | content, timbre and long-form consistency |
| dots.tts-soar | open-domain/inference model | stability and latency under the same reference |
| autoregressive codec TTS | token-by-token baseline | first audio, repetition and error accumulation |
Content, timbre and long-form listening
Verify numbers, abbreviations, polyphones and pauses word by word before judging similarity. Speaker scores can miss omissions and repetitions; ASR can miss copied noise and room coloration.
Compare SOAR and base with identical text and reference. Record time to first audio, real-time factor, peak memory and long-form stability separately rather than summarizing a system with one fluent sentence.
Risks left by voice similarity
Zero-shot voice conditioning is not speaker consent. Public use needs permission, purpose limits and a withdrawal path.
Emotion, accent and environment can be copied with timbre; the model cannot guarantee extraction of a pure identity factor.
Papers and public files for dots.tts
Model names, numbers and limitations trace to the papers, repositories or model cards below.