LongCat-AudioDiT 3.5B: conditioning an audio diffusion transformer on a speaker
Conditional diffusion, APG/CFG, 32 function evaluations and the limits of Seed speaker-similarity measurements.
PAPER READING / ZERO-SHOT VOICE CLONING
LongCat-AudioDiT 3.5B targets text-to-speech and zero-shot cloning. Target text, a speaker reference and prosodic cues meet inside a diffusion transformer that generates acoustic latents. Its published similarity numbers answer a narrower question than naturalness, word error or multilingual accent.
Diffusion objective
A forward process corrupts a clean acoustic latent x₀ into x_t. Conditioned on text and speaker information c, the network predicts noise or velocity:
L=E ||ε−εθ(x_t,t,c)||²
The released quality path uses 32 function evaluations. Unconditioned synthesis uses CFG, while reference-based cloning follows the project's APG method to reduce over-saturation under strong guidance.
From autoregressive audio tokens to AudioDiT
Many zero-shot systems generate discrete acoustic units one by one. LongCat instead restores a continuous acoustic trajectory with a diffusion transformer, while UMT5 supplies text features and the reference supplies identity and prosody. The change moves error accumulation from token order toward iterative denoising and guidance.
Similarity is not naturalness
The official Seed table reports three speaker-similarity splits whose 3.5B values average about 80.0. Those rows can be compared only within the named table; they do not replace MOS, word error, cross-language accent or long-form consistency. A voice can sound similar while misreading a name.
Reference quality and rights
A useful prompt is short, clean and single-speaker. Multiple speakers, music and strong reverb mix environment with identity. Voice similarity is a synthesis property, not an identity-verification signal or permission to impersonate a person.
From autoregressive speech to a diffusion transformer
Zero-shot cloning has often used autoregressive discrete audio tokens: a model predicts acoustic units one by one while a reference supplies speaker conditioning. LongCat-AudioDiT 3.5B instead restores a continuous acoustic latent with a diffusion transformer, using UMT5 for text and the reference for timbre and prosody.
The benefit and cost both follow from global denoising. Sentence structure can be revised across the trajectory, but 32 function evaluations increase latency. APG and CFG also change timbre saturation, pronunciation stability and copied background noise; they are more than a single guidance number.
Speaker conditioning inside diffusion denoising
L=E ||ε−εθ(z_t,t,c_text,c_spk)||²z_tacoustic latent at step tc_texttext conditionc_spkspeaker condition from the reference
This generic conditional-diffusion notation shows text and speaker conditions entering the denoiser together; it is not presented as a line-for-line repository equation. Thirty-two function evaluations repeatedly update the latent.
Speaker similarity and content correctness are separate axes. A noisy prompt can turn room sound into a voice condition, and high similarity does not prove correct names, numbers or pauses.
Paper, code and weights can be followed through LongCat-AudioDiT source / LongCat-AudioDiT 3.5B weights.
Reading the Seed similarity table
The page averages the official ZH, EN and ZH-Hard SIM columns and includes only complete rows. Naturalness and WER remain separate questions.
| Model | Three-split mean SIM | Still audit |
|---|---|---|
| LongCat-AudioDiT 3.5B | about 80.0 | WER, naturalness and latency |
| Qwen3-TTS | about 74.5 | voice design versus cloning |
| VoxCPM | about 74.4 | long-form coherence and RTF |
Clone samples need paired review
Keep both no-reference and reference-conditioned output for the same sentence. Check names, numbers and pauses before timbre, accent and prosody. Listening only for resemblance can hide missing words or copied noise.
A useful reference is short, clean and single-speaker. Multiple voices, strong reverb and music blend identity with environment; repeated samples of the same text reveal variance across similarity, prosody and pronunciation.
Similarity is not identity verification
Speaker similarity measures embedding proximity. It neither proves identity nor grants permission to use a voice. The tool is for reference material the user is entitled to process.
The public repository and 3.5B model card together establish released materials. Similarity numbers remain tied to their named test sets and do not replace separate checks of multilingual pronunciation, long-form stability and perceived naturalness.
Papers and public files for LongCat-AudioDiT source
Model names, numbers and limitations trace to the papers, repositories or model cards below.