AI song generation · Lyrics, style and model comparison

WildSongBench

Choose a dataset to view all metrics for each model. Rows follow the official rank or selected metric. Bold marks each column’s best comparable value, including ties. Different evaluation protocols are ranked separately; missing scores are shown as -.

Deployable entries 10 / source rows 10

192 Chinese/English prompts, 2026-09-12. Public baselines select the lower PER of two candidates using four ASR passes; best-of-8 is separate. Compute is not matched. MiniMax Music 3 includes official prompt rewriting; SongBloom uses a fixed audio prompt. YuE2 uses the YuE2-Vae-legacy evaluation decoder. Author model scores do not measure this site’s Q8 or adapted runtimes.

Rank by metric

WildSongBench
Metric rank	Model	Test setting	SongBench		SongEval aesthetics	SongEval · Musicality (0–5) ↑	AudioBox quality	MuQ-MuLan (0–1) ↑	AllMusicCaps (0–1) ↑	Q3O (0–5) ↑	PER (%) ↓
Metric rank	Model	Test setting	μ7 (0–10) ↑	Musicality (0–10) ↑	SongEval aesthetics	SongEval · Musicality (0–5) ↑	AudioBox quality	MuQ-MuLan (0–1) ↑	AllMusicCaps (0–1) ↑	Q3O (0–5) ↑	PER (%) ↓
1	YuE2 (best-of-8) Public weights · Inference code m-a-p/YuE2-3B@8f24312f187763b9854adeec87e437e2b03bbff1 YuE2-Vae-legacy/model.safetensors Inference code Evaluation settings Weight license (repository declaration): cc-by-nc-4.0	bestOf8	6.963229315476184	6.266602604166664	4.296048333333334	4.250610416666664	8.271365446348986	0.5050805467180908	0.398045457637636	4.70087253751977	9.786539961704843
2	YuE2 Public weights · Inference code m-a-p/YuE2-3B@8f24312f187763b9854adeec87e437e2b03bbff1 YuE2-Vae-legacy/model.safetensors Inference code Evaluation settings Weight license (repository declaration): cc-by-nc-4.0	standard	6.731599404761909	5.907463020833336	4.262452916666666	4.2151218749999995	8.25979095697403	0.5068482221880307	0.40536119728009606	4.681896986601725	8.443630690279864
3	LeVo 2 Public weights · Inference code lglg666/SongGeneration-v2-large@115805364ad74479fb3764fe65970c92faeb1a5a Inference code Evaluation settingsWeight license not declared	standard	6.324671800595234	5.4589927083333345	4.023369479166667	3.981881250000001	8.396623315910498	0.3542381529114209	0.2679880544101252	3.9457878565807953	26.116136186343976
4	MiniMax Music 3 Public weights · Inference code MiniMaxAI/MiniMax-Music3@fbdf52fbaaca799592917417eb05f1899f1255ec Inference code Evaluation settingsWeight license not declared	standard	6.282988392857145	5.348229166666669	4.099352187500001	4.0430729166666675	8.28245102117459	0.39279851930526394	0.36085424468425725	4.43616720498247	6.268414232388418
5	HeartMuLa Public weights · Inference code HeartMuLa/HeartMuLa-oss-3B-happy-new-year@41f6fc68490e11dc43fdabaa6b5767946408c903 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	standard	6.248347544642855	5.496267708333332	4.551859166666667	4.5329015625	8.293305036922296	0.3823466453177389	0.2786190857962841	3.490655976705673	10.705881594471055
6	Muse Public weights · Inference code bolshyC/Muse-0.6b@0fb6b5b0a33ec48d135f046ea09524e52e502a42 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	standard	6.034903943452382	5.169220312499998	3.788667708333335	3.736199999999999	8.051745663086573	0.3936510290513979	0.3465821466803997	4.403758449771435	33.41864858069383
7	ACE-Step 1.5 Public weights · Inference code ACE-Step/Ace-Step1.5@19671f406d603126926c1b7e2adc169acbcade22 Inference code Evaluation settings Weight license (repository declaration): mit	standard	6.011793824404761	5.158814583333332	3.846484166666666	3.805107291666667	8.051833018660545	0.43723272854307044	0.38685302749703016	4.580894501177339	7.457884998484957
8	DiffRhythm 2 Public weights · Inference code ASLP-lab/DiffRhythm2@9aa15742e4889c0eb2e198db6fdab1facf1b6761 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	standard	5.242845982142859	4.477539583333331	3.5244677083333333	3.471093229166668	7.978225273390611	0.37817035724098486	0.32549789230688475	4.087044602340498	18.413265339015286
9	YuE 1 Public weights · Inference code m-a-p/YuE-s1-7B-anneal-en-cot@454c20e1748888800f8e4b3da45125f55482d967 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	standard	4.916463244047622	4.084726562499998	3.214967395833334	3.1524088541666675	7.868339642882347	0.262323199029197	0.2881557388876293	3.7300764989993715	36.37846445448743
10	SongBloom Public weights · Inference code rsxdalv/SongBloom@093351709a0efa263569cc4b679429d38f4508ec Inference code Evaluation settingsWeight license not declared	standard	4.235043750000001	3.4492635416666655	3.2051130208333345	3.204842187499999	8.153916046023369	0.2697471845106823	0.19264957021005102	3.0286910546158463	19.193351857701906

Original records (10)

WildSongBench · 2026-09-12 · 192 ↗Checked: 2026-10-25

LeVo 2 · Table I

Choose a dataset to view all metrics for each model. Rows follow the official rank or selected metric. Bold marks each column’s best comparable value, including ties. Different evaluation protocols are ranked separately; missing scores are shown as -.

Deployable entries 5 / source rows 9

Not ranked: 4

A complete public inference package for these experiments has not been verified. Other weights with the same model name do not qualify.

Suno v5 · Commercial Systems (Closed-source) · Mureka v8 · Commercial Systems (Closed-source) · MiniMax Music 2.5+ · Commercial Systems (Closed-source) · LeVo · Open-source Systems

Table I · 100 Chinese and 100 English songs, 20 professional raters; MOS on a 1–10 scale. Paper sampling: top-k 5000, temperature 0.8.

Rank by metric

LeVo 2 · Table I
Metric rank	Model	Test setting	Subjective Evaluation (MOS ↑ )						Gemini Evaluation ↑			ASR · PER (%) ↓
Metric rank	Model	Test setting	OVL	MEL	ARR	SQ-A	SQ-V	STR	Genre	Emotion	Inst. (%)	ASR · PER (%) ↓
1	LeVo 2 (Ours) Public weights · Inference code lglg666/SongGeneration-v2-large@115805364ad74479fb3764fe65970c92faeb1a5a Inference code Evaluation settingsWeight license not declared	Open-source Systems	5.48	6.12	6.42	7.10	6.53	6.11	6.15	8.72	88.35	8.55
2	ACE-Step 1.5 Public weights · Inference code ACE-Step/Ace-Step1.5@19671f406d603126926c1b7e2adc169acbcade22 Inference code Evaluation settings Weight license (repository declaration): mit	Open-source Systems	4.76	5.71	5.82	6.10	5.62	5.70	7.21	8.35	89.17	19.35
3	HeartMuLa Public weights · Inference code HeartMuLa/HeartMuLa-oss-3B@d12ac79c6b3387d5c9eee456323495d8f08bb09d Inference code Evaluation settingsWeight license not declared	Open-source Systems	4.07	4.94	5.05	5.80	4.98	4.92	4.42	7.55	76.11	19.88
4	YuE Public weights · Inference code m-a-p/YuE-s1-7B-anneal-en-cot@454c20e1748888800f8e4b3da45125f55482d967 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	Open-source Systems	3.05	4.12	4.17	4.23	3.89	3.82	5.45	7.69	80.28	20.09
5	DiffRhythm 2 Public weights · Inference code ASLP-lab/DiffRhythm2@9aa15742e4889c0eb2e198db6fdab1facf1b6761 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	Open-source Systems	2.95	4.05	4.86	4.06	3.83	3.88	5.72	8.02	82.50	11.88

Original records (5)

LeVo 2 · Table I ↗Checked: 2026-10-25

HeartBeats · EN · Table 7

Choose a dataset to view all metrics for each model. Rows follow the official rank or selected metric. Bold marks each column’s best comparable value, including ties. Different evaluation protocols are ranked separately; missing scores are shown as -.

Deployable entries 4 / source rows 10

Not ranked: 6

A complete public inference package for these experiments has not been verified. Other weights with the same model name do not qualify.

Suno-v5 · Suno-v4.5 · Mureka-V7.6 · Udio-v1.5 · MiniMax-2.0 · LeVo

HeartBeats Tables 7–11 evaluate EN, ZH, JA, KO and ES separately. HeartMuLa is the original 3B release and ACE-Step the original release, not HNY or 1.5.

Rank by metric

HeartBeats · EN · Table 7
Metric rank	Model	AudioBox ↑				SongEval ↑						Align ↑ · Tag-Sim	Intel ↓ · PER
Metric rank	Model	CE	CU	PC	PQ	Coh.	Mus.	Mem.	Cla.	Nat.	Avg.	Align ↑ · Tag-Sim	Intel ↓ · PER
1	HeartMuLa (Ours) Public weights · Inference code HeartMuLa/HeartMuLa-oss-3B@d12ac79c6b3387d5c9eee456323495d8f08bb09d Inference code Evaluation settingsWeight license not declared	7.55	7.82	5.89	8.14	4.52	4.36	4.54	4.35	4.24	4.48	0.26	0.09
2	DiffRhythm 2 Public weights · Inference code ASLP-lab/DiffRhythm2@9aa15742e4889c0eb2e198db6fdab1facf1b6761 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	7.23	7.58	6.37	7.88	4.00	3.81	4.03	3.82	3.67	3.87	0.32	0.28
3	ACE-Step Public weights · Inference code ACE-Step/ACE-Step-v1-3.5B@82cd0d7b6322bd28cd4e830fe675ddb6180ce36c Inference code Evaluation settingsWeight license not declared	7.49	7.61	5.86	7.90	3.45	3.17	3.31	3.27	3.06	3.25	0.17	0.21
4	YuE Public weights · Inference code m-a-p/YuE-s1-7B-anneal-en-cot@454c20e1748888800f8e4b3da45125f55482d967 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	6.26	7.21	4.58	7.18	2.80	2.58	2.66	2.54	2.61	2.64	0.14	0.65

Original records (4)

HeartBeats · EN · Table 7 ↗Checked: 2026-10-25

HeartBeats · ZH · Table 8

Choose a dataset to view all metrics for each model. Rows follow the official rank or selected metric. Bold marks each column’s best comparable value, including ties. Different evaluation protocols are ranked separately; missing scores are shown as -.

Deployable entries 4 / source rows 10

Not ranked: 6

A complete public inference package for these experiments has not been verified. Other weights with the same model name do not qualify.

Suno-v5 · Suno-v4.5 · Mureka-V7.6 · Udio-v1.5 · MiniMax-2.0 · LeVo

HeartBeats Tables 7–11 evaluate EN, ZH, JA, KO and ES separately. HeartMuLa is the original 3B release and ACE-Step the original release, not HNY or 1.5.

Rank by metric

HeartBeats · ZH · Table 8
Metric rank	Model	AudioBox ↑				SongEval ↑						Align ↑ · Tag-Sim	Intel ↓ · PER
Metric rank	Model	CE	CU	PC	PQ	Coh.	Mus.	Mem.	Cla.	Nat.	Avg.	Align ↑ · Tag-Sim	Intel ↓ · PER
1	HeartMuLa (Ours) Public weights · Inference code HeartMuLa/HeartMuLa-oss-3B@d12ac79c6b3387d5c9eee456323495d8f08bb09d Inference code Evaluation settingsWeight license not declared	7.46	7.78	5.97	8.12	4.57	4.45	4.58	4.45	4.36	4.48	0.24	0.12
2	DiffRhythm 2 Public weights · Inference code ASLP-lab/DiffRhythm2@9aa15742e4889c0eb2e198db6fdab1facf1b6761 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	7.23	7.59	6.25	8.07	3.92	3.72	3.85	3.76	3.60	3.77	0.29	0.32
3	ACE-Step Public weights · Inference code ACE-Step/ACE-Step-v1-3.5B@82cd0d7b6322bd28cd4e830fe675ddb6180ce36c Inference code Evaluation settingsWeight license not declared	7.60	7.67	6.10	8.14	3.92	3.59	3.73	3.70	3.51	3.69	0.20	0.24
4	YuE Public weights · Inference code m-a-p/YuE-s1-7B-anneal-en-cot@454c20e1748888800f8e4b3da45125f55482d967 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	6.84	7.33	5.24	7.56	3.19	2.97	3.01	2.99	2.98	3.03	0.16	0.50

Original records (4)

HeartBeats · ZH · Table 8 ↗Checked: 2026-10-25

HeartBeats · JA · Table 9

Choose a dataset to view all metrics for each model. Rows follow the official rank or selected metric. Bold marks each column’s best comparable value, including ties. Different evaluation protocols are ranked separately; missing scores are shown as -.

Deployable entries 3 / source rows 8

Not ranked: 5

A complete public inference package for these experiments has not been verified. Other weights with the same model name do not qualify.

Suno-v5 · Suno-v4.5 · Mureka-V7.6 · Udio-v1.5 · MiniMax-2.0

HeartBeats Tables 7–11 evaluate EN, ZH, JA, KO and ES separately. HeartMuLa is the original 3B release and ACE-Step the original release, not HNY or 1.5.

Rank by metric

HeartBeats · JA · Table 9
Metric rank	Model	AudioBox ↑				SongEval ↑						Align ↑ · Tag-Sim	Intel ↓ · PER
Metric rank	Model	CE	CU	PC	PQ	Coh.	Mus.	Mem.	Cla.	Nat.	Avg.	Align ↑ · Tag-Sim	Intel ↓ · PER
1	HeartMuLa (Ours) Public weights · Inference code HeartMuLa/HeartMuLa-oss-3B@d12ac79c6b3387d5c9eee456323495d8f08bb09d Inference code Evaluation settingsWeight license not declared	7.42	7.80	6.32	8.19	4.62	4.47	4.62	4.47	4.38	4.51	0.24	0.20
2	ACE-Step Public weights · Inference code ACE-Step/ACE-Step-v1-3.5B@82cd0d7b6322bd28cd4e830fe675ddb6180ce36c Inference code Evaluation settingsWeight license not declared	7.62	7.69	6.14	8.23	3.71	3.38	3.49	3.42	3.27	3.45	0.23	0.33
3	YuE Public weights · Inference code m-a-p/YuE-s1-7B-anneal-en-cot@454c20e1748888800f8e4b3da45125f55482d967 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	5.79	7.14	3.88	7.25	2.67	2.43	2.45	2.39	2.46	2.48	0.20	0.62

Original records (3)

HeartBeats · JA · Table 9 ↗Checked: 2026-10-25

HeartBeats · KO · Table 10

Choose a dataset to view all metrics for each model. Rows follow the official rank or selected metric. Bold marks each column’s best comparable value, including ties. Different evaluation protocols are ranked separately; missing scores are shown as -.

Deployable entries 3 / source rows 8

Not ranked: 5

A complete public inference package for these experiments has not been verified. Other weights with the same model name do not qualify.

Suno-v5 · Suno-v4.5 · Mureka-V7.6 · Udio-v1.5 · MiniMax-2.0

HeartBeats Tables 7–11 evaluate EN, ZH, JA, KO and ES separately. HeartMuLa is the original 3B release and ACE-Step the original release, not HNY or 1.5.

Rank by metric

HeartBeats · KO · Table 10
Metric rank	Model	AudioBox ↑				SongEval ↑						Align ↑ · Tag-Sim	Intel ↓ · PER
Metric rank	Model	CE	CU	PC	PQ	Coh.	Mus.	Mem.	Cla.	Nat.	Avg.	Align ↑ · Tag-Sim	Intel ↓ · PER
1	HeartMuLa (Ours) Public weights · Inference code HeartMuLa/HeartMuLa-oss-3B@d12ac79c6b3387d5c9eee456323495d8f08bb09d Inference code Evaluation settingsWeight license not declared	7.51	7.79	6.09	8.16	4.56	4.45	4.56	4.43	4.34	4.47	0.21	0.16
2	ACE-Step Public weights · Inference code ACE-Step/ACE-Step-v1-3.5B@82cd0d7b6322bd28cd4e830fe675ddb6180ce36c Inference code Evaluation settingsWeight license not declared	7.53	7.67	6.12	8.17	3.53	3.25	3.33	3.29	3.15	3.31	0.20	0.38
3	YuE Public weights · Inference code m-a-p/YuE-s1-7B-anneal-en-cot@454c20e1748888800f8e4b3da45125f55482d967 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	5.21	6.98	3.61	7.12	2.68	2.43	2.55	2.41	2.44	2.50	0.14	0.56

Original records (3)

HeartBeats · KO · Table 10 ↗Checked: 2026-10-25

HeartBeats · ES · Table 11

Choose a dataset to view all metrics for each model. Rows follow the official rank or selected metric. Bold marks each column’s best comparable value, including ties. Different evaluation protocols are ranked separately; missing scores are shown as -.

Deployable entries 2 / source rows 7

Not ranked: 5

A complete public inference package for these experiments has not been verified. Other weights with the same model name do not qualify.

Suno-v5 · Suno-v4.5 · Mureka-V7.6 · Udio-v1.5 · MiniMax-2.0

HeartBeats Tables 7–11 evaluate EN, ZH, JA, KO and ES separately. HeartMuLa is the original 3B release and ACE-Step the original release, not HNY or 1.5.

Rank by metric

HeartBeats · ES · Table 11
Metric rank	Model	AudioBox ↑				SongEval ↑						Align ↑ · Tag-Sim	Intel ↓ · PER
Metric rank	Model	CE	CU	PC	PQ	Coh.	Mus.	Mem.	Cla.	Nat.	Avg.	Align ↑ · Tag-Sim	Intel ↓ · PER
1	HeartMuLa (Ours) Public weights · Inference code HeartMuLa/HeartMuLa-oss-3B@d12ac79c6b3387d5c9eee456323495d8f08bb09d Inference code Evaluation settingsWeight license not declared	7.39	7.72	6.12	8.16	4.46	4.25	4.46	4.29	4.22	4.34	0.22	0.13
2	ACE-Step Public weights · Inference code ACE-Step/ACE-Step-v1-3.5B@82cd0d7b6322bd28cd4e830fe675ddb6180ce36c Inference code Evaluation settingsWeight license not declared	7.43	7.44	6.18	7.99	3.24	2.92	2.96	2.94	2.87	2.99	0.10	0.21

Original records (2)

HeartBeats · ES · Table 11 ↗Checked: 2026-10-25

HeartBeats · MOS · Table 12

Choose a dataset to view all metrics for each model. Rows follow the official rank or selected metric. Bold marks each column’s best comparable value, including ties. Different evaluation protocols are ranked separately; missing scores are shown as -.

Deployable entries 4 / source rows 5

Not ranked: 1

A complete public inference package for these experiments has not been verified. Other weights with the same model name do not qualify.

Suno-v4.5

Subjective results from Table 12 retain original values and 95% confidence intervals. Each metric is ranked separately.

Rank by metric

HeartBeats · MOS · Table 12
Metric rank	Model	Musicality↑	Harmony↑	Structure↑	Fidelity↑	Creativity↑	Memorability↑	Text Align.↑
1	HeartMuLa (Ours) Public weights · Inference code HeartMuLa/HeartMuLa-oss-3B@d12ac79c6b3387d5c9eee456323495d8f08bb09d Inference code Evaluation settingsWeight license not declared	69.55 ± 4.60	71.06 ± 4.57	73.44 ± 3.99	73.18 ± 3.74	66.73 ± 4.52	65.06 ± 4.93	70.51 ± 4.09
2	ACE-Step Public weights · Inference code ACE-Step/ACE-Step-v1-3.5B@82cd0d7b6322bd28cd4e830fe675ddb6180ce36c Inference code Evaluation settingsWeight license not declared	67.42 ± 4.40	67.16 ± 5.10	69.07 ± 4.31	71.40 ± 3.76	62.04 ± 4.91	61.60 ± 5.21	67.94 ± 4.44
3	DiffRhythm 2 Public weights · Inference code ASLP-lab/DiffRhythm2@9aa15742e4889c0eb2e198db6fdab1facf1b6761 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	57.99 ± 5.38	57.13 ± 5.36	61.63 ± 4.62	62.58 ± 3.82	54.77 ± 5.06	53.15 ± 5.27	61.13 ± 4.45
4	YuE Public weights · Inference code m-a-p/YuE-s1-7B-anneal-en-cot@454c20e1748888800f8e4b3da45125f55482d967 Inference code Evaluation settings Weight license (repository declaration): apache-2.0	56.95 ± 4.88	54.46 ± 5.37	60.07 ± 4.46	60.81 ± 3.99	55.68 ± 4.69	56.29 ± 4.80	61.25 ± 4.15

Original records (4)

HeartBeats · MOS · Table 12 ↗Checked: 2026-10-25

AI music creation

Quick feedback

Models, parameters, and sources

YuE2

LeVo 2 Large

MiniMax Music 3

HeartMuLa 3B

WildSongBench

LeVo 2 · Table I

HeartBeats · EN · Table 7

HeartBeats · ZH · Table 8

HeartBeats · JA · Table 9

HeartBeats · KO · Table 10

HeartBeats · ES · Table 11

HeartBeats · MOS · Table 12

Official / repositories

Papers / technical notes

Parameter guide

Use cases

Workflow

What to check