Explore measured results

Name: Veizik measured results
Creator: Veizik

Choose a task to startModel · Device · workload · metric

Choose a task, then a model, a device, a workload and a metric. Choices that still fit are kept when you change an earlier one.

Include development builds (289)

Pick a result

The result card shows one number with its conditions. It appears when your choices narrow to a single result; nothing is averaged across conditions.

What runs on my device?First-token time Memory requirement

Results

412 results on released builds

Sort Compact rows

JavaScript is off: all 412 released-build results are listed below, unfiltered. Filtering, pagination and the result card need JavaScript. The same data is in the downloadable JSON and CSV.

Measured results. Sorting by value is available only when one metric is selected.
		Metric		Scope	Status	Details
Qwen2.5-0.5B	Apple M4 Pro	Decode speedhigher is better	≥462.8 tok/s	512 in / 128 out	Measuredn=11wide spread	Details for Qwen2.5-0.5B on Apple M4 Pro, Decode speed
Qwen2.5-0.5B	Apple M4 Pro	Decode speedhigher is better	≥358.4 tok/s	3,968 in / 128 out	Measuredn=11	Details for Qwen2.5-0.5B on Apple M4 Pro, Decode speed
Qwen2.5-0.5B	Apple M4 Pro	Decode speedhigher is better	≥353.2 tok/s	8,192 in / 128 out	Measuredn=10	Details for Qwen2.5-0.5B on Apple M4 Pro, Decode speed
Qwen2.5-0.5B	Apple M4 Pro	Decode speedhigher is better	≥279.0 tok/s	16,384 in / 128 out	Measuredn=11wide spread	Details for Qwen2.5-0.5B on Apple M4 Pro, Decode speed
Qwen2.5-0.5B	Apple M4 Pro	Decode speedhigher is better	≥228.6 tok/s	24,576 in / 128 out	Measuredn=9wide spread	Details for Qwen2.5-0.5B on Apple M4 Pro, Decode speed
Qwen2.5-0.5B	Apple M4 Pro	Decode speedhigher is better	≥199.8 tok/s	30,000 in / 128 out	Measuredn=10wide spread	Details for Qwen2.5-0.5B on Apple M4 Pro, Decode speed
Qwen2.5-0.5B	Apple M4 Pro	Prefill speedhigher is better	≥7,979 tok/s	512 in / 128 out	Measuredn=11	Details for Qwen2.5-0.5B on Apple M4 Pro, Prefill speed
Qwen2.5-0.5B	Apple M4 Pro	Prefill speedhigher is better	≥6,929 tok/s	3,968 in / 128 out	Measuredn=11	Details for Qwen2.5-0.5B on Apple M4 Pro, Prefill speed
Qwen2.5-0.5B	Apple M4 Pro	Prefill speedhigher is better	≥5,804 tok/s	8,192 in / 128 out	Measuredn=10wide spread	Details for Qwen2.5-0.5B on Apple M4 Pro, Prefill speed
Qwen2.5-0.5B	Apple M4 Pro	Prefill speedhigher is better	≥4,066 tok/s	16,384 in / 128 out	Measuredn=11wide spread	Details for Qwen2.5-0.5B on Apple M4 Pro, Prefill speed
Qwen2.5-0.5B	Apple M4 Pro	Prefill speedhigher is better	≥3,307 tok/s	24,576 in / 128 out	Measuredn=9wide spread	Details for Qwen2.5-0.5B on Apple M4 Pro, Prefill speed
Qwen2.5-0.5B	Apple M4 Pro	Prefill speedhigher is better	≥2,972 tok/s	30,000 in / 128 out	Measuredn=10wide spread	Details for Qwen2.5-0.5B on Apple M4 Pro, Prefill speed
Qwen2.5-0.5B	Apple M4 Pro	Model weight footprintlower is better	343.6 MB	—	Carried buildsingle run	Details for Qwen2.5-0.5B on Apple M4 Pro, Model weight footprint
Qwen2.5-0.5B	Apple M1 Ultra	Decode speedhigher is better	≥377.4 tok/s	512 in / 128 out	Carried buildn=11wide spread	Details for Qwen2.5-0.5B on Apple M1 Ultra, Decode speed
Qwen2.5-0.5B	Apple M1 Ultra	Decode speedhigher is better	≥305.5 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Ultra, Decode speed
Qwen2.5-0.5B	Apple M1 Ultra	Decode speedhigher is better	≥280.6 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Ultra, Decode speed
Qwen2.5-0.5B	Apple M1 Ultra	Decode speedhigher is better	≥259.1 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Ultra, Decode speed
Qwen2.5-0.5B	Apple M1 Ultra	Decode speedhigher is better	≥205.6 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Ultra, Decode speed
Qwen2.5-0.5B	Apple M1 Ultra	Decode speedhigher is better	≥199.0 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Ultra, Decode speed
Qwen2.5-0.5B	Apple M1 Ultra	Prefill speedhigher is better	≥9,632 tok/s	512 in / 128 out	Carried buildn=11wide spread	Details for Qwen2.5-0.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-0.5B	Apple M1 Ultra	Prefill speedhigher is better	≥13,075 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-0.5B	Apple M1 Ultra	Prefill speedhigher is better	≥11,328 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-0.5B	Apple M1 Ultra	Prefill speedhigher is better	≥8,509 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-0.5B	Apple M1 Ultra	Prefill speedhigher is better	≥6,773 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-0.5B	Apple M1 Ultra	Prefill speedhigher is better	≥5,895 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-0.5B	Apple M1 Ultra	Model weight footprintlower is better	343.6 MB	—	Carried buildsingle run	Details for Qwen2.5-0.5B on Apple M1 Ultra, Model weight footprint
Qwen2.5-0.5B	Apple M1 Max	Decode speedhigher is better	≥273.1 tok/s	512 in / 128 out	Carried buildn=11wide spread	Details for Qwen2.5-0.5B on Apple M1 Max, Decode speed
Qwen2.5-0.5B	Apple M1 Max	Decode speedhigher is better	≥208.9 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Max, Decode speed
Qwen2.5-0.5B	Apple M1 Max	Decode speedhigher is better	≥209.9 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Max, Decode speed
Qwen2.5-0.5B	Apple M1 Max	Decode speedhigher is better	≥170.8 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Max, Decode speed
Qwen2.5-0.5B	Apple M1 Max	Decode speedhigher is better	≥144.1 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Max, Decode speed
Qwen2.5-0.5B	Apple M1 Max	Decode speedhigher is better	≥154.4 tok/s	30,000 in / 128 out	Carried buildn=11wide spread	Details for Qwen2.5-0.5B on Apple M1 Max, Decode speed
Qwen2.5-0.5B	Apple M1 Max	Prefill speedhigher is better	≥7,901 tok/s	512 in / 128 out	Carried buildn=11wide spread	Details for Qwen2.5-0.5B on Apple M1 Max, Prefill speed
Qwen2.5-0.5B	Apple M1 Max	Prefill speedhigher is better	≥7,504 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Max, Prefill speed
Qwen2.5-0.5B	Apple M1 Max	Prefill speedhigher is better	≥6,381 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Max, Prefill speed
Qwen2.5-0.5B	Apple M1 Max	Prefill speedhigher is better	≥4,829 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Max, Prefill speed
Qwen2.5-0.5B	Apple M1 Max	Prefill speedhigher is better	≥3,859 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Max, Prefill speed
Qwen2.5-0.5B	Apple M1 Max	Prefill speedhigher is better	≥3,434 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M1 Max, Prefill speed
Qwen2.5-0.5B	Apple M1 Max	Model weight footprintlower is better	343.6 MB	—	Carried buildsingle run	Details for Qwen2.5-0.5B on Apple M1 Max, Model weight footprint
Qwen2.5-0.5B	Apple M2 Pro	Decode speedhigher is better	≥300.6 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Decode speed
Qwen2.5-0.5B	Apple M2 Pro	Decode speedhigher is better	≥222.0 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Decode speed
Qwen2.5-0.5B	Apple M2 Pro	Decode speedhigher is better	≥192.8 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Decode speed
Qwen2.5-0.5B	Apple M2 Pro	Decode speedhigher is better	≥141.7 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Decode speed
Qwen2.5-0.5B	Apple M2 Pro	Decode speedhigher is better	≥120.9 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Decode speed
Qwen2.5-0.5B	Apple M2 Pro	Decode speedhigher is better	≥110.3 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Decode speed
Qwen2.5-0.5B	Apple M2 Pro	Prefill speedhigher is better	≥5,060 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Prefill speed
Qwen2.5-0.5B	Apple M2 Pro	Prefill speedhigher is better	≥4,664 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Prefill speed
Qwen2.5-0.5B	Apple M2 Pro	Prefill speedhigher is better	≥3,917 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Prefill speed
Qwen2.5-0.5B	Apple M2 Pro	Prefill speedhigher is better	≥2,948 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Prefill speed
Qwen2.5-0.5B	Apple M2 Pro	Prefill speedhigher is better	≥2,356 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Prefill speed
Qwen2.5-0.5B	Apple M2 Pro	Prefill speedhigher is better	≥2,071 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-0.5B on Apple M2 Pro, Prefill speed
Qwen2.5-1.5B	Apple M4 Pro	Decode speedhigher is better	≥215.0 tok/s	512 in / 128 out	Measuredn=11wide spread	Details for Qwen2.5-1.5B on Apple M4 Pro, Decode speed
Qwen2.5-1.5B	Apple M4 Pro	Decode speedhigher is better	≥172.3 tok/s	3,968 in / 128 out	Measuredn=9wide spread	Details for Qwen2.5-1.5B on Apple M4 Pro, Decode speed
Qwen2.5-1.5B	Apple M4 Pro	Decode speedhigher is better	≥159.8 tok/s	8,192 in / 128 out	Measuredn=11	Details for Qwen2.5-1.5B on Apple M4 Pro, Decode speed
Qwen2.5-1.5B	Apple M4 Pro	Decode speedhigher is better	≥134.1 tok/s	16,384 in / 128 out	Measuredn=11wide spread	Details for Qwen2.5-1.5B on Apple M4 Pro, Decode speed
Qwen2.5-1.5B	Apple M4 Pro	Decode speedhigher is better	≥117.8 tok/s	24,576 in / 128 out	Measuredn=11	Details for Qwen2.5-1.5B on Apple M4 Pro, Decode speed
Qwen2.5-1.5B	Apple M4 Pro	Decode speedhigher is better	≥108.8 tok/s	30,000 in / 128 out	Measuredn=7	Details for Qwen2.5-1.5B on Apple M4 Pro, Decode speed
Qwen2.5-1.5B	Apple M4 Pro	Prefill speedhigher is better	≥2,296 tok/s	512 in / 128 out	Measuredn=11	Details for Qwen2.5-1.5B on Apple M4 Pro, Prefill speed
Qwen2.5-1.5B	Apple M4 Pro	Prefill speedhigher is better	≥2,188 tok/s	3,968 in / 128 out	Measuredn=9wide spread	Details for Qwen2.5-1.5B on Apple M4 Pro, Prefill speed
Qwen2.5-1.5B	Apple M4 Pro	Prefill speedhigher is better	≥1,824 tok/s	8,192 in / 128 out	Measuredn=11wide spread	Details for Qwen2.5-1.5B on Apple M4 Pro, Prefill speed
Qwen2.5-1.5B	Apple M4 Pro	Prefill speedhigher is better	≥1,506 tok/s	16,384 in / 128 out	Measuredn=11wide spread	Details for Qwen2.5-1.5B on Apple M4 Pro, Prefill speed
Qwen2.5-1.5B	Apple M4 Pro	Prefill speedhigher is better	≥1,266 tok/s	24,576 in / 128 out	Measuredn=11wide spread	Details for Qwen2.5-1.5B on Apple M4 Pro, Prefill speed
Qwen2.5-1.5B	Apple M4 Pro	Prefill speedhigher is better	≥1,169 tok/s	30,000 in / 128 out	Measuredn=7wide spread	Details for Qwen2.5-1.5B on Apple M4 Pro, Prefill speed
Qwen2.5-1.5B	Apple M4 Pro	Model weight footprintlower is better	926.9 MB	—	Carried buildsingle run	Details for Qwen2.5-1.5B on Apple M4 Pro, Model weight footprint
Qwen2.5-1.5B	Apple M1 Ultra	Decode speedhigher is better	≥260.2 tok/s	512 in / 128 out	Carried buildn=11wide spread	Details for Qwen2.5-1.5B on Apple M1 Ultra, Decode speed
Qwen2.5-1.5B	Apple M1 Ultra	Decode speedhigher is better	≥233.9 tok/s	3,968 in / 128 out	Carried buildn=11wide spread	Details for Qwen2.5-1.5B on Apple M1 Ultra, Decode speed
Qwen2.5-1.5B	Apple M1 Ultra	Decode speedhigher is better	≥201.0 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Ultra, Decode speed
Qwen2.5-1.5B	Apple M1 Ultra	Decode speedhigher is better	≥169.1 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Ultra, Decode speed
Qwen2.5-1.5B	Apple M1 Ultra	Decode speedhigher is better	≥145.8 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Ultra, Decode speed
Qwen2.5-1.5B	Apple M1 Ultra	Decode speedhigher is better	≥142.6 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Ultra, Decode speed
Qwen2.5-1.5B	Apple M1 Ultra	Prefill speedhigher is better	≥4,032 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-1.5B	Apple M1 Ultra	Prefill speedhigher is better	≥4,470 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-1.5B	Apple M1 Ultra	Prefill speedhigher is better	≥4,021 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-1.5B	Apple M1 Ultra	Prefill speedhigher is better	≥3,309 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-1.5B	Apple M1 Ultra	Prefill speedhigher is better	≥2,796 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-1.5B	Apple M1 Ultra	Prefill speedhigher is better	≥2,518 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Ultra, Prefill speed
Qwen2.5-1.5B	Apple M1 Ultra	Model weight footprintlower is better	926.9 MB	—	Carried buildsingle run	Details for Qwen2.5-1.5B on Apple M1 Ultra, Model weight footprint
Qwen2.5-1.5B	Apple M1 Max	Decode speedhigher is better	≥181.1 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Decode speed
Qwen2.5-1.5B	Apple M1 Max	Decode speedhigher is better	≥155.3 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Decode speed
Qwen2.5-1.5B	Apple M1 Max	Decode speedhigher is better	≥136.9 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Decode speed
Qwen2.5-1.5B	Apple M1 Max	Decode speedhigher is better	≥119.6 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Decode speed
Qwen2.5-1.5B	Apple M1 Max	Decode speedhigher is better	≥99.27 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Decode speed
Qwen2.5-1.5B	Apple M1 Max	Decode speedhigher is better	≥91.45 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Decode speed
Qwen2.5-1.5B	Apple M1 Max	Prefill speedhigher is better	≥2,440 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Prefill speed
Qwen2.5-1.5B	Apple M1 Max	Prefill speedhigher is better	≥2,443 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Prefill speed
Qwen2.5-1.5B	Apple M1 Max	Prefill speedhigher is better	≥2,217 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Prefill speed
Qwen2.5-1.5B	Apple M1 Max	Prefill speedhigher is better	≥1,831 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Prefill speed
Qwen2.5-1.5B	Apple M1 Max	Prefill speedhigher is better	≥1,554 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Prefill speed
Qwen2.5-1.5B	Apple M1 Max	Prefill speedhigher is better	≥1,411 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M1 Max, Prefill speed
Qwen2.5-1.5B	Apple M1 Max	Model weight footprintlower is better	926.9 MB	—	Carried buildsingle run	Details for Qwen2.5-1.5B on Apple M1 Max, Model weight footprint
Qwen2.5-1.5B	Apple M2 Pro	Decode speedhigher is better	≥144.6 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Decode speed
Qwen2.5-1.5B	Apple M2 Pro	Decode speedhigher is better	≥113.7 tok/s	3,968 in / 128 out	Carried buildn=11wide spread	Details for Qwen2.5-1.5B on Apple M2 Pro, Decode speed
Qwen2.5-1.5B	Apple M2 Pro	Decode speedhigher is better	≥102.4 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Decode speed
Qwen2.5-1.5B	Apple M2 Pro	Decode speedhigher is better	≥86.31 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Decode speed
Qwen2.5-1.5B	Apple M2 Pro	Decode speedhigher is better	≥74.09 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Decode speed
Qwen2.5-1.5B	Apple M2 Pro	Decode speedhigher is better	≥67.84 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Decode speed
Qwen2.5-1.5B	Apple M2 Pro	Prefill speedhigher is better	≥1,506 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Prefill speed
Qwen2.5-1.5B	Apple M2 Pro	Prefill speedhigher is better	≥1,448 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Prefill speed
Qwen2.5-1.5B	Apple M2 Pro	Prefill speedhigher is better	≥1,273 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Prefill speed
Qwen2.5-1.5B	Apple M2 Pro	Prefill speedhigher is better	≥1,025 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Prefill speed
Qwen2.5-1.5B	Apple M2 Pro	Prefill speedhigher is better	≥854.5 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Prefill speed
Qwen2.5-1.5B	Apple M2 Pro	Prefill speedhigher is better	≥769.6 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-1.5B on Apple M2 Pro, Prefill speed
Qwen2.5-7B	Apple M4 Pro	Decode speedhigher is better	≥54.86 tok/s	512 in / 128 out	Measuredn=11wide spread	Details for Qwen2.5-7B on Apple M4 Pro, Decode speed
Qwen2.5-7B	Apple M4 Pro	Decode speedhigher is better	≥47.89 tok/s	3,968 in / 128 out	Measuredn=10wide spread	Details for Qwen2.5-7B on Apple M4 Pro, Decode speed
Qwen2.5-7B	Apple M4 Pro	Decode speedhigher is better	≥47.36 tok/s	8,192 in / 128 out	Measuredn=6	Details for Qwen2.5-7B on Apple M4 Pro, Decode speed
Qwen2.5-7B	Apple M4 Pro	Decode speedhigher is better	≥42.3 tok/s	16,384 in / 128 out	Measuredn=11	Details for Qwen2.5-7B on Apple M4 Pro, Decode speed
Qwen2.5-7B	Apple M4 Pro	Decode speedhigher is better	≥38.45 tok/s	24,576 in / 128 out	Measuredn=10	Details for Qwen2.5-7B on Apple M4 Pro, Decode speed
Qwen2.5-7B	Apple M4 Pro	Prefill speedhigher is better	≥517.1 tok/s	512 in / 128 out	Measuredn=11wide spread	Details for Qwen2.5-7B on Apple M4 Pro, Prefill speed
Qwen2.5-7B	Apple M4 Pro	Prefill speedhigher is better	≥460.2 tok/s	3,968 in / 128 out	Measuredn=10wide spread	Details for Qwen2.5-7B on Apple M4 Pro, Prefill speed
Qwen2.5-7B	Apple M4 Pro	Prefill speedhigher is better	≥435.5 tok/s	8,192 in / 128 out	Measuredn=6wide spread	Details for Qwen2.5-7B on Apple M4 Pro, Prefill speed
Qwen2.5-7B	Apple M4 Pro	Prefill speedhigher is better	≥417.9 tok/s	16,384 in / 128 out	Measuredn=11	Details for Qwen2.5-7B on Apple M4 Pro, Prefill speed
Qwen2.5-7B	Apple M4 Pro	Prefill speedhigher is better	≥376.7 tok/s	24,576 in / 128 out	Measuredn=10wide spread	Details for Qwen2.5-7B on Apple M4 Pro, Prefill speed
Qwen2.5-7B	Apple M4 Pro	Model weight footprintlower is better	4,114 MB	—	Carried buildsingle run	Details for Qwen2.5-7B on Apple M4 Pro, Model weight footprint
Qwen2.5-7B	Apple M1 Ultra	Decode speedhigher is better	≥104.7 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Decode speed
Qwen2.5-7B	Apple M1 Ultra	Decode speedhigher is better	≥88.25 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Decode speed
Qwen2.5-7B	Apple M1 Ultra	Decode speedhigher is better	≥83.96 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Decode speed
Qwen2.5-7B	Apple M1 Ultra	Decode speedhigher is better	≥72.82 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Decode speed
Qwen2.5-7B	Apple M1 Ultra	Decode speedhigher is better	≥65.66 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Decode speed
Qwen2.5-7B	Apple M1 Ultra	Decode speedhigher is better	≥60.89 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Decode speed
Qwen2.5-7B	Apple M1 Ultra	Prefill speedhigher is better	≥1,009 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Prefill speed
Qwen2.5-7B	Apple M1 Ultra	Prefill speedhigher is better	≥1,038 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Prefill speed
Qwen2.5-7B	Apple M1 Ultra	Prefill speedhigher is better	≥975.9 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Prefill speed
Qwen2.5-7B	Apple M1 Ultra	Prefill speedhigher is better	≥865.2 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Prefill speed
Qwen2.5-7B	Apple M1 Ultra	Prefill speedhigher is better	≥769.2 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Prefill speed
Qwen2.5-7B	Apple M1 Ultra	Prefill speedhigher is better	≥720.1 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Ultra, Prefill speed
Qwen2.5-7B	Apple M1 Ultra	Model weight footprintlower is better	4,114 MB	—	Carried buildsingle run	Details for Qwen2.5-7B on Apple M1 Ultra, Model weight footprint
Qwen2.5-7B	Apple M1 Max	Decode speedhigher is better	≥60.42 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Decode speed
Qwen2.5-7B	Apple M1 Max	Decode speedhigher is better	≥51.67 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Decode speed
Qwen2.5-7B	Apple M1 Max	Decode speedhigher is better	≥50.51 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Decode speed
Qwen2.5-7B	Apple M1 Max	Decode speedhigher is better	≥44.34 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Decode speed
Qwen2.5-7B	Apple M1 Max	Decode speedhigher is better	≥39.42 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Decode speed
Qwen2.5-7B	Apple M1 Max	Decode speedhigher is better	≥36.75 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Decode speed
Qwen2.5-7B	Apple M1 Max	Prefill speedhigher is better	≥545.9 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Prefill speed
Qwen2.5-7B	Apple M1 Max	Prefill speedhigher is better	≥552.0 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Prefill speed
Qwen2.5-7B	Apple M1 Max	Prefill speedhigher is better	≥520.9 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Prefill speed
Qwen2.5-7B	Apple M1 Max	Prefill speedhigher is better	≥465.1 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Prefill speed
Qwen2.5-7B	Apple M1 Max	Prefill speedhigher is better	≥419.2 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Prefill speed
Qwen2.5-7B	Apple M1 Max	Prefill speedhigher is better	≥392.6 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M1 Max, Prefill speed
Qwen2.5-7B	Apple M1 Max	Model weight footprintlower is better	4,114 MB	—	Carried buildsingle run	Details for Qwen2.5-7B on Apple M1 Max, Model weight footprint
Qwen2.5-7B	Apple M2 Pro	Decode speedhigher is better	≥38.28 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Decode speed
Qwen2.5-7B	Apple M2 Pro	Decode speedhigher is better	≥32.96 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Decode speed
Qwen2.5-7B	Apple M2 Pro	Decode speedhigher is better	≥32.33 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Decode speed
Qwen2.5-7B	Apple M2 Pro	Decode speedhigher is better	≥28.51 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Decode speed
Qwen2.5-7B	Apple M2 Pro	Decode speedhigher is better	≥25.27 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Decode speed
Qwen2.5-7B	Apple M2 Pro	Decode speedhigher is better	≥23.53 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Decode speed
Qwen2.5-7B	Apple M2 Pro	Prefill speedhigher is better	≥333.1 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Prefill speed
Qwen2.5-7B	Apple M2 Pro	Prefill speedhigher is better	≥327.2 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Prefill speed
Qwen2.5-7B	Apple M2 Pro	Prefill speedhigher is better	≥306.8 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Prefill speed
Qwen2.5-7B	Apple M2 Pro	Prefill speedhigher is better	≥271.9 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Prefill speed
Qwen2.5-7B	Apple M2 Pro	Prefill speedhigher is better	≥243.6 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Prefill speed
Qwen2.5-7B	Apple M2 Pro	Prefill speedhigher is better	≥227.6 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen2.5-7B on Apple M2 Pro, Prefill speed
Qwen3-1.7B	Apple M4 Pro	Decode speedhigher is better	≥204.3 tok/s	512 in / 128 out	Measuredn=11	Details for Qwen3-1.7B on Apple M4 Pro, Decode speed
Qwen3-1.7B	Apple M4 Pro	Decode speedhigher is better	≥150.1 tok/s	3,968 in / 128 out	Measuredn=10	Details for Qwen3-1.7B on Apple M4 Pro, Decode speed
Qwen3-1.7B	Apple M4 Pro	Decode speedhigher is better	≥111.3 tok/s	8,192 in / 128 out	Measuredn=11	Details for Qwen3-1.7B on Apple M4 Pro, Decode speed
Qwen3-1.7B	Apple M4 Pro	Decode speedhigher is better	≥78.09 tok/s	16,384 in / 128 out	Measuredn=10	Details for Qwen3-1.7B on Apple M4 Pro, Decode speed
Qwen3-1.7B	Apple M4 Pro	Decode speedhigher is better	≥58.06 tok/s	24,576 in / 128 out	Measuredn=11	Details for Qwen3-1.7B on Apple M4 Pro, Decode speed
Qwen3-1.7B	Apple M4 Pro	Decode speedhigher is better	≥50.03 tok/s	30,000 in / 128 out	Measuredn=10wide spread	Details for Qwen3-1.7B on Apple M4 Pro, Decode speed
Qwen3-1.7B	Apple M4 Pro	Prefill speedhigher is better	≥2,161 tok/s	512 in / 128 out	Measuredn=11	Details for Qwen3-1.7B on Apple M4 Pro, Prefill speed
Qwen3-1.7B	Apple M4 Pro	Prefill speedhigher is better	≥1,969 tok/s	3,968 in / 128 out	Measuredn=10wide spread	Details for Qwen3-1.7B on Apple M4 Pro, Prefill speed
Qwen3-1.7B	Apple M4 Pro	Prefill speedhigher is better	≥1,629 tok/s	8,192 in / 128 out	Measuredn=11wide spread	Details for Qwen3-1.7B on Apple M4 Pro, Prefill speed
Qwen3-1.7B	Apple M4 Pro	Prefill speedhigher is better	≥1,346 tok/s	16,384 in / 128 out	Measuredn=10wide spread	Details for Qwen3-1.7B on Apple M4 Pro, Prefill speed
Qwen3-1.7B	Apple M4 Pro	Prefill speedhigher is better	≥1,133 tok/s	24,576 in / 128 out	Measuredn=11wide spread	Details for Qwen3-1.7B on Apple M4 Pro, Prefill speed
Qwen3-1.7B	Apple M4 Pro	Prefill speedhigher is better	≥1,020 tok/s	30,000 in / 128 out	Measuredn=10	Details for Qwen3-1.7B on Apple M4 Pro, Prefill speed
Qwen3-1.7B	Apple M4 Pro	Model weight footprintlower is better	968.0 MB	—	Carried buildsingle run	Details for Qwen3-1.7B on Apple M4 Pro, Model weight footprint
Qwen3-1.7B	Apple M1 Ultra	Decode speedhigher is better	289.8 tok/s	512 in / 128 out	Carried buildn=11wide spread	Details for Qwen3-1.7B on Apple M1 Ultra, Decode speed
Qwen3-1.7B	Apple M1 Ultra	Decode speedhigher is better	252.8 tok/s	3,968 in / 128 out	Carried buildn=11wide spread	Details for Qwen3-1.7B on Apple M1 Ultra, Decode speed
Qwen3-1.7B	Apple M1 Ultra	Decode speedhigher is better	200.7 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Ultra, Decode speed
Qwen3-1.7B	Apple M1 Ultra	Decode speedhigher is better	155.7 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Ultra, Decode speed
Qwen3-1.7B	Apple M1 Ultra	Decode speedhigher is better	127.0 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Ultra, Decode speed
Qwen3-1.7B	Apple M1 Ultra	Decode speedhigher is better	114.3 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Ultra, Decode speed
Qwen3-1.7B	Apple M1 Ultra	Prefill speedhigher is better	4,213 tok/s	512 in / 128 out	Carried buildn=11wide spread	Details for Qwen3-1.7B on Apple M1 Ultra, Prefill speed
Qwen3-1.7B	Apple M1 Ultra	Prefill speedhigher is better	4,171 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Ultra, Prefill speed
Qwen3-1.7B	Apple M1 Ultra	Prefill speedhigher is better	3,732 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Ultra, Prefill speed
Qwen3-1.7B	Apple M1 Ultra	Prefill speedhigher is better	3,001 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Ultra, Prefill speed
Qwen3-1.7B	Apple M1 Ultra	Prefill speedhigher is better	2,503 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Ultra, Prefill speed
Qwen3-1.7B	Apple M1 Ultra	Prefill speedhigher is better	2,235 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Ultra, Prefill speed
Qwen3-1.7B	Apple M1 Ultra	Model weight footprintlower is better	968.0 MB	—	Carried buildsingle run	Details for Qwen3-1.7B on Apple M1 Ultra, Model weight footprint
Qwen3-1.7B	Apple M1 Max	Decode speedhigher is better	207.4 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Decode speed
Qwen3-1.7B	Apple M1 Max	Decode speedhigher is better	163.2 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Decode speed
Qwen3-1.7B	Apple M1 Max	Decode speedhigher is better	127.8 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Decode speed
Qwen3-1.7B	Apple M1 Max	Decode speedhigher is better	96.64 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Decode speed
Qwen3-1.7B	Apple M1 Max	Decode speedhigher is better	76.77 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Decode speed
Qwen3-1.7B	Apple M1 Max	Decode speedhigher is better	67.47 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Decode speed
Qwen3-1.7B	Apple M1 Max	Prefill speedhigher is better	2,417 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Prefill speed
Qwen3-1.7B	Apple M1 Max	Prefill speedhigher is better	2,233 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Prefill speed
Qwen3-1.7B	Apple M1 Max	Prefill speedhigher is better	1,958 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Prefill speed
Qwen3-1.7B	Apple M1 Max	Prefill speedhigher is better	1,562 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Prefill speed
Qwen3-1.7B	Apple M1 Max	Prefill speedhigher is better	1,296 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Prefill speed
Qwen3-1.7B	Apple M1 Max	Prefill speedhigher is better	1,156 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M1 Max, Prefill speed
Qwen3-1.7B	Apple M1 Max	Model weight footprintlower is better	968.0 MB	—	Carried buildsingle run	Details for Qwen3-1.7B on Apple M1 Max, Model weight footprint
Qwen3-1.7B	Apple M2 Pro	Decode speedhigher is better	143.8 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Decode speed
Qwen3-1.7B	Apple M2 Pro	Decode speedhigher is better	106.6 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Decode speed
Qwen3-1.7B	Apple M2 Pro	Decode speedhigher is better	83.93 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Decode speed
Qwen3-1.7B	Apple M2 Pro	Decode speedhigher is better	60.08 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Decode speed
Qwen3-1.7B	Apple M2 Pro	Decode speedhigher is better	46.36 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Decode speed
Qwen3-1.7B	Apple M2 Pro	Decode speedhigher is better	40.26 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Decode speed
Qwen3-1.7B	Apple M2 Pro	Prefill speedhigher is better	1,459 tok/s	512 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Prefill speed
Qwen3-1.7B	Apple M2 Pro	Prefill speedhigher is better	1,306 tok/s	3,968 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Prefill speed
Qwen3-1.7B	Apple M2 Pro	Prefill speedhigher is better	1,112 tok/s	8,192 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Prefill speed
Qwen3-1.7B	Apple M2 Pro	Prefill speedhigher is better	880.2 tok/s	16,384 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Prefill speed
Qwen3-1.7B	Apple M2 Pro	Prefill speedhigher is better	727.0 tok/s	24,576 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Prefill speed
Qwen3-1.7B	Apple M2 Pro	Prefill speedhigher is better	651.1 tok/s	30,000 in / 128 out	Carried buildn=11	Details for Qwen3-1.7B on Apple M2 Pro, Prefill speed
Qwen3.5-0.8B	RTX 3090	Decode speedhigher is better	1,052 tok/s	20 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX 3090, Decode speed
Qwen3.5-0.8B	RTX 3090	Decode speedhigher is better	967.5 tok/s	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 3090, Decode speed
Qwen3.5-0.8B	RTX 3090	Decode speedhigher is better	972.0 tok/s	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 3090, Decode speed
Qwen3.5-0.8B	RTX 3090	Decode speedhigher is better	880.8 tok/s	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 3090, Decode speed
Qwen3.5-0.8B	RTX 3090	Server throughputhigher is better	531.9 tok/s	36 in / 128 out · 1 request at a time · balanced	Carried buildn=3	Details for Qwen3.5-0.8B on RTX 3090, Server throughput
Qwen3.5-0.8B	RTX 3090	Server throughputhigher is better	1,778 tok/s	36 in / 128 out · 4 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.5-0.8B on RTX 3090, Server throughput
Qwen3.5-0.8B	RTX 3090	Server throughputhigher is better	3,125 tok/s	36 in / 128 out · 8 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.5-0.8B on RTX 3090, Server throughput
Qwen3.5-0.8B	RTX 3090	Time to first tokenlower is better	24.6 ms	36 in / 128 out · 1 request at a time · balanced	Carried buildn=3wide spread	Details for Qwen3.5-0.8B on RTX 3090, Time to first token
Qwen3.5-0.8B	RTX 3090	Time to first tokenlower is better	62.9 ms	36 in / 128 out · 4 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.5-0.8B on RTX 3090, Time to first token
Qwen3.5-0.8B	RTX 3090	Time to first tokenlower is better	92.6 ms	36 in / 128 out · 8 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.5-0.8B on RTX 3090, Time to first token
Qwen3.5-0.8B	RTX 3090	Prompt pass timelower is better	4.18 ms	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 3090, Prompt pass time
Qwen3.5-0.8B	RTX 3090	Prompt pass timelower is better	4.09 ms	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 3090, Prompt pass time
Qwen3.5-0.8B	RTX 3090	Prompt pass timelower is better	4.09 ms	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 3090, Prompt pass time
Qwen3.5-0.8B	RTX 3090	Prompt pass timelower is better	4.38 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX 3090, Prompt pass time
Qwen3.5-0.8B	RTX 3090	GPU memory in uselower is better	2.27 GiB	balanced	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX 3090, GPU memory in use
Qwen3.5-0.8B	RTX 3090	GPU memory in uselower is better	2.27 GiB	fast	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX 3090, GPU memory in use
Qwen3.5-0.8B	RTX 3090	GPU memory in uselower is better	2.42 GiB	quality	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX 3090, GPU memory in use
Qwen3.5-0.8B	RTX 4090	Decode speedhigher is better	1,439 tok/s	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX 4090, Decode speed
Qwen3.5-0.8B	RTX 4090	Decode speedhigher is better	1,318 tok/s	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 4090, Decode speed
Qwen3.5-0.8B	RTX 4090	Decode speedhigher is better	1,316 tok/s	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 4090, Decode speed
Qwen3.5-0.8B	RTX 4090	Decode speedhigher is better	1,156 tok/s	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 4090, Decode speed
Qwen3.5-0.8B	RTX 4090	Prompt pass timelower is better	3.01 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX 4090, Prompt pass time
Qwen3.5-0.8B	RTX 4090	Prompt pass timelower is better	3 ms	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 4090, Prompt pass time
Qwen3.5-0.8B	RTX 4090	Prompt pass timelower is better	3.01 ms	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 4090, Prompt pass time
Qwen3.5-0.8B	RTX 4090	Prompt pass timelower is better	3.11 ms	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 4090, Prompt pass time
Qwen3.5-0.8B	RTX 4090	GPU memory in uselower is better	2.4 GiB	balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 4090, GPU memory in use
Qwen3.5-0.8B	RTX 4090	GPU memory in uselower is better	2.4 GiB	fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 4090, GPU memory in use
Qwen3.5-0.8B	RTX 4090	GPU memory in uselower is better	2.55 GiB	quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 4090, GPU memory in use
Qwen3.5-0.8B	RTX 5090	Decode speedhigher is better	1,676 tok/s	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX 5090, Decode speed
Qwen3.5-0.8B	RTX 5090	Decode speedhigher is better	1,529 tok/s	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 5090, Decode speed
Qwen3.5-0.8B	RTX 5090	Decode speedhigher is better	1,531 tok/s	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 5090, Decode speed
Qwen3.5-0.8B	RTX 5090	Decode speedhigher is better	1,438 tok/s	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 5090, Decode speed
Qwen3.5-0.8B	RTX 5090	Prompt pass timelower is better	2.75 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX 5090, Prompt pass time
Qwen3.5-0.8B	RTX 5090	Prompt pass timelower is better	2.74 ms	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 5090, Prompt pass time
Qwen3.5-0.8B	RTX 5090	Prompt pass timelower is better	2.74 ms	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 5090, Prompt pass time
Qwen3.5-0.8B	RTX 5090	Prompt pass timelower is better	2.94 ms	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 5090, Prompt pass time
Qwen3.5-0.8B	RTX 5090	GPU memory in uselower is better	2.51 GiB	balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 5090, GPU memory in use
Qwen3.5-0.8B	RTX 5090	GPU memory in uselower is better	2.51 GiB	fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 5090, GPU memory in use
Qwen3.5-0.8B	RTX 5090	GPU memory in uselower is better	2.66 GiB	quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX 5090, GPU memory in use
Qwen3.5-0.8B	RTX A5000	Decode speedhigher is better	776.7 tok/s	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX A5000, Decode speed
Qwen3.5-0.8B	RTX A5000	Decode speedhigher is better	771.9 tok/s	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX A5000, Decode speed
Qwen3.5-0.8B	RTX A5000	Decode speedhigher is better	714.7 tok/s	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX A5000, Decode speed
Qwen3.5-0.8B	RTX A5000	Decode speedhigher is better	916.1 tok/s	921 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX A5000, Decode speed
Qwen3.5-0.8B	RTX A5000	Prompt pass timelower is better	4.86 ms	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX A5000, Prompt pass time
Qwen3.5-0.8B	RTX A5000	Prompt pass timelower is better	4.89 ms	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX A5000, Prompt pass time
Qwen3.5-0.8B	RTX A5000	Prompt pass timelower is better	4.65 ms	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX A5000, Prompt pass time
Qwen3.5-0.8B	RTX A5000	Prompt pass timelower is better	28.57 ms	921 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-0.8B on RTX A5000, Prompt pass time
Qwen3.5-0.8B	RTX A5000	GPU memory in uselower is better	2.22 GiB	balanced	Carried buildn=2	Details for Qwen3.5-0.8B on RTX A5000, GPU memory in use
Qwen3.5-0.8B	RTX A5000	GPU memory in uselower is better	2.22 GiB	fast	Carried buildn=2	Details for Qwen3.5-0.8B on RTX A5000, GPU memory in use
Qwen3.5-0.8B	RTX A5000	GPU memory in uselower is better	2.37 GiB	quality	Carried buildn=2	Details for Qwen3.5-0.8B on RTX A5000, GPU memory in use
Qwen3.5-2B	RTX 3090	Decode speedhigher is better	560.4 tok/s	20 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-2B on RTX 3090, Decode speed
Qwen3.5-2B	RTX 3090	Decode speedhigher is better	586.5 tok/s	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-2B on RTX 3090, Decode speed
Qwen3.5-2B	RTX 3090	Decode speedhigher is better	585.1 tok/s	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-2B on RTX 3090, Decode speed
Qwen3.5-2B	RTX 3090	Decode speedhigher is better	542.7 tok/s	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-2B on RTX 3090, Decode speed
Qwen3.5-2B	RTX 3090	Prompt pass timelower is better	6.63 ms	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-2B on RTX 3090, Prompt pass time
Qwen3.5-2B	RTX 3090	Prompt pass timelower is better	6.67 ms	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-2B on RTX 3090, Prompt pass time
Qwen3.5-2B	RTX 3090	Prompt pass timelower is better	6.9 ms	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-2B on RTX 3090, Prompt pass time
Qwen3.5-2B	RTX 3090	Prompt pass timelower is better	7.12 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-2B on RTX 3090, Prompt pass time
Qwen3.5-2B	RTX 3090	GPU memory in uselower is better	3.48 GiB	balanced	Carried buildsingle run	Details for Qwen3.5-2B on RTX 3090, GPU memory in use
Qwen3.5-2B	RTX 3090	GPU memory in uselower is better	3.48 GiB	fast	Carried buildsingle run	Details for Qwen3.5-2B on RTX 3090, GPU memory in use
Qwen3.5-2B	RTX 3090	GPU memory in uselower is better	3.8 GiB	quality	Carried buildsingle run	Details for Qwen3.5-2B on RTX 3090, GPU memory in use
Qwen3.5-2B	RTX 4090	Decode speedhigher is better	868.8 tok/s	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-2B on RTX 4090, Decode speed
Qwen3.5-2B	RTX 4090	Decode speedhigher is better	886.9 tok/s	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-2B on RTX 4090, Decode speed
Qwen3.5-2B	RTX 4090	Decode speedhigher is better	895.1 tok/s	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-2B on RTX 4090, Decode speed
Qwen3.5-2B	RTX 4090	Decode speedhigher is better	740.9 tok/s	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-2B on RTX 4090, Decode speed
Qwen3.5-2B	RTX 4090	Prompt pass timelower is better	4.38 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-2B on RTX 4090, Prompt pass time
Qwen3.5-2B	RTX 4090	Prompt pass timelower is better	4.49 ms	21 in / 128 out · balanced	Carried buildn=2wide spread	Details for Qwen3.5-2B on RTX 4090, Prompt pass time
Qwen3.5-2B	RTX 4090	Prompt pass timelower is better	4.37 ms	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-2B on RTX 4090, Prompt pass time
Qwen3.5-2B	RTX 4090	Prompt pass timelower is better	4.69 ms	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-2B on RTX 4090, Prompt pass time
Qwen3.5-2B	RTX 4090	GPU memory in uselower is better	3.61 GiB	balanced	Carried buildn=2	Details for Qwen3.5-2B on RTX 4090, GPU memory in use
Qwen3.5-2B	RTX 4090	GPU memory in uselower is better	3.61 GiB	fast	Carried buildn=2	Details for Qwen3.5-2B on RTX 4090, GPU memory in use
Qwen3.5-2B	RTX 4090	GPU memory in uselower is better	3.93 GiB	quality	Carried buildn=2	Details for Qwen3.5-2B on RTX 4090, GPU memory in use
Qwen3.5-2B	RTX 5090	Decode speedhigher is better	1,107 tok/s	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-2B on RTX 5090, Decode speed
Qwen3.5-2B	RTX 5090	Decode speedhigher is better	1,138 tok/s	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-2B on RTX 5090, Decode speed
Qwen3.5-2B	RTX 5090	Decode speedhigher is better	1,137 tok/s	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-2B on RTX 5090, Decode speed
Qwen3.5-2B	RTX 5090	Decode speedhigher is better	1,041 tok/s	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-2B on RTX 5090, Decode speed
Qwen3.5-2B	RTX 5090	Prompt pass timelower is better	3.79 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-2B on RTX 5090, Prompt pass time
Qwen3.5-2B	RTX 5090	Prompt pass timelower is better	3.78 ms	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-2B on RTX 5090, Prompt pass time
Qwen3.5-2B	RTX 5090	Prompt pass timelower is better	3.83 ms	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-2B on RTX 5090, Prompt pass time
Qwen3.5-2B	RTX 5090	Prompt pass timelower is better	4.11 ms	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-2B on RTX 5090, Prompt pass time
Qwen3.5-2B	RTX 5090	GPU memory in uselower is better	3.72 GiB	balanced	Carried buildn=2	Details for Qwen3.5-2B on RTX 5090, GPU memory in use
Qwen3.5-2B	RTX 5090	GPU memory in uselower is better	3.72 GiB	fast	Carried buildn=2	Details for Qwen3.5-2B on RTX 5090, GPU memory in use
Qwen3.5-2B	RTX 5090	GPU memory in uselower is better	4.03 GiB	quality	Carried buildn=2	Details for Qwen3.5-2B on RTX 5090, GPU memory in use
Qwen3.5-2B	RTX A5000	Decode speedhigher is better	467.3 tok/s	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-2B on RTX A5000, Decode speed
Qwen3.5-2B	RTX A5000	Decode speedhigher is better	464.6 tok/s	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-2B on RTX A5000, Decode speed
Qwen3.5-2B	RTX A5000	Decode speedhigher is better	435.2 tok/s	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-2B on RTX A5000, Decode speed
Qwen3.5-2B	RTX A5000	Decode speedhigher is better	449.1 tok/s	921 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-2B on RTX A5000, Decode speed
Qwen3.5-2B	RTX A5000	Prompt pass timelower is better	8.39 ms	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-2B on RTX A5000, Prompt pass time
Qwen3.5-2B	RTX A5000	Prompt pass timelower is better	8.41 ms	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-2B on RTX A5000, Prompt pass time
Qwen3.5-2B	RTX A5000	Prompt pass timelower is better	9.02 ms	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-2B on RTX A5000, Prompt pass time
Qwen3.5-2B	RTX A5000	Prompt pass timelower is better	58.49 ms	921 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-2B on RTX A5000, Prompt pass time
Qwen3.5-2B	RTX A5000	GPU memory in uselower is better	3.43 GiB	balanced	Carried buildn=2	Details for Qwen3.5-2B on RTX A5000, GPU memory in use
Qwen3.5-2B	RTX A5000	GPU memory in uselower is better	3.43 GiB	fast	Carried buildn=2	Details for Qwen3.5-2B on RTX A5000, GPU memory in use
Qwen3.5-2B	RTX A5000	GPU memory in uselower is better	3.75 GiB	quality	Carried buildn=2	Details for Qwen3.5-2B on RTX A5000, GPU memory in use
Qwen3.5-4B	RTX 3090	Decode speedhigher is better	397.8 tok/s	20 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-4B on RTX 3090, Decode speed
Qwen3.5-4B	RTX 3090	Decode speedhigher is better	376.8 tok/s	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX 3090, Decode speed
Qwen3.5-4B	RTX 3090	Decode speedhigher is better	376.8 tok/s	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-4B on RTX 3090, Decode speed
Qwen3.5-4B	RTX 3090	Decode speedhigher is better	342.1 tok/s	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-4B on RTX 3090, Decode speed
Qwen3.5-4B	RTX 3090	Prompt pass timelower is better	13.51 ms	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX 3090, Prompt pass time
Qwen3.5-4B	RTX 3090	Prompt pass timelower is better	13.11 ms	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-4B on RTX 3090, Prompt pass time
Qwen3.5-4B	RTX 3090	Prompt pass timelower is better	14.03 ms	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-4B on RTX 3090, Prompt pass time
Qwen3.5-4B	RTX 3090	Prompt pass timelower is better	13.9 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-4B on RTX 3090, Prompt pass time
Qwen3.5-4B	RTX 3090	GPU memory in uselower is better	5.49 GiB	balanced	Carried buildsingle run	Details for Qwen3.5-4B on RTX 3090, GPU memory in use
Qwen3.5-4B	RTX 3090	GPU memory in uselower is better	5.49 GiB	fast	Carried buildsingle run	Details for Qwen3.5-4B on RTX 3090, GPU memory in use
Qwen3.5-4B	RTX 3090	GPU memory in uselower is better	6.27 GiB	quality	Carried buildsingle run	Details for Qwen3.5-4B on RTX 3090, GPU memory in use
Qwen3.5-4B	RTX 4090	Decode speedhigher is better	568.2 tok/s	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-4B on RTX 4090, Decode speed
Qwen3.5-4B	RTX 4090	Decode speedhigher is better	531.8 tok/s	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX 4090, Decode speed
Qwen3.5-4B	RTX 4090	Decode speedhigher is better	535.4 tok/s	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-4B on RTX 4090, Decode speed
Qwen3.5-4B	RTX 4090	Decode speedhigher is better	441.8 tok/s	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-4B on RTX 4090, Decode speed
Qwen3.5-4B	RTX 4090	Prompt pass timelower is better	8.6 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-4B on RTX 4090, Prompt pass time
Qwen3.5-4B	RTX 4090	Prompt pass timelower is better	8.73 ms	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX 4090, Prompt pass time
Qwen3.5-4B	RTX 4090	Prompt pass timelower is better	8.68 ms	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-4B on RTX 4090, Prompt pass time
Qwen3.5-4B	RTX 4090	Prompt pass timelower is better	10.54 ms	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-4B on RTX 4090, Prompt pass time
Qwen3.5-4B	RTX 4090	GPU memory in uselower is better	5.62 GiB	balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX 4090, GPU memory in use
Qwen3.5-4B	RTX 4090	GPU memory in uselower is better	5.62 GiB	fast	Carried buildn=2	Details for Qwen3.5-4B on RTX 4090, GPU memory in use
Qwen3.5-4B	RTX 4090	GPU memory in uselower is better	6.4 GiB	quality	Carried buildn=2	Details for Qwen3.5-4B on RTX 4090, GPU memory in use
Qwen3.5-4B	RTX 5090	Decode speedhigher is better	822.2 tok/s	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-4B on RTX 5090, Decode speed
Qwen3.5-4B	RTX 5090	Decode speedhigher is better	772.2 tok/s	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX 5090, Decode speed
Qwen3.5-4B	RTX 5090	Decode speedhigher is better	771.4 tok/s	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-4B on RTX 5090, Decode speed
Qwen3.5-4B	RTX 5090	Decode speedhigher is better	670.5 tok/s	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-4B on RTX 5090, Decode speed
Qwen3.5-4B	RTX 5090	Prompt pass timelower is better	6.91 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-4B on RTX 5090, Prompt pass time
Qwen3.5-4B	RTX 5090	Prompt pass timelower is better	6.88 ms	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX 5090, Prompt pass time
Qwen3.5-4B	RTX 5090	Prompt pass timelower is better	6.92 ms	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-4B on RTX 5090, Prompt pass time
Qwen3.5-4B	RTX 5090	Prompt pass timelower is better	7.9 ms	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-4B on RTX 5090, Prompt pass time
Qwen3.5-4B	RTX 5090	GPU memory in uselower is better	5.73 GiB	balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX 5090, GPU memory in use
Qwen3.5-4B	RTX 5090	GPU memory in uselower is better	5.73 GiB	fast	Carried buildn=2	Details for Qwen3.5-4B on RTX 5090, GPU memory in use
Qwen3.5-4B	RTX 5090	GPU memory in uselower is better	6.5 GiB	quality	Carried buildn=2	Details for Qwen3.5-4B on RTX 5090, GPU memory in use
Qwen3.5-4B	RTX A5000	Decode speedhigher is better	280.5 tok/s	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX A5000, Decode speed
Qwen3.5-4B	RTX A5000	Decode speedhigher is better	278.9 tok/s	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-4B on RTX A5000, Decode speed
Qwen3.5-4B	RTX A5000	Decode speedhigher is better	256.5 tok/s	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-4B on RTX A5000, Decode speed
Qwen3.5-4B	RTX A5000	Decode speedhigher is better	228.6 tok/s	921 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-4B on RTX A5000, Decode speed
Qwen3.5-4B	RTX A5000	Prompt pass timelower is better	17.22 ms	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX A5000, Prompt pass time
Qwen3.5-4B	RTX A5000	Prompt pass timelower is better	17.21 ms	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-4B on RTX A5000, Prompt pass time
Qwen3.5-4B	RTX A5000	Prompt pass timelower is better	19.18 ms	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-4B on RTX A5000, Prompt pass time
Qwen3.5-4B	RTX A5000	Prompt pass timelower is better	154.0 ms	921 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-4B on RTX A5000, Prompt pass time
Qwen3.5-4B	RTX A5000	GPU memory in uselower is better	5.44 GiB	balanced	Carried buildn=2	Details for Qwen3.5-4B on RTX A5000, GPU memory in use
Qwen3.5-4B	RTX A5000	GPU memory in uselower is better	5.44 GiB	fast	Carried buildn=2	Details for Qwen3.5-4B on RTX A5000, GPU memory in use
Qwen3.5-4B	RTX A5000	GPU memory in uselower is better	6.22 GiB	quality	Carried buildn=2	Details for Qwen3.5-4B on RTX A5000, GPU memory in use
Qwen3.5-9B	RTX 3090	Decode speedhigher is better	170.2 tok/s	20 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-9B on RTX 3090, Decode speed
Qwen3.5-9B	RTX 3090	Decode speedhigher is better	190.2 tok/s	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX 3090, Decode speed
Qwen3.5-9B	RTX 3090	Decode speedhigher is better	190.4 tok/s	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-9B on RTX 3090, Decode speed
Qwen3.5-9B	RTX 3090	Decode speedhigher is better	172.8 tok/s	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-9B on RTX 3090, Decode speed
Qwen3.5-9B	RTX 3090	Server throughputhigher is better	101.4 tok/s	34 in / 128 out · 1 request at a time · balanced	Carried buildn=3	Details for Qwen3.5-9B on RTX 3090, Server throughput
Qwen3.5-9B	RTX 3090	Server throughputhigher is better	366.4 tok/s	34 in / 128 out · 4 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.5-9B on RTX 3090, Server throughput
Qwen3.5-9B	RTX 3090	Server throughputhigher is better	663.0 tok/s	34 in / 128 out · 8 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.5-9B on RTX 3090, Server throughput
Qwen3.5-9B	RTX 3090	Time to first tokenlower is better	55.6 ms	34 in / 128 out · 1 request at a time · balanced	Carried buildn=3	Details for Qwen3.5-9B on RTX 3090, Time to first token
Qwen3.5-9B	RTX 3090	Time to first tokenlower is better	176.2 ms	34 in / 128 out · 4 concurrent requests · balanced	Carried buildn=3wide spread	Details for Qwen3.5-9B on RTX 3090, Time to first token
Qwen3.5-9B	RTX 3090	Time to first tokenlower is better	308.8 ms	34 in / 128 out · 8 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.5-9B on RTX 3090, Time to first token
Qwen3.5-9B	RTX 3090	Prompt pass timelower is better	21.72 ms	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX 3090, Prompt pass time
Qwen3.5-9B	RTX 3090	Prompt pass timelower is better	21.77 ms	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-9B on RTX 3090, Prompt pass time
Qwen3.5-9B	RTX 3090	Prompt pass timelower is better	23.25 ms	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-9B on RTX 3090, Prompt pass time
Qwen3.5-9B	RTX 3090	Prompt pass timelower is better	22.42 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-9B on RTX 3090, Prompt pass time
Qwen3.5-9B	RTX 3090	GPU memory in uselower is better	8.44 GiB	balanced	Carried buildsingle run	Details for Qwen3.5-9B on RTX 3090, GPU memory in use
Qwen3.5-9B	RTX 3090	GPU memory in uselower is better	8.44 GiB	fast	Carried buildsingle run	Details for Qwen3.5-9B on RTX 3090, GPU memory in use
Qwen3.5-9B	RTX 3090	GPU memory in uselower is better	9.82 GiB	quality	Carried buildsingle run	Details for Qwen3.5-9B on RTX 3090, GPU memory in use
Qwen3.5-9B	RTX 4090	Decode speedhigher is better	303.4 tok/s	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-9B on RTX 4090, Decode speed
Qwen3.5-9B	RTX 4090	Decode speedhigher is better	347.0 tok/s	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX 4090, Decode speed
Qwen3.5-9B	RTX 4090	Decode speedhigher is better	346.6 tok/s	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-9B on RTX 4090, Decode speed
Qwen3.5-9B	RTX 4090	Decode speedhigher is better	272.8 tok/s	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-9B on RTX 4090, Decode speed
Qwen3.5-9B	RTX 4090	Prompt pass timelower is better	16.62 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-9B on RTX 4090, Prompt pass time
Qwen3.5-9B	RTX 4090	Prompt pass timelower is better	16.82 ms	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX 4090, Prompt pass time
Qwen3.5-9B	RTX 4090	Prompt pass timelower is better	16.47 ms	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-9B on RTX 4090, Prompt pass time
Qwen3.5-9B	RTX 4090	Prompt pass timelower is better	21.62 ms	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-9B on RTX 4090, Prompt pass time
Qwen3.5-9B	RTX 4090	GPU memory in uselower is better	8.57 GiB	balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX 4090, GPU memory in use
Qwen3.5-9B	RTX 4090	GPU memory in uselower is better	8.57 GiB	fast	Carried buildn=2	Details for Qwen3.5-9B on RTX 4090, GPU memory in use
Qwen3.5-9B	RTX 4090	GPU memory in uselower is better	9.95 GiB	quality	Carried buildn=2	Details for Qwen3.5-9B on RTX 4090, GPU memory in use
Qwen3.5-9B	RTX 5090	Decode speedhigher is better	434.6 tok/s	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-9B on RTX 5090, Decode speed
Qwen3.5-9B	RTX 5090	Decode speedhigher is better	498.7 tok/s	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX 5090, Decode speed
Qwen3.5-9B	RTX 5090	Decode speedhigher is better	498.5 tok/s	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-9B on RTX 5090, Decode speed
Qwen3.5-9B	RTX 5090	Decode speedhigher is better	423.4 tok/s	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-9B on RTX 5090, Decode speed
Qwen3.5-9B	RTX 5090	Prompt pass timelower is better	11.12 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-9B on RTX 5090, Prompt pass time
Qwen3.5-9B	RTX 5090	Prompt pass timelower is better	11.06 ms	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX 5090, Prompt pass time
Qwen3.5-9B	RTX 5090	Prompt pass timelower is better	11.09 ms	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-9B on RTX 5090, Prompt pass time
Qwen3.5-9B	RTX 5090	Prompt pass timelower is better	13.02 ms	21 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-9B on RTX 5090, Prompt pass time
Qwen3.5-9B	RTX 5090	GPU memory in uselower is better	8.67 GiB	balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX 5090, GPU memory in use
Qwen3.5-9B	RTX 5090	GPU memory in uselower is better	8.67 GiB	fast	Carried buildn=2	Details for Qwen3.5-9B on RTX 5090, GPU memory in use
Qwen3.5-9B	RTX 5090	GPU memory in uselower is better	10.05 GiB	quality	Carried buildn=2	Details for Qwen3.5-9B on RTX 5090, GPU memory in use
Qwen3.5-9B	RTX A5000	Decode speedhigher is better	151.0 tok/s	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX A5000, Decode speed
Qwen3.5-9B	RTX A5000	Decode speedhigher is better	149.1 tok/s	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-9B on RTX A5000, Decode speed
Qwen3.5-9B	RTX A5000	Decode speedhigher is better	137.8 tok/s	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-9B on RTX A5000, Decode speed
Qwen3.5-9B	RTX A5000	Decode speedhigher is better	124.8 tok/s	921 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-9B on RTX A5000, Decode speed
Qwen3.5-9B	RTX A5000	Prompt pass timelower is better	28.99 ms	20 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX A5000, Prompt pass time
Qwen3.5-9B	RTX A5000	Prompt pass timelower is better	29.21 ms	20 in / 128 out · fast	Carried buildn=2	Details for Qwen3.5-9B on RTX A5000, Prompt pass time
Qwen3.5-9B	RTX A5000	Prompt pass timelower is better	30.95 ms	20 in / 128 out · quality	Carried buildn=2	Details for Qwen3.5-9B on RTX A5000, Prompt pass time
Qwen3.5-9B	RTX A5000	Prompt pass timelower is better	269.9 ms	921 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.5-9B on RTX A5000, Prompt pass time
Qwen3.5-9B	RTX A5000	GPU memory in uselower is better	8.39 GiB	balanced	Carried buildn=2	Details for Qwen3.5-9B on RTX A5000, GPU memory in use
Qwen3.5-9B	RTX A5000	GPU memory in uselower is better	8.39 GiB	fast	Carried buildn=2	Details for Qwen3.5-9B on RTX A5000, GPU memory in use
Qwen3.5-9B	RTX A5000	GPU memory in uselower is better	9.77 GiB	quality	Carried buildn=2	Details for Qwen3.5-9B on RTX A5000, GPU memory in use
Qwen3.6-35B-A3B	RTX 3090	Decode speedhigher is better	209.7 tok/s	22 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.6-35B-A3B on RTX 3090, Decode speed
Qwen3.6-35B-A3B	RTX 3090	Decode speedhigher is better	194.3 tok/s	22 in / 128 out · balanced	Carried buildn=3	Details for Qwen3.6-35B-A3B on RTX 3090, Decode speed
Qwen3.6-35B-A3B	RTX 3090	Decode speedhigher is better	172.7 tok/s	22 in / 128 out · fast	Carried buildn=2	Details for Qwen3.6-35B-A3B on RTX 3090, Decode speed
Qwen3.6-35B-A3B	RTX 3090	Decode speedhigher is better	197.9 tok/s	22 in / 128 out · single-token decoding	Carried buildn=3	Details for Qwen3.6-35B-A3B on RTX 3090, Decode speed
Qwen3.6-35B-A3B	RTX 3090	Server throughputhigher is better	160.9 tok/s	34 in / 128 out · 1 request at a time · balanced	Carried buildn=3	Details for Qwen3.6-35B-A3B on RTX 3090, Server throughput
Qwen3.6-35B-A3B	RTX 3090	Server throughputhigher is better	222.4 tok/s	34 in / 128 out · 4 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.6-35B-A3B on RTX 3090, Server throughput
Qwen3.6-35B-A3B	RTX 3090	Server throughputhigher is better	230.6 tok/s	34 in / 128 out · 8 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.6-35B-A3B on RTX 3090, Server throughput
Qwen3.6-35B-A3B	RTX 3090	Time to first tokenlower is better	182.7 ms	34 in / 128 out · 1 request at a time · balanced	Carried buildn=3	Details for Qwen3.6-35B-A3B on RTX 3090, Time to first token
Qwen3.6-35B-A3B	RTX 3090	Time to first tokenlower is better	625.5 ms	34 in / 128 out · 4 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.6-35B-A3B on RTX 3090, Time to first token
Qwen3.6-35B-A3B	RTX 3090	Time to first tokenlower is better	1,001 ms	34 in / 128 out · 8 concurrent requests · balanced	Carried buildn=3	Details for Qwen3.6-35B-A3B on RTX 3090, Time to first token
Qwen3.6-35B-A3B	RTX 3090	Prompt pass timelower is better	107.2 ms	22 in / 128 out · balanced	Carried buildn=3	Details for Qwen3.6-35B-A3B on RTX 3090, Prompt pass time
Qwen3.6-35B-A3B	RTX 3090	Prompt pass timelower is better	107.6 ms	22 in / 128 out · fast	Carried buildn=2	Details for Qwen3.6-35B-A3B on RTX 3090, Prompt pass time
Qwen3.6-35B-A3B	RTX 3090	GPU memory in uselower is better	21.44 GiB	balanced	Carried buildsingle run	Details for Qwen3.6-35B-A3B on RTX 3090, GPU memory in use
Qwen3.6-35B-A3B	RTX 3090	GPU memory in uselower is better	21.51 GiB	fast	Carried buildsingle run	Details for Qwen3.6-35B-A3B on RTX 3090, GPU memory in use
Qwen3.6-35B-A3B	RTX 5090	Decode speedhigher is better	449.6 tok/s	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.6-35B-A3B on RTX 5090, Decode speed
Qwen3.6-35B-A3B	RTX 5090	Decode speedhigher is better	449.9 tok/s	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.6-35B-A3B on RTX 5090, Decode speed
Qwen3.6-35B-A3B	RTX 5090	Decode speedhigher is better	544.7 tok/s	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.6-35B-A3B on RTX 5090, Decode speed
Qwen3.6-35B-A3B	RTX 5090	Prompt pass timelower is better	61.48 ms	21 in / 1,024 out · balanced	Carried buildsingle run	Details for Qwen3.6-35B-A3B on RTX 5090, Prompt pass time
Qwen3.6-35B-A3B	RTX 5090	Prompt pass timelower is better	61.25 ms	21 in / 128 out · balanced	Carried buildn=2	Details for Qwen3.6-35B-A3B on RTX 5090, Prompt pass time
Qwen3.6-35B-A3B	RTX 5090	Prompt pass timelower is better	61.27 ms	21 in / 128 out · fast	Carried buildn=2	Details for Qwen3.6-35B-A3B on RTX 5090, Prompt pass time
Qwen3.6-35B-A3B	RTX 5090	GPU memory in uselower is better	21.67 GiB	balanced	Carried buildn=2	Details for Qwen3.6-35B-A3B on RTX 5090, GPU memory in use
Qwen3.6-35B-A3B	RTX 5090	GPU memory in uselower is better	21.74 GiB	fast	Carried buildn=2	Details for Qwen3.6-35B-A3B on RTX 5090, GPU memory in use

Published elsewhere · conditions differReference

This explorer lists Veizik’s own measurements only. Figures other teams or engines have published are measured under their own conditions, so they are not placed beside ours, and no ratio between them and ours is computed here.

To compare, run both on your hardware with the same prompts and settings. The reproduction guide gives the matched setup for each engine.

Powered by Stability AI Weights are not included. Model availability and use are governed by the model provider's license and regional restrictions. Weights are not included. Model availability and use are governed by the model provider's license and regional restrictions.