Benchmarks

Reproduce our benchmarks.

Run our published measurement and the matching setup of the engine we compared it against, on your own hardware. We publish only our own figures. Your run produces the comparison.

How to use this page

Same machine, same settings, each engine on its own clock.

Each section gives our recipe and the other engine’s recipe: its name, version, install command, model revision, precision and settings, and how its run was timed.
Run both on one machine, with nothing else on the GPU, at the same power limit and driver, with clocks at their defaults. Use the same prompt, resolution, steps or tokens, and seed, and the same warm-up and number of runs.
Veizik publishes only its own figures. We do not publish the other engine’s numbers, and we make no claim about them: the figures you get on your hardware are the comparison.
Our figures, their run counts, ranges and conditions are on the benchmarks page.

NVIDIA · CUDA

Image and video, and language models.

Stable Diffusion 3.5 Medium

Measured on: RTX 3090, RTX 4090, RTX 5090, RTX A5000, A100-SXM4-80GB. Our figures for these GPUs are on the benchmarks page.

Veizik

Build: A development build of the SD3.5 engine, named with each figure on the benchmarks page. It is not distributed.
Note: These figures were measured on a development build, and no Veizik download for SD3.5 is offered at the moment, so our side cannot be rerun yet. The operating point below is the one our figures used.
Operating point: Prompt “A red colored car.” · Resolution 1024x1024 · Steps 40 · Seed 42
Mode: quality, balanced, fast, ultrafast and low-memory; each row names its mode
Warm-up and runs: 3 runs per mode for time per step and peak memory; 1 run, from a fresh start, for the finished image
What is timed: Per step: the engine's own time per denoising step
Finished image: first image after a fresh start of the command: process start, text encoding, denoising, decoding and writing the PNG
Peak memory: peak device memory over the whole render, sampled with nvidia-smi
Hardware: Dedicated single-GPU cloud host; the table below gives the GPU, power limit and driver of each of our runs.

diffusers

Engine: diffusers 0.39.0
Also installed: PyTorch 2.8.0+cu128, CUDA runtime 12.8, Python 3.11.10, transformers 5.14.1, accelerate 1.15.0
Model: stabilityai/stable-diffusion-3.5-medium at revision b940f670f0eda2d07fbb75229e779da1ad11eb80
Precision: bf16
Pipeline: StableDiffusion3Pipeline, whole model on the GPU, no CPU offload
Scheduler: FlowMatchEulerDiscreteScheduler, the repository default
Settings: Prompt “A red colored car.” · Negative prompt “” · Resolution 1024x1024 · Steps 40 · Guidance 4.5 · Seed of the recorded runs 7
Warm-up and runs: 1 warm-up, then 5 timed
What is timed: the pipe() call with the weights already loaded; it holds text encoding, denoising and decoding
Where the recipe comes from: Written from the recorded pipeline class, dtype and call arguments. The recorded runs used seed 7; the snippet uses seed 42 so both engines draw the same image. Use one seed for both.

install · diffusers

python3 -m venv ~/dfenv && ~/dfenv/bin/pip install torch==2.8.0 torchvision==0.23.0 diffusers==0.39.0 transformers==5.14.1 accelerate==1.15.0 safetensors==0.8.0 "numpy>=2" pillow ftfy sentencepiece protobuf huggingface_hub --extra-index-url https://download.pytorch.org/whl/cu128

run · diffusers

import time, torch
from diffusers import StableDiffusion3Pipeline
pipe = StableDiffusion3Pipeline.from_pretrained(
    "stabilityai/stable-diffusion-3.5-medium",
    revision="b940f670f0eda2d07fbb75229e779da1ad11eb80",
    torch_dtype=torch.bfloat16).to("cuda")
args = dict(prompt="A red colored car.", height=1024, width=1024,
            num_inference_steps=40, guidance_scale=4.5)
pipe(**args, generator=torch.Generator("cuda").manual_seed(42))   # warm-up, not timed
for rep in range(5):
    torch.cuda.synchronize(); t0 = time.perf_counter()
    image = pipe(**args, generator=torch.Generator("cuda").manual_seed(42)).images[0]
    torch.cuda.synchronize(); print("render_s", time.perf_counter() - t0)

GPU	Veizik run	diffusers run	Same host and session
RTX 3090	NVIDIA GeForce RTX 3090 (24 GB, sm_86); power 350 W, uncapped; driver 580.126.20; dedicated single-GPU cloud host; no quiet gate.	350 W limit · driver 580.159.03	No
RTX 4090	NVIDIA GeForce RTX 4090 (24 GB, sm_89); power 450 W, uncapped; driver 580.159.04; dedicated single-GPU cloud host; no quiet gate.	450 W limit · driver 580.126.20	No
RTX 5090	NVIDIA GeForce RTX 5090 (32 GB, sm_120); power 575 W, uncapped; driver 590.48.01; dedicated single-GPU cloud host; no quiet gate.	575 W limit · driver 580.126.20	No
RTX A5000	NVIDIA RTX A5000 (24 GB); 230 W, uncapped; driver 580.159.04; dedicated single-GPU cloud host; no quiet gate.	230 W limit · driver 580.159.04	No
A100-SXM4-80GB	NVIDIA A100-SXM4-80GB (81920 MiB, sm_80); power 400 W, uncapped; driver 580.159.03; dedicated single-GPU cloud host; no quiet gate.	400 W limit · driver 580.159.03	No

Recorded on separate dedicated cloud hosts of the same GPU model, not in the same session as our runs.

Time both the same way

Time per step: on both engines, time the denoising steps only and divide by the step count. For diffusers, take timestamps in a callback_on_step_end function; dividing the whole pipe() time by 40 also counts text encoding and decoding.
Finished image: on both engines, time from starting the command to the image written to disk, from a fresh process.
Peak memory: sample nvidia-smi memory.used for both engines during the run, from the same idle baseline.

What must match

GPU model; driver; power limit; clocks left at default; prompt; resolution; steps; guidance; seed; model revision; warm-up; number of timed runs; nothing else running on the GPU.

Wan2.1 T2V 1.3B

Measured on: RTX 4090, RTX 5090, A100-SXM4-80GB. Our figures for these GPUs are on the benchmarks page.

Veizik

Build: A development build of the WAN engine, named with each figure on the benchmarks page. It is not distributed.
Command: ./veizik run wan21_t2v_1_3b --out DIR
Note: This is the command of the NVIDIA WAN package, run from its directory; it writes the frames as PNG files to DIR/frames/. Our figures were measured on a development build, so the same bytes cannot be rerun. Set the operating point below with the options the package's command guide lists.
Operating point: Prompt “In a still frame, a stop sign” · Resolution 832x480 · Frames 81 · Steps 50 · Guidance 6 · Seed 7
Mode: balanced
Warm-up and runs: 1 warm-up render, then 2 timed renders; the figure is their median
What is timed: Render: the warm render call with the weights already loaded; loading is excluded
Hardware: Dedicated single-GPU cloud host; the table below gives the GPU, power limit and driver of each of our runs.

diffusers

Engine: diffusers 0.39.0
Also installed: PyTorch 2.8.0+cu128, CUDA runtime 12.8, transformers 5.14.1, accelerate 1.15.0
Model: Wan-AI/Wan2.1-T2V-1.3B-Diffusers at revision 0fad780a534b6463e45facd96134c9f345acfa5b
Precision: bf16
Pipeline: WanPipeline, whole model on the GPU, no CPU offload, no VAE slicing
Scheduler: the repository default (UniPCMultistepScheduler); its flow shift was left at the repository value
Settings: Prompt “In a still frame, a stop sign” · Negative prompt “” · Resolution 832x480 · Frames 81 · Steps 50 · Guidance 6
Warm-up and runs: 1 warm-up, then 2 timed
What is timed: the warm pipe() call, with torch.cuda.synchronize() before and after; loading is excluded
Where the recipe comes from: Written from the recorded pipeline class, dtype and call arguments. In the same-machine runs both engines started from one shared initial latent instead of a seed; the snippet uses seed 7 on both sides.

install · diffusers

python3 -m venv ~/dfenv && ~/dfenv/bin/pip install torch==2.8.0 torchvision==0.23.0 diffusers==0.39.0 transformers==5.14.1 accelerate==1.15.0 safetensors==0.8.0 "numpy>=2" pillow ftfy sentencepiece protobuf huggingface_hub --extra-index-url https://download.pytorch.org/whl/cu128

run · diffusers

import time, torch
from diffusers import WanPipeline
pipe = WanPipeline.from_pretrained(
    "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
    revision="0fad780a534b6463e45facd96134c9f345acfa5b",
    torch_dtype=torch.bfloat16).to("cuda")
args = dict(prompt="In a still frame, a stop sign", negative_prompt="",
            height=480, width=832, num_frames=81,
            num_inference_steps=50, guidance_scale=6.0)
pipe(**args, generator=torch.Generator("cuda").manual_seed(7))    # warm-up, not timed
for rep in range(2):
    torch.cuda.synchronize(); t0 = time.perf_counter()
    frames = pipe(**args, generator=torch.Generator("cuda").manual_seed(7)).frames[0]
    torch.cuda.synchronize(); print("render_s", time.perf_counter() - t0)

GPU	Veizik run	diffusers run	Same host and session
RTX 4090	NVIDIA GeForce RTX 4090; 450 W limit, mean draw 446.95 W; driver 580.126.20; dedicated single-GPU cloud host; two runs.	450 W limit · driver 580.126.20	Yes
A100-SXM4-80GB	NVIDIA A100-SXM4-80GB; 400 W limit, mean draw 387.65 W; driver 580.159.04; dedicated single-GPU cloud host; two runs.	400 W limit · driver 580.159.04	Yes
RTX 5090	NVIDIA GeForce RTX 5090; 575 W, uncapped; driver 570.195.03; dedicated single-GPU cloud host; two runs.	575 W limit · driver 580.173.02	No

On the RTX 4090 and the A100 both engines ran on the same cloud host in one session, at the same power limit and driver. The RTX 5090 setup was recorded on a different host a month earlier, with a different driver from our run.

Time both the same way

Render: on both engines, time the render call with the weights already loaded, after one warm-up render that is not counted. Synchronise the GPU before reading the clock.
Run the two engines on the same machine in one session when you can; a different host or driver changes the result.

What must match

GPU model; driver; power limit; clocks left at default; prompt and negative prompt; resolution; frames; steps; guidance; seed; model revision; warm-up; number of timed runs; nothing else running on the GPU.

Qwen3.5 and Qwen3.6 on NVIDIA GPUs

No version and run command of another engine is recorded for these figures, so no comparison setup is given here. Our side runs with the package's own recipe below.

CUDA decode (own figures)

Command: veizik run <model> "$(cat fact_chatml.txt)" --mode <mode> --tokens 128 --seed 42 --out ./out
Prompt: repro_kit/prompts/fact_chatml.txt in veizikhq/veizik-cuda (a ChatML user turn; the engine reports 21 prompt tokens). The kit's bench_grid.sh runs the whole grid through the installed CLI only.
Tokens: 128 generated tokens, 2 runs per cell; the 1,024-token cell is 1 run with --tokens 1024. Greedy decoding.
Machine: One NVIDIA GPU on native Linux with nothing else running on it, at its default board power, recorded with the driver version (cuda_deploy/bench/REPRODUCIBILITY.md in veizikhq/veizik-cuda).

CUDA prompt pass (own figures)

Command: veizik run <model> "$(cat fact_chatml.txt)" --mode <mode> --tokens 128 --seed 42 --out ./out
Prompt: repro_kit/prompts/fact_chatml.txt in veizikhq/veizik-cuda (a ChatML user turn; the engine reports 21 prompt tokens). The kit's bench_grid.sh runs the whole grid through the installed CLI only.
Tokens: 128 generated tokens, 2 runs per cell; the 1,024-token cell is 1 run with --tokens 1024. Greedy decoding.
Machine: One NVIDIA GPU on native Linux with nothing else running on it, at its default board power, recorded with the driver version (cuda_deploy/bench/REPRODUCIBILITY.md in veizikhq/veizik-cuda).

CUDA resident memory (own figures)

Command: veizik run <model> "$(cat fact_chatml.txt)" --mode <mode> --tokens 128 --seed 42 --out ./out
Prompt: repro_kit/prompts/fact_chatml.txt in veizikhq/veizik-cuda (a ChatML user turn; the engine reports 21 prompt tokens). The kit's bench_grid.sh runs the whole grid through the installed CLI only.
Tokens: Read during the same runs as the decode figures, after load.
Machine: One NVIDIA GPU on native Linux with nothing else running on it, at its default board power, recorded with the driver version (cuda_deploy/bench/REPRODUCIBILITY.md in veizikhq/veizik-cuda).

Apple Silicon · Metal

Language models.

The reproduction kit runs both engines with the same prompts and protocol. Download it from the benchmarks page.

Qwen2.5 and Qwen3 decode and prefill on Apple Silicon

Measured on: Apple M1 Ultra, Apple M2 Pro. Our figures for these Macs are on the benchmarks page.

Veizik

Build: The Apple Silicon build named with each figure on the benchmarks page
Command: veizik run <model> --tokens 128 --ignore-eos "$(cat prompts/p<N>_qwen25.txt)"
Note: Run through the reproduction kit's bench.sh, which drives both engines. Use the prompts/p<N>_qwen3.txt files for Qwen3-1.7B.
Operating point: Prompt lengths 512, 3968, 8192, 16384, 24576 and 30000 tokens · Generated tokens 128 · Decoding greedy, end-of-text ignored
Mode: the default mode of the build
Warm-up and runs: 1 warm-up run of 8 tokens per engine, then 11 rounds; each round runs every engine once and the order reverses every round; the figure is the median
What is timed: Decode: the engine's own clock: the 127 decoded tokens after the first one
Prefill: the engine's own clock for the prompt pass
Hardware: Apple M1 Ultra (macOS 26.5.1) and Apple M2 Pro (macOS 15.5), on AC power, Low Power Mode off, checked quiet before every run

MLX

Engine: MLX, mlx 0.32.2, mlx-lm 0.31.3
Model: Qwen/Qwen2.5-0.5B at 060db6499f32faf8b98477b0a26969ef7d8b9987
Qwen/Qwen2.5-1.5B at 8faed761d45a263340a0528343f099c05c9a4323
Qwen/Qwen3-1.7B at 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
Qwen/Qwen2.5-7B at d149729398750b98c0af14eb82c78cfe92750796
model.safetensors written by the conversion; the kit's README lists its sha256 per model
Precision: 4-bit g64, converted from the snapshot
Settings: Generated tokens 128 · Decoding greedy, end-of-text masked
Warm-up and runs: 1 warm-up, then 11 timed
What is timed: mlx_lm's own Prompt: and Generation: lines; its generation window holds 127 tokens
Where the recipe comes from: the reproduction kit served at /bench/veizik-bench-kit-pd1.tgz, the kit these rows were measured with

MLX bf16

Engine: MLX, mlx 0.32.2, mlx-lm 0.31.3
Model: Qwen/Qwen2.5-0.5B at 060db6499f32faf8b98477b0a26969ef7d8b9987
Qwen/Qwen2.5-1.5B at 8faed761d45a263340a0528343f099c05c9a4323
Qwen/Qwen3-1.7B at 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
Qwen/Qwen2.5-7B at d149729398750b98c0af14eb82c78cfe92750796
the snapshot as published
Precision: bf16
Settings: Generated tokens 128 · Decoding greedy, end-of-text masked
Warm-up and runs: 1 warm-up, then 11 timed
What is timed: mlx_lm's own Prompt: and Generation: lines
Where the recipe comes from: the reproduction kit served at /bench/veizik-bench-kit-pd1.tgz

install · MLX

python3 -m venv ~/mlxenv && ~/mlxenv/bin/pip install mlx==0.32.2 mlx-lm==0.31.3 huggingface_hub

convert · MLX

python -m mlx_lm convert --hf-path <snapshot> --mlx-path mlx_q4/<model>-q4g64 -q --q-bits 4 --q-group-size 64

install · MLX bf16

python3 -m venv ~/mlxenv && ~/mlxenv/bin/pip install mlx==0.32.2 mlx-lm==0.31.3 huggingface_hub

Time both the same way

Read each engine's own clock: Veizik's prefill and generated lines on stderr, and mlx_lm's Prompt: and Generation: lines. The kit's README gives the rate definitions it applies to both.
Interleave the engines run by run, and check the machine is quiet before each run. bench.sh does both.

What must match

Mac model and macOS version; AC power, Low Power Mode off; prompt file (the kit checks its sha256); prompt length; generated tokens; end-of-text ignored on both; model revision; warm-up; 11 interleaved runs; nothing else using the GPU.

Qwen2.5 and Qwen3 weights in device memory on Apple Silicon

Measured on: Apple M1 Ultra, Apple M1 Max, Apple M4 Pro. Our figures for these Macs are on the benchmarks page.

Veizik

Build: The Apple Silicon build named with each figure on the benchmarks page
Command: veizik run <model> --tokens 2 "Hello."
Note: Read the allocated-bytes line the engine prints on stderr as it loads.
Operating point: Prompt “Hello.” · Generated tokens 2
Mode: the default mode of the build
Warm-up and runs: 1 run; the figure does not move between runs or machines
What is timed: Device memory: the weights alone, as allocated at load; no cache
Hardware: Apple M1 Max, M1 Ultra and M4 Pro

llama.cpp (llama-bench, Metal)

Engine: llama.cpp, commit 64e9bceb2c3a856efed96feda784a50947049feb on the M1 Max and M1 Ultra; build 11146, commit 7fe450e19, on the M4 Pro
Model: qwen2.5-0.5B-Q4_K_M.gguf sha256 ca8c9ec09fad5368067bfb6fa9eb40293561b171df44e207b52e0f2e88794839
qwen2.5-1.5B-Q4_K_M.gguf sha256 bc611b132d5f9fc22334f8afbe4d6b37583beddfe4d32008dd1e14a27bdd8922
qwen3-1.7B-Q4_K_M.gguf sha256 740b6373413005af99611f27c21f19711398b6289d758e60276a698e5e9652fa
qwen2.5-7B-Q4_K_M.gguf sha256 2d6e42b4f8a14fe9acee20ed6b8f9f7c48b47fc73516ea71965f9fa2ec9dc1bb
Q4_K_M GGUF files, matched by sha256 (M1 Max and M1 Ultra); where they were downloaded from is not recorded
Precision: Q4_K_M GGUF
Warm-up and runs: 1 reading
What is timed: the model size llama-bench prints
Where the recipe comes from: llama-bench prints the size of the file it loaded; the figure is that size. The other flags of the recorded run are not recorded and do not change it.

MLX

Engine: MLX, mlx 0.32.2, mlx-lm 0.31.3 (M1 Max and M1 Ultra)
Model: Qwen/Qwen2.5-0.5B at 060db6499f32faf8b98477b0a26969ef7d8b9987
Qwen/Qwen2.5-1.5B at 8faed761d45a263340a0528343f099c05c9a4323
Qwen/Qwen3-1.7B at 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
Qwen/Qwen2.5-7B at d149729398750b98c0af14eb82c78cfe92750796
Precision: 4-bit g64, converted from the snapshot
Warm-up and runs: 1 reading
What is timed: the weight payload of the files MLX loads
Where the recipe comes from: MLX reports only a peak that includes its cache, so its weights figure is measured from the files it loads

install · llama.cpp (llama-bench, Metal)

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp && git checkout 64e9bceb2c3a856efed96feda784a50947049feb && cmake -B build && cmake --build build --config Release

run · llama.cpp (llama-bench, Metal)

build/bin/llama-bench -m <model>-Q4_K_M.gguf

install · MLX

python3 -m venv ~/mlxenv && ~/mlxenv/bin/pip install mlx==0.32.2 mlx-lm==0.31.3 huggingface_hub

convert · MLX

python -m mlx_lm convert --hf-path <snapshot> --mlx-path mlx_q4/<model>-q4g64 -q --q-bits 4 --q-group-size 64

Time both the same way

Nothing is timed. Weights only on every side, never the cache that grows with a conversation.

What must match

model revision or file; weights only (no cache).

Not covered

Figures with no comparison setup here, and why.

Figures	Why there is no setup
Qwen3.5 and Qwen3.6 on NVIDIA GPUs: decode, prompt pass, GPU memory, serving, and the development-build figures	No version and run command of another engine is recorded for these figures, so no comparison setup is given. Our own recipe is in the NVIDIA section.
Stable Diffusion 3.5 Large on RTX 3090, RTX 4090 and RTX A5000	No completed run of another engine is recorded on these cards at this operating point.
Stable Diffusion 3.5 Large on RTX 5090	The recorded run of the other engine used 28 steps and guidance 3.5; our figures use 40 steps. That is not a matching setup.
Stable Diffusion 3.5 Large on A100 80GB PCIe	A matching run exists, but its model revision, driver and power limit were not recorded.
Wan2.1 T2V 1.3B on RTX 3080 Ti	No completed run of another engine is recorded on this card at these operating points.
Wan2.2 T2V A14B on RTX 4090 and RTX 5090	No completed run of another engine exists at this operating point.
Apple M1 Max decode and prefill	Our own figures only; no comparison was run.
Installed size	Our own figure only.