Qwen3.5
Run the Qwen3.5 dense checkpoints in veizik's native runtime. Weights are user-supplied, not shipped.
- Status
- Available
- Platform
- NVIDIA · CUDA
- Class
- Language
- Publisher
- Alibaba (Qwen)
- Architecture
- Qwen3.5
- Weights
- Supplied by you
What this package does
From the documents of veizik-qwen-cuda-1.1.3-linux-x86_64.vzpack (release v1.1.3).
- Five language models in one package: Qwen3.5-0.8B, Qwen3.5-2B, Qwen3.5-4B, Qwen3.5-9B and Qwen3.6-35B-A3B (mixture of experts, about 3B active per token). Every model does completion and chat.
veizik run <model> "prompt"generates once and writes the text to gen.txt in the folder you pass with--out. Greedy decoding is the default and gives the same output every time.--tokens N(or--max-tokens N) sets the length, and--ignore-eosemits exactly N tokens. Sampling is opt-in with--seed,--temperature,--top-pand--top-k, and is experimental.- Run options are one vocabulary across engines, and the command passes on only the ones an engine accepts. For Qwen:
--seed,--steps,--mode,--temperature,--top-p,--top-k,--tokens,--max-tokens, and on Qwen3.6-35B-A3B also--image,--video,--image-max-pixels,--video-max-pixelsand--video-fps. Any other option is refused with<flag> does not apply to this model.Every run also takes--out,--name,--prompt-file,--resume,--resume-fromand--checkpoint-every. veizik chat <model>holds an interactive conversation, using the package's signed model catalogue. It needsveizik activateonce per device and the model's weights (--weights DIRorVZ_WEIGHTS).veizik serve <model>is a local OpenAI-compatible HTTP server on 127.0.0.1 only:POST /v1/chat/completions(with streaming),POST /v1/completions,GET /v1/models, and health endpoints. Turn on an API key with--require-auth --keyfile FILE. Each request may use up to 4,096 tokens, prompt plus output. It needsveizik activateonce per device and the model's weights (--weights DIRorVZ_WEIGHTS).veizik doctorchecks the install root, the activation service and the CLI binary;veizik statusshows the device's activation and plan;veizik versionshows the version and backend.- Five modes: quality (most memory, slowest), balanced (the default), fast, ultrafast (the fastest) and low-memory (smallest footprint, lowest quality).
- GPU code for text on sm_80, sm_86, sm_89, sm_90a, sm_100a and sm_120a. One GPU runs one model.
- Activation is once per device, and one approval covers every model:
veizik activatein a browser, orveizik activate --key KEY --password-stdinwithout one.veizik deviceslists the devices on your licence and switches them on or off, andveizik device-idprints this machine's id. veizik installdownloads the package from its public GitHub release and refuses it unless its SHA-256 matches the published release; the engine checks its own files at every start, and a changed, added or missing file stops the run with exit code 4.
How to run it
- Get the veizik command
Get the
veizikcommand for Linux x86-64:veizik$ curl -fsSL https://veizik.com/install.sh | sh # if the installer asks you to put ~/.local/bin on your PATH $ export PATH="$HOME/.local/bin:$PATH"
- Install the model engine
This model is in
veizik-qwen-cuda-1.1.3-linux-x86_64.vzpack.veizik installdownloads it from its GitHub release and refuses it unless its SHA-256 matches the one on the download page.install# accepted names: qwen, qwen35, qwen36 or a full model key (qwen35_0p8b, qwen35_2b, qwen35_4b, qwen35_9b, qwen36_35b_a3b); any case; '.' and '-' work too (qwen3.6) $ veizik install qwen35
- Activate this device
Approve the machine once; it receives a licence bound to it.
activate# have the Qwen3.5-0.8B checkpoint on this machine first: activation ends with a short check generation that uses it $ export VZ_WEIGHTS=/data/models $ veizik activate # without a browser (SSH, servers): create a key and a CLI password on your account page, then $ veizik activate --key KEY --password-stdin < pwfile
- Run it
The first commands for this model, as the package’s own documents give them.
run$ veizik run qwen35_2b "The capital of France is" --tokens 32 $ veizik chat qwen35_2b $ veizik serve qwen35_2b --weights /data/models/Qwen3.5-2B --port 8000 # then POST http://127.0.0.1:8000/v1/chat/completions
Where it runs.
Evidence and availability are separate: a verified device is not a download, and an available model is not verified on every device.
| Family | RTX 3080 Ti | RTX 3090 | RTX 4090 | RTX 5090 | RTX A5000 | A40 | A100 80GB PCIe | A100-SXM4-80GB | H100 80GB HBM3 | RTX PRO 4500 Blackwell | M4 Pro | M1 Ultra | M1 Max | M2 Pro |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.5 all variants | ◐Architecture-supported | ●Verified | ●Verified | ●Verified | ●Verified | ◐Architecture-supported | ○Expected, untested | ○Expected, untested | ○Expected, untested | ◐Architecture-supported | —Not offered | —Not offered | —Not offered | —Not offered |
Availability by variant
| Model id | Availability | Platform | Context | Weights in memory |
|---|---|---|---|---|
qwen35-0.8b | Available | NVIDIA | 4,096 | — |
qwen35-2b | Available | NVIDIA | 4,096 | — |
qwen35-4b | Available | NVIDIA | 4,096 | — |
qwen35-9b | Available | NVIDIA | 4,096 | — |
Verified on
- GeForce RTX 3090 · build
cuda-qwen-1.1.3· 2026-10-22 · published measurement - NVIDIA GeForce RTX 4090 · build
cuda-qwen-1.1.3· 2026-10-21 · published measurement - NVIDIA GeForce RTX 5090 · build
cuda-qwen-1.1.3· 2026-10-21 · published measurement - NVIDIA RTX A5000 · build
cuda-qwen-1.1.3· 2026-10-21 · published measurement
“Verified” means this family ran end to end on that exact device, on a published build. Architecture membership never upgrades a device to verified.
4 checkpoints
| Model id | Size | Checkpoint | On disk | Context | Weights in memory |
|---|---|---|---|---|---|
qwen35-0.8b | 0.8B | Qwen/Qwen3.5-0.8B | — | — | — |
qwen35-2b | 2B | Qwen/Qwen3.5-2B | — | — | — |
qwen35-4b | 4B | Qwen/Qwen3.5-4B | — | — | — |
qwen35-9b | 9B | Qwen/Qwen3.5-9B | — | — | — |
Our figures for this build
Each table shows the GPUs a model was measured on and one figure per measure: the default mode and the most common workload. Hover a figure for its build, date, runs and range; open All figures for every mode and workload. A figure marked earlier build is our own measurement of an earlier build of the engine, shown because the computation it exercises did not change in substance; it is replaced when the package you download is measured on that GPU. A figure whose build reads carried, measured on another build was measured on that earlier build of the same package and stands for the package named, because the change between the two did not alter the computation.
Qwen3.5-0.8B
| GPU | Decode tok/s | Decode, one token per step tok/s | Prompt pass ms | GPU memory in use GiB | Throughput, many at once tok/s |
|---|---|---|---|---|---|
| RTX 3090 | 967.5balanced · 20-token prompt | 795.7512-token promptearlier build | 4.2balanced · 20-token prompt | 2.27balanced | 3,1258 concurrent requests |
| RTX 4090 | 1,318balanced · 21-token prompt | 1,008512-token promptearlier build | 3.0balanced · 21-token prompt | 2.40balanced | — |
| RTX 5090 | 1,529balanced · 21-token prompt | — | 2.7balanced · 21-token prompt | 2.51balanced | — |
| RTX A5000 | 776.7balanced · 20-token prompt | — | 4.9balanced · 20-token prompt | 2.22balanced | — |
| A100-SXM4-80GB | — | 688.1512-token promptearlier build | — | — | — |
| H100 80GB HBM3 | — | 1,004512-token promptearlier build | — | — | — |
All 54 figures for Qwen3.5-0.8B
| GPU | Measure | Mode | Workload | Value | Runs | Build | Date |
|---|---|---|---|---|---|---|---|
| RTX 3090 | Decode | balanced | 20 / 128 | 967.5 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | balanced | 20 / 1024 | 1,052 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | fast | 20 / 128 | 972.0 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | quality | 20 / 128 | 880.8 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | single-token decoding | 512 / — | 795.7 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| RTX 3090 | Prompt pass | balanced | 20 / 128 | 4.2 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Prompt pass | balanced | 21 / 1024 | 4.4 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 3090 | Prompt pass | fast | 20 / 128 | 4.1 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Prompt pass | quality | 20 / 128 | 4.1 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | balanced | — / — | 2.27 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | fast | — / — | 2.27 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | quality | — / — | 2.42 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Throughput | 1 request at a time | veizik serve, 1 concurrent request, 128 tokens each | 531.9 tok/s | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Throughput | 4 concurrent requests | veizik serve, 4 concurrent requests, 128 tokens each | 1,778 tok/s | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Throughput | 8 concurrent requests | veizik serve, 8 concurrent requests, 128 tokens each | 3,125 tok/s | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | First token | 1 request at a time | veizik serve, 1 concurrent request, 128 tokens each | 24.6 ms | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | First token | 4 concurrent requests | veizik serve, 4 concurrent requests, 128 tokens each | 62.9 ms | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | First token | 8 concurrent requests | veizik serve, 8 concurrent requests, 128 tokens each | 92.6 ms | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 4090 | Decode | balanced | 21 / 128 | 1,318 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | balanced | 21 / 1024 | 1,439 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | fast | 21 / 128 | 1,316 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | quality | 21 / 128 | 1,156 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | single-token decoding | 512 / — | 1,008 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| RTX 4090 | Prompt pass | balanced | 21 / 128 | 3.0 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | balanced | 21 / 1024 | 3.0 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | fast | 21 / 128 | 3.0 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | quality | 21 / 128 | 3.1 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | balanced | — / — | 2.40 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | fast | — / — | 2.40 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | quality | — / — | 2.55 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | balanced | 21 / 128 | 1,529 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | balanced | 21 / 1024 | 1,676 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | fast | 21 / 128 | 1,531 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | quality | 21 / 128 | 1,438 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | balanced | 21 / 128 | 2.7 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | balanced | 21 / 1024 | 2.7 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | fast | 21 / 128 | 2.7 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | quality | 21 / 128 | 2.9 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | balanced | — / — | 2.51 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | fast | — / — | 2.51 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | quality | — / — | 2.66 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | balanced | 20 / 128 | 776.7 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | balanced | 921 / 1024 | 916.1 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | fast | 20 / 128 | 771.9 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | quality | 20 / 128 | 714.7 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | balanced | 20 / 128 | 4.9 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | balanced | 921 / 1024 | 28.6 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | fast | 20 / 128 | 4.9 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | quality | 20 / 128 | 4.6 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | balanced | — / — | 2.22 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | fast | — / — | 2.22 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | quality | — / — | 2.37 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| A100-SXM4-80GB | Decode | single-token decoding | 512 / — | 688.1 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| H100 80GB HBM3 | Decode | single-token decoding | 512 / — | 1,004 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
Qwen3.5-2B
| GPU | Decode tok/s | Decode, one token per step tok/s | Prompt pass ms | GPU memory in use GiB |
|---|---|---|---|---|
| RTX 3090 | 586.5balanced · 20-token prompt | 435.9512-token promptearlier build | 6.6balanced · 20-token prompt | 3.48balanced |
| RTX 4090 | 886.9balanced · 21-token prompt | 548.4512-token promptearlier build | 4.5balanced · 21-token prompt | 3.61balanced |
| RTX 5090 | 1,138balanced · 21-token prompt | — | 3.8balanced · 21-token prompt | 3.72balanced |
| RTX A5000 | 467.3balanced · 20-token prompt | — | 8.4balanced · 20-token prompt | 3.43balanced |
| A100-SXM4-80GB | — | 439.4512-token promptearlier build | — | — |
| H100 80GB HBM3 | — | 660.2512-token promptearlier build | — | — |
All 48 figures for Qwen3.5-2B
| GPU | Measure | Mode | Workload | Value | Runs | Build | Date |
|---|---|---|---|---|---|---|---|
| RTX 3090 | Decode | balanced | 20 / 128 | 586.5 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | balanced | 20 / 1024 | 560.4 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | fast | 20 / 128 | 585.1 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | quality | 20 / 128 | 542.7 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | single-token decoding | 512 / — | 435.9 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| RTX 3090 | Prompt pass | balanced | 20 / 128 | 6.6 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Prompt pass | balanced | 21 / 1024 | 7.1 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 3090 | Prompt pass | fast | 20 / 128 | 6.7 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Prompt pass | quality | 20 / 128 | 6.9 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | balanced | — / — | 3.48 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | fast | — / — | 3.48 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | quality | — / — | 3.80 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 4090 | Decode | balanced | 21 / 128 | 886.9 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | balanced | 21 / 1024 | 868.8 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | fast | 21 / 128 | 895.1 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | quality | 21 / 128 | 740.9 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | single-token decoding | 512 / — | 548.4 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| RTX 4090 | Prompt pass | balanced | 21 / 128 | 4.5 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | balanced | 21 / 1024 | 4.4 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | fast | 21 / 128 | 4.4 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | quality | 21 / 128 | 4.7 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | balanced | — / — | 3.61 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | fast | — / — | 3.61 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | quality | — / — | 3.93 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | balanced | 21 / 128 | 1,138 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | balanced | 21 / 1024 | 1,107 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | fast | 21 / 128 | 1,137 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | quality | 21 / 128 | 1,041 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | balanced | 21 / 128 | 3.8 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | balanced | 21 / 1024 | 3.8 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | fast | 21 / 128 | 3.8 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | quality | 21 / 128 | 4.1 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | balanced | — / — | 3.72 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | fast | — / — | 3.72 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | quality | — / — | 4.03 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | balanced | 20 / 128 | 467.3 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | balanced | 921 / 1024 | 449.1 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | fast | 20 / 128 | 464.6 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | quality | 20 / 128 | 435.2 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | balanced | 20 / 128 | 8.4 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | balanced | 921 / 1024 | 58.5 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | fast | 20 / 128 | 8.4 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | quality | 20 / 128 | 9.0 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | balanced | — / — | 3.43 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | fast | — / — | 3.43 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | quality | — / — | 3.75 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| A100-SXM4-80GB | Decode | single-token decoding | 512 / — | 439.4 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| H100 80GB HBM3 | Decode | single-token decoding | 512 / — | 660.2 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
Qwen3.5-4B
| GPU | Decode tok/s | Decode, one token per step tok/s | Prompt pass ms | GPU memory in use GiB |
|---|---|---|---|---|
| RTX 3090 | 376.8balanced · 20-token prompt | 208.2512-token promptearlier build | 13.5balanced · 20-token prompt | 5.49balanced |
| RTX 4090 | 531.8balanced · 21-token prompt | 247.5512-token promptearlier build | 8.7balanced · 21-token prompt | 5.62balanced |
| RTX 5090 | 772.2balanced · 21-token prompt | — | 6.9balanced · 21-token prompt | 5.73balanced |
| RTX A5000 | 280.5balanced · 20-token prompt | — | 17.2balanced · 20-token prompt | 5.44balanced |
| A100-SXM4-80GB | — | 224.3512-token promptearlier build | — | — |
| H100 80GB HBM3 | — | 365.1512-token promptearlier build | — | — |
All 48 figures for Qwen3.5-4B
| GPU | Measure | Mode | Workload | Value | Runs | Build | Date |
|---|---|---|---|---|---|---|---|
| RTX 3090 | Decode | balanced | 20 / 128 | 376.8 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | balanced | 20 / 1024 | 397.8 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | fast | 20 / 128 | 376.8 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | quality | 20 / 128 | 342.1 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | single-token decoding | 512 / — | 208.2 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| RTX 3090 | Prompt pass | balanced | 20 / 128 | 13.5 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Prompt pass | balanced | 21 / 1024 | 13.9 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 3090 | Prompt pass | fast | 20 / 128 | 13.1 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Prompt pass | quality | 20 / 128 | 14.0 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | balanced | — / — | 5.49 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | fast | — / — | 5.49 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | quality | — / — | 6.27 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 4090 | Decode | balanced | 21 / 128 | 531.8 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | balanced | 21 / 1024 | 568.2 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | fast | 21 / 128 | 535.4 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | quality | 21 / 128 | 441.8 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | single-token decoding | 512 / — | 247.5 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| RTX 4090 | Prompt pass | balanced | 21 / 128 | 8.7 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | balanced | 21 / 1024 | 8.6 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | fast | 21 / 128 | 8.7 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | quality | 21 / 128 | 10.5 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | balanced | — / — | 5.62 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | fast | — / — | 5.62 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | quality | — / — | 6.40 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | balanced | 21 / 128 | 772.2 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | balanced | 21 / 1024 | 822.2 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | fast | 21 / 128 | 771.4 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | quality | 21 / 128 | 670.5 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | balanced | 21 / 128 | 6.9 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | balanced | 21 / 1024 | 6.9 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | fast | 21 / 128 | 6.9 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | quality | 21 / 128 | 7.9 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | balanced | — / — | 5.73 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | fast | — / — | 5.73 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | quality | — / — | 6.50 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | balanced | 20 / 128 | 280.5 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | balanced | 921 / 1024 | 228.6 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | fast | 20 / 128 | 278.9 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | quality | 20 / 128 | 256.5 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | balanced | 20 / 128 | 17.2 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | balanced | 921 / 1024 | 154.0 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | fast | 20 / 128 | 17.2 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | quality | 20 / 128 | 19.2 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | balanced | — / — | 5.44 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | fast | — / — | 5.44 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | quality | — / — | 6.22 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| A100-SXM4-80GB | Decode | single-token decoding | 512 / — | 224.3 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| H100 80GB HBM3 | Decode | single-token decoding | 512 / — | 365.1 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
Qwen3.5-9B
| GPU | Decode tok/s | Decode, one token per step tok/s | Prompt pass ms | GPU memory in use GiB | Throughput, many at once tok/s |
|---|---|---|---|---|---|
| RTX 3090 | 190.2balanced · 20-token prompt | 117.0512-token promptearlier build | 21.7balanced · 20-token prompt | 8.44balanced | 663.08 concurrent requests |
| RTX 4090 | 347.0balanced · 21-token prompt | 154.9512-token promptearlier build | 16.8balanced · 21-token prompt | 8.57balanced | — |
| RTX 5090 | 498.7balanced · 21-token prompt | 242.4512-token promptearlier build | 11.1balanced · 21-token prompt | 8.67balanced | — |
| RTX A5000 | 151.0balanced · 20-token prompt | — | 29.0balanced · 20-token prompt | 8.39balanced | — |
| A100-SXM4-80GB | — | 129.4512-token promptearlier build | — | — | — |
| H100 80GB HBM3 | — | 241.6512-token promptearlier build | — | — | — |
All 55 figures for Qwen3.5-9B
| GPU | Measure | Mode | Workload | Value | Runs | Build | Date |
|---|---|---|---|---|---|---|---|
| RTX 3090 | Decode | balanced | 20 / 128 | 190.2 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | balanced | 20 / 1024 | 170.2 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | fast | 20 / 128 | 190.4 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | quality | 20 / 128 | 172.8 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Decode | single-token decoding | 512 / — | 117.0 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| RTX 3090 | Prompt pass | balanced | 20 / 128 | 21.7 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Prompt pass | balanced | 21 / 1024 | 22.4 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 3090 | Prompt pass | fast | 20 / 128 | 21.8 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Prompt pass | quality | 20 / 128 | 23.2 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | balanced | — / — | 8.44 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | fast | — / — | 8.44 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | GPU memory in use | quality | — / — | 9.82 GiB | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Throughput | 1 request at a time | veizik serve, 1 concurrent request, 128 tokens each | 101.4 tok/s | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Throughput | 4 concurrent requests | veizik serve, 4 concurrent requests, 128 tokens each | 366.4 tok/s | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | Throughput | 8 concurrent requests | veizik serve, 8 concurrent requests, 128 tokens each | 663.0 tok/s | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | First token | 1 request at a time | veizik serve, 1 concurrent request, 128 tokens each | 55.6 ms | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | First token | 4 concurrent requests | veizik serve, 4 concurrent requests, 128 tokens each | 176.2 ms | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 3090 | First token | 8 concurrent requests | veizik serve, 8 concurrent requests, 128 tokens each | 308.8 ms | 3 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.0 | 2026-10-22 |
| RTX 4090 | Decode | balanced | 21 / 128 | 347.0 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | balanced | 21 / 1024 | 303.4 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | fast | 21 / 128 | 346.6 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | quality | 21 / 128 | 272.8 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Decode | single-token decoding | 512 / — | 154.9 tok/s | 2 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| RTX 4090 | Prompt pass | balanced | 21 / 128 | 16.8 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | balanced | 21 / 1024 | 16.6 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | fast | 21 / 128 | 16.5 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | Prompt pass | quality | 21 / 128 | 21.6 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | balanced | — / — | 8.57 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | fast | — / — | 8.57 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 4090 | GPU memory in use | quality | — / — | 9.95 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | balanced | 21 / 128 | 498.7 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | balanced | 21 / 1024 | 434.6 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | fast | 21 / 128 | 498.5 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | quality | 21 / 128 | 423.4 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Decode | single-token decoding | 512 / — | 242.4 tok/s | 3 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| RTX 5090 | Prompt pass | balanced | 21 / 128 | 11.1 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | balanced | 21 / 1024 | 11.1 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | fast | 21 / 128 | 11.1 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | Prompt pass | quality | 21 / 128 | 13.0 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | balanced | — / — | 8.67 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | fast | — / — | 8.67 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX 5090 | GPU memory in use | quality | — / — | 10.05 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | balanced | 20 / 128 | 151.0 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | balanced | 921 / 1024 | 124.8 tok/s | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | fast | 20 / 128 | 149.1 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Decode | quality | 20 / 128 | 137.8 tok/s | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | balanced | 20 / 128 | 29.0 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | balanced | 921 / 1024 | 269.9 ms | 1 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | fast | 20 / 128 | 29.2 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | Prompt pass | quality | 20 / 128 | 31.0 ms | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | balanced | — / — | 8.39 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | fast | — / — | 8.39 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| RTX A5000 | GPU memory in use | quality | — / — | 9.77 GiB | 2 | cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a65 | 2026-10-21 |
| A100-SXM4-80GB | Decode | single-token decoding | 512 / — | 129.4 tok/s | 2 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
| H100 80GB HBM3 | Decode | single-token decoding | 512 / — | 241.6 tok/s | 1 | earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled) | 2026-10-13 |
Measurement conditions
Default board power (limit not recorded); no quiet gate; dedicated single-GPU cloud host, native Linux; driver 595.91.07. NVIDIA A100-SXM4-80GB; board power not recorded; dedicated single-GPU cloud host; no quiet gate; one run; a measurement of the shipped package on this card will replace it. NVIDIA A100-SXM4-80GB; board power not recorded; dedicated single-GPU cloud host; no quiet gate; two runs on separate pods; a measurement of the shipped package on this card will replace it. NVIDIA GeForce RTX 3090; dedicated single-GPU cloud host, native Linux; default board limit 370 W, never reached (no power throttle, GPU at or below 44 C); driver 580.95.05; quiet pod. NVIDIA GeForce RTX 3090; dedicated single-GPU cloud host, native Linux; default board limit 370 W, never reached (no power throttle, GPU at or below 44 C); driver 580.95.05; quiet re-run. NVIDIA GeForce RTX 3090; dedicated single-GPU cloud host; board power not recorded; no quiet gate. NVIDIA GeForce RTX 4090; dedicated single-GPU cloud host; board power not recorded; no quiet gate. NVIDIA GeForce RTX 4090; dedicated single-GPU cloud host; board power not recorded; runs on separate pods; no quiet gate. NVIDIA GeForce RTX 5090; dedicated single-GPU cloud host; board power not recorded; runs on separate pods; no quiet gate. NVIDIA H100 80GB HBM3 (SXM); board power not recorded; dedicated single-GPU cloud host; no quiet gate; one run; a measurement of the shipped package on this card will replace it. No quiet gate; dedicated single-GPU cloud host, native Linux; board power default (not recorded); driver not recorded in the log.
Weights and terms
Qwen3.5 weights are published by Alibaba under Apache-2.0 and are supplied by the user, not shipped. Veizik ships only the inference engine and neither redistributes nor relicenses the weights.
Other models linked in the sidebar:
Stable Diffusion 3.5: Powered by Stability AI HunyuanImage: Weights are not included. Model availability and use are governed by the model provider's license and regional restrictions.