VeizikNVIDIA and Apple Silicon builds are available now.Release notes →
← Models
Language · NVIDIA · CUDA

Qwen3.5

Run the Qwen3.5 dense checkpoints in veizik's native runtime. Weights are user-supplied, not shipped.

Status
Available
Platform
NVIDIA · CUDA
Class
Language
Publisher
Alibaba (Qwen)
Architecture
Qwen3.5
Weights
Supplied by you
Package

What this package does

From the documents of veizik-qwen-cuda-1.1.3-linux-x86_64.vzpack (release v1.1.3).

  • Five language models in one package: Qwen3.5-0.8B, Qwen3.5-2B, Qwen3.5-4B, Qwen3.5-9B and Qwen3.6-35B-A3B (mixture of experts, about 3B active per token). Every model does completion and chat.
  • veizik run <model> "prompt" generates once and writes the text to gen.txt in the folder you pass with --out. Greedy decoding is the default and gives the same output every time. --tokens N (or --max-tokens N) sets the length, and --ignore-eos emits exactly N tokens. Sampling is opt-in with --seed, --temperature, --top-p and --top-k, and is experimental.
  • Run options are one vocabulary across engines, and the command passes on only the ones an engine accepts. For Qwen: --seed, --steps, --mode, --temperature, --top-p, --top-k, --tokens, --max-tokens, and on Qwen3.6-35B-A3B also --image, --video, --image-max-pixels, --video-max-pixels and --video-fps. Any other option is refused with <flag> does not apply to this model. Every run also takes --out, --name, --prompt-file, --resume, --resume-from and --checkpoint-every.
  • veizik chat <model> holds an interactive conversation, using the package's signed model catalogue. It needs veizik activate once per device and the model's weights (--weights DIR or VZ_WEIGHTS).
  • veizik serve <model> is a local OpenAI-compatible HTTP server on 127.0.0.1 only: POST /v1/chat/completions (with streaming), POST /v1/completions, GET /v1/models, and health endpoints. Turn on an API key with --require-auth --keyfile FILE. Each request may use up to 4,096 tokens, prompt plus output. It needs veizik activate once per device and the model's weights (--weights DIR or VZ_WEIGHTS).
  • veizik doctor checks the install root, the activation service and the CLI binary; veizik status shows the device's activation and plan; veizik version shows the version and backend.
  • Five modes: quality (most memory, slowest), balanced (the default), fast, ultrafast (the fastest) and low-memory (smallest footprint, lowest quality).
  • GPU code for text on sm_80, sm_86, sm_89, sm_90a, sm_100a and sm_120a. One GPU runs one model.
  • Activation is once per device, and one approval covers every model: veizik activate in a browser, or veizik activate --key KEY --password-stdin without one. veizik devices lists the devices on your licence and switches them on or off, and veizik device-id prints this machine's id.
  • veizik install downloads the package from its public GitHub release and refuses it unless its SHA-256 matches the published release; the engine checks its own files at every start, and a changed, added or missing file stops the run with exit code 4.
Guide

How to run it

  1. Get the veizik command

    Get the veizik command for Linux x86-64:

    veizik
    $ curl -fsSL https://veizik.com/install.sh | sh
    # if the installer asks you to put ~/.local/bin on your PATH
    $ export PATH="$HOME/.local/bin:$PATH"
  2. Install the model engine

    This model is in veizik-qwen-cuda-1.1.3-linux-x86_64.vzpack. veizik install downloads it from its GitHub release and refuses it unless its SHA-256 matches the one on the download page.

    install
    # accepted names: qwen, qwen35, qwen36 or a full model key (qwen35_0p8b, qwen35_2b, qwen35_4b, qwen35_9b, qwen36_35b_a3b); any case; '.' and '-' work too (qwen3.6)
    $ veizik install qwen35
  3. Activate this device

    Approve the machine once; it receives a licence bound to it.

    activate
    # have the Qwen3.5-0.8B checkpoint on this machine first: activation ends with a short check generation that uses it
    $ export VZ_WEIGHTS=/data/models
    $ veizik activate
    # without a browser (SSH, servers): create a key and a CLI password on your account page, then
    $ veizik activate --key KEY --password-stdin < pwfile
  4. Run it

    The first commands for this model, as the package’s own documents give them.

    run
    $ veizik run qwen35_2b "The capital of France is" --tokens 32
    $ veizik chat qwen35_2b
    $ veizik serve qwen35_2b --weights /data/models/Qwen3.5-2B --port 8000
    # then POST http://127.0.0.1:8000/v1/chat/completions
Support

Where it runs.

Evidence and availability are separate: a verified device is not a download, and an available model is not verified on every device.

VerifiedVerifiedRun end to end on this exact device, on a published build.Architecture-supportedArchitecture-supportedVerified on another device of the same architecture; this device itself is untested.Expected, untestedExpected, untestedThe engine targets this architecture; nobody has run this model on it. No claim is made.In preparationIn preparationNo build is downloadable yet, and no figure is offered.Not offeredNot offeredThis family is not offered on this platform.UnknownUnknownNo information yet — the platform is announced with the release.
FamilyRTX 3080 TiRTX 3090RTX 4090RTX 5090RTX A5000A40A100 80GB PCIeA100-SXM4-80GBH100 80GB HBM3RTX PRO 4500 BlackwellM4 ProM1 UltraM1 MaxM2 Pro
Qwen3.5
all variants
Architecture-supportedVerifiedVerifiedVerifiedVerifiedArchitecture-supportedExpected, untestedExpected, untestedExpected, untestedArchitecture-supportedNot offeredNot offeredNot offeredNot offered

Availability by variant

Model idAvailabilityPlatformContextWeights in memory
qwen35-0.8bAvailableNVIDIA4,096—
qwen35-2bAvailableNVIDIA4,096—
qwen35-4bAvailableNVIDIA4,096—
qwen35-9bAvailableNVIDIA4,096—

Verified on

  • GeForce RTX 3090 · build cuda-qwen-1.1.3 · 2026-10-22 · published measurement
  • NVIDIA GeForce RTX 4090 · build cuda-qwen-1.1.3 · 2026-10-21 · published measurement
  • NVIDIA GeForce RTX 5090 · build cuda-qwen-1.1.3 · 2026-10-21 · published measurement
  • NVIDIA RTX A5000 · build cuda-qwen-1.1.3 · 2026-10-21 · published measurement

“Verified” means this family ran end to end on that exact device, on a published build. Architecture membership never upgrades a device to verified.

Variants

4 checkpoints

Model idSizeCheckpointOn diskContextWeights in memory
qwen35-0.8b0.8BQwen/Qwen3.5-0.8B———
qwen35-2b2BQwen/Qwen3.5-2B———
qwen35-4b4BQwen/Qwen3.5-4B———
qwen35-9b9BQwen/Qwen3.5-9B———
Performance

Our figures for this build

Each table shows the GPUs a model was measured on and one figure per measure: the default mode and the most common workload. Hover a figure for its build, date, runs and range; open All figures for every mode and workload. A figure marked earlier build is our own measurement of an earlier build of the engine, shown because the computation it exercises did not change in substance; it is replaced when the package you download is measured on that GPU. A figure whose build reads carried, measured on another build was measured on that earlier build of the same package and stands for the package named, because the change between the two did not alter the computation.

Qwen3.5-0.8B

GPUDecode tok/sDecode, one token per step tok/sPrompt pass msGPU memory in use GiBThroughput, many at once tok/s
RTX 3090967.5balanced · 20-token prompt795.7512-token promptearlier build4.2balanced · 20-token prompt2.27balanced3,1258 concurrent requests
RTX 40901,318balanced · 21-token prompt1,008512-token promptearlier build3.0balanced · 21-token prompt2.40balanced—
RTX 50901,529balanced · 21-token prompt—2.7balanced · 21-token prompt2.51balanced—
RTX A5000776.7balanced · 20-token prompt—4.9balanced · 20-token prompt2.22balanced—
A100-SXM4-80GB—688.1512-token promptearlier build———
H100 80GB HBM3—1,004512-token promptearlier build———
All 54 figures for Qwen3.5-0.8B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Decodebalanced20 / 128967.5 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodebalanced20 / 10241,052 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodefast20 / 128972.0 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodequality20 / 128880.8 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodesingle-token decoding512 / —795.7 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 3090Prompt passbalanced20 / 1284.2 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passbalanced21 / 10244.4 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 3090Prompt passfast20 / 1284.1 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passquality20 / 1284.1 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usebalanced— / —2.27 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usefast— / —2.27 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usequality— / —2.42 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput1 request at a timeveizik serve, 1 concurrent request, 128 tokens each531.9 tok/s3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each1,778 tok/s3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each3,125 tok/s3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token1 request at a timeveizik serve, 1 concurrent request, 128 tokens each24.6 ms3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each62.9 ms3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each92.6 ms3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 4090Decodebalanced21 / 1281,318 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodebalanced21 / 10241,439 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodefast21 / 1281,316 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodequality21 / 1281,156 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodesingle-token decoding512 / —1,008 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 4090Prompt passbalanced21 / 1283.0 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passbalanced21 / 10243.0 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passfast21 / 1283.0 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passquality21 / 1283.1 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usebalanced— / —2.40 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usefast— / —2.40 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usequality— / —2.55 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 1281,529 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 10241,676 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodefast21 / 1281,531 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodequality21 / 1281,438 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 1282.7 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 10242.7 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passfast21 / 1282.7 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passquality21 / 1282.9 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usebalanced— / —2.51 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usefast— / —2.51 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usequality— / —2.66 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced20 / 128776.7 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced921 / 1024916.1 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodefast20 / 128771.9 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodequality20 / 128714.7 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced20 / 1284.9 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced921 / 102428.6 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passfast20 / 1284.9 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passquality20 / 1284.6 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usebalanced— / —2.22 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usefast— / —2.22 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usequality— / —2.37 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
A100-SXM4-80GBDecodesingle-token decoding512 / —688.1 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
H100 80GB HBM3Decodesingle-token decoding512 / —1,004 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13

Qwen3.5-2B

GPUDecode tok/sDecode, one token per step tok/sPrompt pass msGPU memory in use GiB
RTX 3090586.5balanced · 20-token prompt435.9512-token promptearlier build6.6balanced · 20-token prompt3.48balanced
RTX 4090886.9balanced · 21-token prompt548.4512-token promptearlier build4.5balanced · 21-token prompt3.61balanced
RTX 50901,138balanced · 21-token prompt—3.8balanced · 21-token prompt3.72balanced
RTX A5000467.3balanced · 20-token prompt—8.4balanced · 20-token prompt3.43balanced
A100-SXM4-80GB—439.4512-token promptearlier build——
H100 80GB HBM3—660.2512-token promptearlier build——
All 48 figures for Qwen3.5-2B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Decodebalanced20 / 128586.5 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodebalanced20 / 1024560.4 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodefast20 / 128585.1 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodequality20 / 128542.7 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodesingle-token decoding512 / —435.9 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 3090Prompt passbalanced20 / 1286.6 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passbalanced21 / 10247.1 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 3090Prompt passfast20 / 1286.7 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passquality20 / 1286.9 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usebalanced— / —3.48 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usefast— / —3.48 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usequality— / —3.80 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 4090Decodebalanced21 / 128886.9 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodebalanced21 / 1024868.8 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodefast21 / 128895.1 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodequality21 / 128740.9 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodesingle-token decoding512 / —548.4 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 4090Prompt passbalanced21 / 1284.5 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passbalanced21 / 10244.4 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passfast21 / 1284.4 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passquality21 / 1284.7 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usebalanced— / —3.61 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usefast— / —3.61 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usequality— / —3.93 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 1281,138 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 10241,107 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodefast21 / 1281,137 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodequality21 / 1281,041 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 1283.8 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 10243.8 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passfast21 / 1283.8 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passquality21 / 1284.1 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usebalanced— / —3.72 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usefast— / —3.72 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usequality— / —4.03 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced20 / 128467.3 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced921 / 1024449.1 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodefast20 / 128464.6 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodequality20 / 128435.2 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced20 / 1288.4 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced921 / 102458.5 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passfast20 / 1288.4 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passquality20 / 1289.0 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usebalanced— / —3.43 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usefast— / —3.43 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usequality— / —3.75 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
A100-SXM4-80GBDecodesingle-token decoding512 / —439.4 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
H100 80GB HBM3Decodesingle-token decoding512 / —660.2 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13

Qwen3.5-4B

GPUDecode tok/sDecode, one token per step tok/sPrompt pass msGPU memory in use GiB
RTX 3090376.8balanced · 20-token prompt208.2512-token promptearlier build13.5balanced · 20-token prompt5.49balanced
RTX 4090531.8balanced · 21-token prompt247.5512-token promptearlier build8.7balanced · 21-token prompt5.62balanced
RTX 5090772.2balanced · 21-token prompt—6.9balanced · 21-token prompt5.73balanced
RTX A5000280.5balanced · 20-token prompt—17.2balanced · 20-token prompt5.44balanced
A100-SXM4-80GB—224.3512-token promptearlier build——
H100 80GB HBM3—365.1512-token promptearlier build——
All 48 figures for Qwen3.5-4B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Decodebalanced20 / 128376.8 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodebalanced20 / 1024397.8 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodefast20 / 128376.8 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodequality20 / 128342.1 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodesingle-token decoding512 / —208.2 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 3090Prompt passbalanced20 / 12813.5 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passbalanced21 / 102413.9 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 3090Prompt passfast20 / 12813.1 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passquality20 / 12814.0 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usebalanced— / —5.49 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usefast— / —5.49 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usequality— / —6.27 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 4090Decodebalanced21 / 128531.8 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodebalanced21 / 1024568.2 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodefast21 / 128535.4 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodequality21 / 128441.8 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodesingle-token decoding512 / —247.5 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 4090Prompt passbalanced21 / 1288.7 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passbalanced21 / 10248.6 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passfast21 / 1288.7 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passquality21 / 12810.5 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usebalanced— / —5.62 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usefast— / —5.62 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usequality— / —6.40 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 128772.2 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 1024822.2 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodefast21 / 128771.4 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodequality21 / 128670.5 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 1286.9 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 10246.9 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passfast21 / 1286.9 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passquality21 / 1287.9 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usebalanced— / —5.73 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usefast— / —5.73 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usequality— / —6.50 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced20 / 128280.5 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced921 / 1024228.6 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodefast20 / 128278.9 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodequality20 / 128256.5 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced20 / 12817.2 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced921 / 1024154.0 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passfast20 / 12817.2 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passquality20 / 12819.2 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usebalanced— / —5.44 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usefast— / —5.44 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usequality— / —6.22 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
A100-SXM4-80GBDecodesingle-token decoding512 / —224.3 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
H100 80GB HBM3Decodesingle-token decoding512 / —365.1 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13

Qwen3.5-9B

GPUDecode tok/sDecode, one token per step tok/sPrompt pass msGPU memory in use GiBThroughput, many at once tok/s
RTX 3090190.2balanced · 20-token prompt117.0512-token promptearlier build21.7balanced · 20-token prompt8.44balanced663.08 concurrent requests
RTX 4090347.0balanced · 21-token prompt154.9512-token promptearlier build16.8balanced · 21-token prompt8.57balanced—
RTX 5090498.7balanced · 21-token prompt242.4512-token promptearlier build11.1balanced · 21-token prompt8.67balanced—
RTX A5000151.0balanced · 20-token prompt—29.0balanced · 20-token prompt8.39balanced—
A100-SXM4-80GB—129.4512-token promptearlier build———
H100 80GB HBM3—241.6512-token promptearlier build———
All 55 figures for Qwen3.5-9B
GPUMeasureModeWorkloadValueRunsBuildDate
RTX 3090Decodebalanced20 / 128190.2 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodebalanced20 / 1024170.2 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodefast20 / 128190.4 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodequality20 / 128172.8 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Decodesingle-token decoding512 / —117.0 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 3090Prompt passbalanced20 / 12821.7 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passbalanced21 / 102422.4 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 3090Prompt passfast20 / 12821.8 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Prompt passquality20 / 12823.2 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usebalanced— / —8.44 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usefast— / —8.44 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090GPU memory in usequality— / —9.82 GiB1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput1 request at a timeveizik serve, 1 concurrent request, 128 tokens each101.4 tok/s3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each366.4 tok/s3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090Throughput8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each663.0 tok/s3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token1 request at a timeveizik serve, 1 concurrent request, 128 tokens each55.6 ms3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token4 concurrent requestsveizik serve, 4 concurrent requests, 128 tokens each176.2 ms3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 3090First token8 concurrent requestsveizik serve, 8 concurrent requests, 128 tokens each308.8 ms3cuda-qwen-1.1.3 · carried, measured on cuda-qwen-1.0.02026-10-22
RTX 4090Decodebalanced21 / 128347.0 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodebalanced21 / 1024303.4 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodefast21 / 128346.6 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodequality21 / 128272.8 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Decodesingle-token decoding512 / —154.9 tok/s2earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 4090Prompt passbalanced21 / 12816.8 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passbalanced21 / 102416.6 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passfast21 / 12816.5 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090Prompt passquality21 / 12821.6 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usebalanced— / —8.57 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usefast— / —8.57 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 4090GPU memory in usequality— / —9.95 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 128498.7 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodebalanced21 / 1024434.6 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodefast21 / 128498.5 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodequality21 / 128423.4 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Decodesingle-token decoding512 / —242.4 tok/s3earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
RTX 5090Prompt passbalanced21 / 12811.1 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passbalanced21 / 102411.1 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passfast21 / 12811.1 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090Prompt passquality21 / 12813.0 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usebalanced— / —8.67 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usefast— / —8.67 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX 5090GPU memory in usequality— / —10.05 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced20 / 128151.0 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodebalanced921 / 1024124.8 tok/s1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodefast20 / 128149.1 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Decodequality20 / 128137.8 tok/s2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced20 / 12829.0 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passbalanced921 / 1024269.9 ms1cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passfast20 / 12829.2 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000Prompt passquality20 / 12831.0 ms2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usebalanced— / —8.39 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usefast— / —8.39 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
RTX A5000GPU memory in usequality— / —9.77 GiB2cuda-qwen-1.1.3 · carried, measured on cuda-qwen-llm-1.0.0-be389a652026-10-21
A100-SXM4-80GBDecodesingle-token decoding512 / —129.4 tok/s2earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
H100 80GB HBM3Decodesingle-token decoding512 / —241.6 tok/s1earlier build Qwen previous build cf4687d9 (2026-10-13; decode kernels source-identical to the shipped package, recompiled)2026-10-13
Measurement conditions

Default board power (limit not recorded); no quiet gate; dedicated single-GPU cloud host, native Linux; driver 595.91.07. NVIDIA A100-SXM4-80GB; board power not recorded; dedicated single-GPU cloud host; no quiet gate; one run; a measurement of the shipped package on this card will replace it. NVIDIA A100-SXM4-80GB; board power not recorded; dedicated single-GPU cloud host; no quiet gate; two runs on separate pods; a measurement of the shipped package on this card will replace it. NVIDIA GeForce RTX 3090; dedicated single-GPU cloud host, native Linux; default board limit 370 W, never reached (no power throttle, GPU at or below 44 C); driver 580.95.05; quiet pod. NVIDIA GeForce RTX 3090; dedicated single-GPU cloud host, native Linux; default board limit 370 W, never reached (no power throttle, GPU at or below 44 C); driver 580.95.05; quiet re-run. NVIDIA GeForce RTX 3090; dedicated single-GPU cloud host; board power not recorded; no quiet gate. NVIDIA GeForce RTX 4090; dedicated single-GPU cloud host; board power not recorded; no quiet gate. NVIDIA GeForce RTX 4090; dedicated single-GPU cloud host; board power not recorded; runs on separate pods; no quiet gate. NVIDIA GeForce RTX 5090; dedicated single-GPU cloud host; board power not recorded; runs on separate pods; no quiet gate. NVIDIA H100 80GB HBM3 (SXM); board power not recorded; dedicated single-GPU cloud host; no quiet gate; one run; a measurement of the shipped package on this card will replace it. No quiet gate; dedicated single-GPU cloud host, native Linux; board power default (not recorded); driver not recorded in the log.

Licence

Weights and terms

Qwen3.5 weights are published by Alibaba under Apache-2.0 and are supplied by the user, not shipped. Veizik ships only the inference engine and neither redistributes nor relicenses the weights.

Full licence terms →

Other models linked in the sidebar:

Stable Diffusion 3.5: Powered by Stability AI HunyuanImage: Weights are not included. Model availability and use are governed by the model provider's license and regional restrictions.