Decode that holds as the prompt grows.
A long prompt is only useful if the model stays fast while it reads it. Qwen3.6-35B-A3B on veizik keeps a near-flat decode rate out to 16K tokens on a consumer card, and past 128K on a datacenter one — these are rates measured at each length, not a spec-sheet maximum.
Nearly flat, a long way out.
Decode rate against prompt length — the shape is the point.
Qwen3.6-35B-A3B decode tokens per second as the prompt grows. On a consumer RTX 4090 the rate is nearly flat out to 16K tokens; on an H100 it holds past 128K. Development-build figures, shown as a record.
Pay for the prompt once.
- The long prompt is read once. The cost of the prompt is paid in the prefill; after that, each new token is decoded against the stored context, so a long prompt slows the first token, not the whole generation.
- Decode work grows gently. From 512 to 16,384 tokens of context the per-token rate drops by about a sixth on a consumer card — the curve bends, it does not fall off a cliff.
- It is a measured length, not a declared one. These are rates taken at each prompt length on real hardware, not a maximum printed on a spec sheet.
Short prompt, long prompt, compare.
The behaviour is easy to see: give it a short prompt, then a long one, and compare the decode rate the run prints. It stays close as the prompt grows.
# feed a long document as the prompt and generate from it $ veizik run qwen36_35b_a3b --prompt-file long_document.txt --max-new 256 # the run reports prefill time (paid once for the long prompt) and decode tok/s
Decode tok/s and prefill time are both printed by the run. The long-context figures above are development-build measurements, shown as a record; the released build is an early release. Method is on Benchmarks.