qwen3.5-9b
Architecture dense32 LayersLicense apache-2.0Maximum context 256kVocabulary 248,320
At Q4_K_M it is a 5.29 GiB file, and it fits on 18 of the 18 machines we measure (the one with the least memory: PC with an RTX 4060 8 GB). Watch the context, though: at its maximum of 256k tokens it takes 8.00 GiB, more than the file itself — and the same content in German takes ×1.31 the tokens of English.
Model size at Q4_K_M
Measured by Local AI Scope
Context memory at 8k tokens
Measured by Local AI Scope
What German costs versus English
Measured by Local AI Scope
Videos: qwen3.5-9b 2
The best walk-throughs we found. Press play and it opens right here.
The videos live on YouTube, and nothing loads from there until you press play (no-cookie mode). The thumbnail is served by us.
Sizes of the real published files
These are the sizes of the files actually published by the author, added up when the model ships split into parts. Not an estimate from the parameter count.
| Compression | Model size |
|---|---|
Q8_0 |
8.87 GiB |
Q6_K |
6.95 GiB |
Q5_K_M |
6.13 GiB |
Q5_K_S |
5.92 GiB |
Q4_K_M |
5.29 GiB |
Q4_K_S |
5.02 GiB |
IQ4_XS |
4.81 GiB |
Q3_K_M |
4.35 GiB |
This model also reads images. For that it needs a second file, the vision projector (mmproj-F16.gguf, 0.86 GiB), published next to the model and not included in the figures on this page: they count text only. If you are going to send it images, add it to the model size.
Context memory
Computed from the attention pattern this model declares layer by layer, not from the generic formula. That is why the figure is often far smaller than other calculators tell you.
- 8k tokens → 0.25 GiB
- 32k tokens → 1.00 GiB
- 128k tokens → 4.00 GiB
What it costs in each language
Measured by running this model’s own tokenizer over the same content in four languages. Tokens are what you pay for in context, in memory and in your cloud bill.
| Kind of text | EN | ES | FR | DE |
|---|---|---|---|---|
| Code | — | ×1.20 | ×1.30 | ×1.46 |
| Contracts | — | ×1.16 | ×1.23 | ×1.31 |
| Business email | — | ×1.12 | ×1.22 | ×1.17 |
| Support tickets | — | ×1.24 | ×1.40 | ×1.25 |
English is the baseline: every figure is how many times more tokens the same content costs in that language. Average across the whole corpus: DE ×1.31.
Where it fits
Fits means the model, its context memory and the working headroom all fit at the stated context, leaving the system its share. 18 of the 18 machine profiles tested (Q4_K_M, 8k tokens).
Before you hit download: as of 12 October 2026, every program on our list opens this model. Every stable release of llama.cpp includes it (on 16 October 2026 the latest was v0.7.0, from 15 October 2026), and so does every daily build since b7990; Ollama has offered it since 0.17.5; LM Studio has it in its own catalogue, which asks for at least 7 GB of system memory; mlx-lm runs it from 0.31.0, text only; and vLLM from 0.17.0, with the original BF16 weights (17.98 GiB) in graphics-card memory: of the PCs on our list with a graphics card, only the RTX 4090 and the server with two RTX 3090s have room for them, and on a Mac vLLM only runs on the CPU, experimentally. By default it thinks before it answers (a habit we would recommend to anyone), and its authors advise leaving it at least 128k tokens of context so it can: by the table above, the machines with 16 GB or more get there, the 12 GB cards stop at 64k and the 8 GB one at 16k. The table counts text only. To send it images you also load the vision projector (0.86 GiB), and Ollama loads it every time, even for text: its qwen3.5:9b is 6.6 GB with the projector included (on a Mac, Ollama runs its MLX version, 7.6 GB). With the projector, the 16 GB machines reach 64k instead of 128k, the 12 GB cards stay at 64k, and the RTX 4060 8 GB reaches 8k with 0.10 GiB or less to spare, too little to rely on; from 24 GB up, nothing changes: 256k.