Home›Hardware›Workstation with an RTX 4090 24 GB
Workstation with an RTX 4090 24 GB
NVIDIA graphics card24 GiB1,008 GB/s82.6 TFLOPS
Once the system takes its share, this machine leaves 23.20 GiB for the model: room for 17 of the 26 models we measure at the reference compression, and the heaviest one that fits is qwen3.6-35b-a3b. And how fast does it answer? That’s not down to the chip, but to its 1,008 GB/s of memory bandwidth.
Left for the model
Estimated
Memory bandwidth
Vendor declared
Models that fit
Measured by Local AI Scope
Videos: Workstation with an RTX 4090 24 GB 2
The best walk-throughs we found. Press play and it opens right here.
The videos live on YouTube, and nothing loads from there until you press play (no-cookie mode). The thumbnail is served by us.
The memory you can actually use
This is why the number on the box is not the number you get. The desktop, the compositor and the graphics context never give their share back, and a model that does not leave them room does not load — it crashes.
| Memory | GiB |
|---|---|
| Memory on the box | 24 |
| Reserved for the system | −0.80 |
| Left for the model | 23.20 |
Memory bandwidth
Two different limits, and they are easy to mix up. While the model is writing its answer, bandwidth rules: every token means reading the weights it uses again — all of them in a dense model; in a mixture of experts, the shared layers plus only the active experts — so tokens per second is roughly bandwidth divided by that amount. While the model is reading your document, compute rules: the whole input is processed at once. That is why a laptop with no GPU can take minutes before the first word appears and then type at a tolerable pace.
1,008 GB/s — Vendor declared · checked against the manufacturer’s page on 2026-08-25 (Source).
Compute: 82.6 TFLOPS (FP32 shader TFLOPS) — Vendor declared, under a different name · the manufacturer publishes this number as FP32 shader TFLOPS, not as FP16; checked against the manufacturer’s page on 2026-08-25 (Source). The figures in this line do not all come from the same measure across machines, because each manufacturer publishes a different one. Compare them between machines only with that in mind.
Which models fit here, and which do not
Fitting means three things at once fit in the memory left over: the model file, its context memory, and the runtime’s working headroom. Context memory is computed from each model’s declared attention pattern layer by layer, not from the generic formula — which is why several models fit here that the generic formula would rule out.
| Model | Compression shown | Model size | 8k | 32k | 128k | Largest context that fits |
|---|---|---|---|---|---|---|
| qwen3-1.7b | Q4_K_M |
1.03 GiB | yes | yes | — | 32k |
| qwen3-4b-2507 | Q4_K_M |
2.33 GiB | yes | yes | no | 64k |
| gemma4-e2b | Q4_K_M |
2.89 GiB | yes | yes | yes | 128k |
| gemma4-e4b | Q4_K_M |
4.63 GiB | yes | yes | yes | 128k |
| qwen3-8b | Q4_K_M |
4.68 GiB | yes | yes | — | 32k |
| granite-4.2-8b | Q4_K_M |
4.98 GiB | yes | yes | no | 64k |
| qwen3.5-9b | Q4_K_M |
5.29 GiB | yes | yes | yes | 256k |
| gemma4-12b | Q4_K_M |
6.63 GiB | yes | yes | yes | 256k |
| gpt-oss-20b | Q4_K_M |
10.83 GiB | yes | yes | yes | 128k |
| mistral-small-24b | Q4_K_M |
13.35 GiB | yes | yes | no | 32k |
| qwen3.8-27b | Q4_K_M |
15.33 GiB | yes | yes | no | 64k |
| muse-glimmer-30b | Q4_K_M |
15.61 GiB | yes | yes | yes | 128k |
| qwen3.6-27b | Q4_K_M |
15.66 GiB | yes | yes | no | 64k |
| gemma4-26b-a4b | Q4_K_M |
15.78 GiB | yes | yes | yes | 128k |
| gemma4-31b | Q4_K_M |
17.07 GiB | yes | yes | no | 32k |
| qwen3-coder-30b | Q4_K_M |
17.28 GiB | yes | yes | no | 32k |
| qwen3.6-35b-a3b | Q4_K_M |
20.61 GiB | yes | yes | no | 32k |
| flint-40b | Q4_K_M |
23.61 GiB | no | no | no | — |
| nemotron-3.5-lightning-30b | Q4_K_M |
23.73 GiB | no | no | no | — |
| gpt-oss-120b | Q4_K_M |
58.46 GiB | no | no | no | — |
| llama4-scout-17b | Q4_K_M |
60.87 GiB | no | no | no | — |
| nemotron-3.5-super-120b | Q4_K_M |
65.29 GiB | no | no | no | — |
| qwen3.8-flash-next | IQ4_XS |
87.25 GiB | no | no | no | — |
| deepseek-v4-flash | IQ4_XS |
127.28 GiB | no | no | no | — |
| glm-5.3-flash | Q4_K_M |
187.04 GiB | no | no | no | — |
| glm-5.2 | Q4_K_M |
433.83 GiB | no | no | no | — |
Sizes are the real published files at the reference compression, one step per row: Q4_K_M where the author publishes it, the nearest neighbour where they do not. Every row states which one it is showing. Q4_K_M is our quality floor: below it the loss is audible in the answers. A model that would only fit here at a harsher compression is listed as not fitting, on purpose. A dash in a context column means the model itself does not offer that context, so there is nothing to fit.
Models that fit — 17 of the 26 models measured
- qwen3-1.7b
Q4_K_M— context 32k tokens - qwen3-4b-2507
Q4_K_M— context 64k tokens - gemma4-e2b
Q4_K_M— context 128k tokens - gemma4-e4b
Q4_K_M— context 128k tokens - qwen3-8b
Q4_K_M— context 32k tokens - granite-4.2-8b
Q4_K_M— context 64k tokens - qwen3.5-9b
Q4_K_M— context 256k tokens - gemma4-12b
Q4_K_M— context 256k tokens - gpt-oss-20b
Q4_K_M— context 128k tokens - mistral-small-24b
Q4_K_M— context 32k tokens - qwen3.8-27b
Q4_K_M— context 64k tokens - muse-glimmer-30b
Q4_K_M— context 128k tokens - qwen3.6-27b
Q4_K_M— context 64k tokens - gemma4-26b-a4b
Q4_K_M— context 128k tokens - gemma4-31b
Q4_K_M— context 32k tokens - qwen3-coder-30b
Q4_K_M— context 32k tokens - qwen3.6-35b-a3b
Q4_K_M— context 32k tokens
Models that do not fit — 9 of the 26 models measured
- flint-40b
Q4_K_M— short by 1.22 GiB - nemotron-3.5-lightning-30b
Q4_K_M— short by 1.23 GiB - gpt-oss-120b
Q4_K_M— short by 36.08 GiB - llama4-scout-17b
Q4_K_M— short by 39.10 GiB - nemotron-3.5-super-120b
Q4_K_M— short by 42.80 GiB - qwen3.8-flash-next
IQ4_XS— short by 64.82 GiB - deepseek-v4-flash
IQ4_XS— short by 105.09 GiB - glm-5.3-flash
Q4_K_M— short by 164.60 GiB - glm-5.2
Q4_K_M— short by 411.92 GiB
Willing to compress a bit more? Then not everything on this list is out of reach. In the 23.20 GiB of VRAM left for the model, flint-40b runs at Q4_K_S (22.19 GiB) with up to 8k tokens of context, and nemotron-3.5-lightning-30b runs at Q4_K_S (21.61 GiB) with up to 32k tokens of context. Below the compression in the table a model loses quality; that is why our count sticks to the table.
The machines on either side
- Office laptop, CPU only, 16 GB — 13.00 GiB left for the model, 9 / 26 models
- Mac with M4 Pro and 24 GB — 21.00 GiB left for the model, 16 / 26 models
- Mac mini with M6 and 32 GB — 29.00 GiB left for the model, 19 / 26 models
- Mini-PC with an NPU and 32 GB LPDDR5X — 29.00 GiB left for the model, 19 / 26 models
Price and availability
What a machine costs decides as much as what fits inside it, so the price belongs on this page. What follows is only what the manufacturer publishes, with the market it applies to and the day it was read.
There is no price: this row is a class of machine, not a single product with a price list.
What we earn on this page
Nothing, today. There is no affiliate link on this page and no commercial agreement behind any figure on it. If a buying link ever appears here, it will be marked as such and the commission declared in this same block. Two rules do not change on that day: the ranking on this site is computed before commercial availability is applied, and «best» never means «pays most».