Home›Hardware›Server with 2× RTX 3090 (48 GB)
Server with 2× RTX 3090 (48 GB)
NVIDIA graphics card48 GiB936 GB/s71.0 TFLOPS
Once the system takes its share, this machine leaves 47.20 GiB for the model: room for 19 of the 26 models we measure at the reference compression, and the heaviest one that fits is nemotron-3.5-lightning-30b. And how fast does it answer? That’s not down to the chip, but to its 936 GB/s of memory bandwidth.
Left for the model
Estimated
Memory bandwidth
Vendor declared
Models that fit
Measured by Local AI Scope
Videos: Server with 2× RTX 3090 (48 GB) 3
The best walk-throughs we found. Press play and it opens right here.
The videos live on YouTube, and nothing loads from there until you press play (no-cookie mode). The thumbnail is served by us.
The memory you can actually use
This is why the number on the box is not the number you get. The desktop, the compositor and the graphics context never give their share back, and a model that does not leave them room does not load — it crashes.
| Memory | GiB |
|---|---|
| Memory on the box | 48 |
| Reserved for the system | −0.80 |
| Left for the model | 47.20 |
Those 48 GiB are two separate pools of 24 GiB, not one. They only add up when the runtime splits the model across both cards by layers; a model that has to sit on a single card still faces a 24 GiB ceiling. The fit figures below assume the split works.
Memory bandwidth
Two different limits, and they are easy to mix up. While the model is writing its answer, bandwidth rules: every token means reading the weights it uses again — all of them in a dense model; in a mixture of experts, the shared layers plus only the active experts — so tokens per second is roughly bandwidth divided by that amount. While the model is reading your document, compute rules: the whole input is processed at once. That is why a laptop with no GPU can take minutes before the first word appears and then type at a tolerable pace.
936 GB/s — Vendor declared · checked against the manufacturer’s page on 2026-08-25 (Source).
The 936 GB/s is per card. Two 3090s do not pool into 48 GB at 1872 GB/s: each has its own 24 GB, and when a model is split across both, the link between them is the bottleneck, not the bandwidth of either one.
Compute: 71.0 TFLOPS (FP16 Tensor TFLOPS with FP32 accumulate, dense) — Vendor declared, under a different name · the manufacturer publishes this number as FP16 Tensor TFLOPS with FP32 accumulate, dense, not as FP16; checked against the manufacturer’s page on 2026-08-25 (Source). The figures in this line do not all come from the same measure across machines, because each manufacturer publishes a different one. Compare them between machines only with that in mind.
Which models fit here, and which do not
Fitting means three things at once fit in the memory left over: the model file, its context memory, and the runtime’s working headroom. Context memory is computed from each model’s declared attention pattern layer by layer, not from the generic formula — which is why several models fit here that the generic formula would rule out.
| Model | Compression shown | Model size | 8k | 32k | 128k | Largest context that fits |
|---|---|---|---|---|---|---|
| qwen3-1.7b | Q4_K_M |
1.03 GiB | yes | yes | — | 32k |
| qwen3-4b-2507 | Q4_K_M |
2.33 GiB | yes | yes | yes | 256k |
| gemma4-e2b | Q4_K_M |
2.89 GiB | yes | yes | yes | 128k |
| gemma4-e4b | Q4_K_M |
4.63 GiB | yes | yes | yes | 128k |
| qwen3-8b | Q4_K_M |
4.68 GiB | yes | yes | — | 32k |
| granite-4.2-8b | Q4_K_M |
4.98 GiB | yes | yes | yes | 128k |
| qwen3.5-9b | Q4_K_M |
5.29 GiB | yes | yes | yes | 256k |
| gemma4-12b | Q4_K_M |
6.63 GiB | yes | yes | yes | 256k |
| gpt-oss-20b | Q4_K_M |
10.83 GiB | yes | yes | yes | 128k |
| mistral-small-24b | Q4_K_M |
13.35 GiB | yes | yes | yes | 128k |
| qwen3.8-27b | Q4_K_M |
15.33 GiB | yes | yes | yes | 256k |
| muse-glimmer-30b | Q4_K_M |
15.61 GiB | yes | yes | yes | 128k |
| qwen3.6-27b | Q4_K_M |
15.66 GiB | yes | yes | yes | 256k |
| gemma4-26b-a4b | Q4_K_M |
15.78 GiB | yes | yes | yes | 256k |
| gemma4-31b | Q4_K_M |
17.07 GiB | yes | yes | yes | 256k |
| qwen3-coder-30b | Q4_K_M |
17.28 GiB | yes | yes | yes | 256k |
| qwen3.6-35b-a3b | Q4_K_M |
20.61 GiB | yes | yes | yes | 256k |
| flint-40b | Q4_K_M |
23.61 GiB | yes | yes | yes | 256k |
| nemotron-3.5-lightning-30b | Q4_K_M |
23.73 GiB | yes | yes | yes | 256k |
| gpt-oss-120b | Q4_K_M |
58.46 GiB | no | no | no | — |
| llama4-scout-17b | Q4_K_M |
60.87 GiB | no | no | no | — |
| nemotron-3.5-super-120b | Q4_K_M |
65.29 GiB | no | no | no | — |
| qwen3.8-flash-next | IQ4_XS |
87.25 GiB | no | no | no | — |
| deepseek-v4-flash | IQ4_XS |
127.28 GiB | no | no | no | — |
| glm-5.3-flash | Q4_K_M |
187.04 GiB | no | no | no | — |
| glm-5.2 | Q4_K_M |
433.83 GiB | no | no | no | — |
Sizes are the real published files at the reference compression, one step per row: Q4_K_M where the author publishes it, the nearest neighbour where they do not. Every row states which one it is showing. Q4_K_M is our quality floor: below it the loss is audible in the answers. A model that would only fit here at a harsher compression is listed as not fitting, on purpose. A dash in a context column means the model itself does not offer that context, so there is nothing to fit.
Models that fit — 19 of the 26 models measured
- qwen3-1.7b
Q4_K_M— context 32k tokens - qwen3-4b-2507
Q4_K_M— context 256k tokens - gemma4-e2b
Q4_K_M— context 128k tokens - gemma4-e4b
Q4_K_M— context 128k tokens - qwen3-8b
Q4_K_M— context 32k tokens - granite-4.2-8b
Q4_K_M— context 128k tokens - qwen3.5-9b
Q4_K_M— context 256k tokens - gemma4-12b
Q4_K_M— context 256k tokens - gpt-oss-20b
Q4_K_M— context 128k tokens - mistral-small-24b
Q4_K_M— context 128k tokens - qwen3.8-27b
Q4_K_M— context 256k tokens - muse-glimmer-30b
Q4_K_M— context 128k tokens - qwen3.6-27b
Q4_K_M— context 256k tokens - gemma4-26b-a4b
Q4_K_M— context 256k tokens - gemma4-31b
Q4_K_M— context 256k tokens - qwen3-coder-30b
Q4_K_M— context 256k tokens - qwen3.6-35b-a3b
Q4_K_M— context 256k tokens - flint-40b
Q4_K_M— context 256k tokens - nemotron-3.5-lightning-30b
Q4_K_M— context 256k tokens
Models that do not fit — 7 of the 26 models measured
- gpt-oss-120b
Q4_K_M— short by 12.08 GiB - llama4-scout-17b
Q4_K_M— short by 15.10 GiB - nemotron-3.5-super-120b
Q4_K_M— short by 18.80 GiB - qwen3.8-flash-next
IQ4_XS— short by 40.82 GiB - deepseek-v4-flash
IQ4_XS— short by 81.09 GiB - glm-5.3-flash
Q4_K_M— short by 140.60 GiB - glm-5.2
Q4_K_M— short by 387.92 GiB
The machines on either side
- Mac mini with M6 and 32 GB — 29.00 GiB left for the model, 19 / 26 models
- Mini-PC with an NPU and 32 GB LPDDR5X — 29.00 GiB left for the model, 19 / 26 models
- Mac with M4 Max and 64 GB — 61.00 GiB left for the model, 20 / 26 models
- Mac mini with M5 Pro and 64 GB — 61.00 GiB left for the model, 20 / 26 models
Price and availability
What a machine costs decides as much as what fits inside it, so the price belongs on this page. What follows is only what the manufacturer publishes, with the market it applies to and the day it was read.
There is no price: this row is a class of machine, not a single product with a price list.
What we earn on this page
Nothing, today. There is no affiliate link on this page and no commercial agreement behind any figure on it. If a buying link ever appears here, it will be marked as such and the commission declared in this same block. Two rules do not change on that day: the ranking on this site is computed before commercial availability is applied, and «best» never means «pays most».