A model fits when its weights, its context cache and the runtime’s own margin all fit in the memory left after the system takes its share. The cache is where the generic formula goes wrong: it treats every layer as full attention. gemma4-12b at 32k needs 2.31 GiB of cache, not the 12.00 GiB the usual formula returns.
That gap is not a rounding error. It is the difference between a calculator telling a person with a 12 GB card that the model will not run, and the model running.
Why the usual formula overstates the cache
The textbook expression multiplies two (for keys and values) by the layer count, by the number of key-value heads, by the head dimension, by the context length, by two bytes. It assumes every layer keeps every past token.
Models shipped in 2026 do not work that way. gemma4-12b declares 48 layers, and only 8 of them are full attention: the other 40 use a sliding window of 1024 tokens, so they never hold more than 1024 positions no matter how long the conversation gets. qwen3.8-27b declares 64 layers with 16 full and 48 linear. The pattern is in each model’s own config.json, published by the vendor, and it is what our calculator reads.
Counting layer by layer against counting them all as full attention, at 32k of context and a 16-bit cache:
| Model | Full / sliding layers | Window | Real cache | Generic formula | Overstated by |
|---|---|---|---|---|---|
gemma4-e4b |
7 / 35 | 512 | 0.47 GiB | 2.62 GiB | 5.6x |
gemma4-12b |
8 / 40 | 1024 | 2.31 GiB | 12.00 GiB | 5.2x |
gemma4-31b |
10 / 50 | 1024 | 5.78 GiB | 30.00 GiB | 5.2x |
qwen3.8-27b |
16 / 48 | linear | 2.00 GiB | 8.00 GiB | 4.0x |
qwen3.6-35b-a3b |
10 / 30 | linear | 0.62 GiB | 2.50 GiB | 4.0x |
gpt-oss-20b |
12 / 12 | 128 | 0.75 GiB | 1.50 GiB | 2.0x |
Computed by Local AI Scope from each vendor’s published config.json. Cache in FP16.
The overstatement grows with the context you ask for
The same model can look fine at 8k and impossible at 32k, and the size of the error moves too. gemma4-12b is overstated 3.7x at 8k and 5.2x at 32k, because the sliding layers stay pinned at 1024 tokens while the generic formula keeps scaling all 48 layers linearly. Quoting a single multiplier without saying at what context is how a correct number becomes a wrong one.
So the honest form of the claim is: at 32k, on the models above, the generic formula asks for between 2.0x and 5.6x the cache the architecture actually needs, depending on the model.
What “fits” has to include
The budget has to cover the weights, the cache and the runtime margin at once. A calculator that reports only the first is not answering the question:
- The weights, at the size of the file the author actually published. We use Q4_K_M as the reference, and where an author does not publish it the sheet drops to the nearest compression and says which one. Treating Q4_K_M as the floor is a declared editorial criterion, not a measurement: below it quality degrades, but by how much we have not measured. A size in one compression compared against a size in another is not a comparison.
- The context cache, computed from the declared attention pattern rather than assumed.
- The runtime margin and the system’s share. A 16 GB machine does not offer 16 GB. Our machine profiles carry the memory that is actually left, and where that figure is derived rather than published by the manufacturer, the cell says so.
Your language changes the answer
Context is paid in tokens, and the same document does not cost the same number of tokens in every language. Measured on our own parallel corpus with each model’s real tokenizer, the same four texts cost 1.29x in German with gpt-oss-20b and 1.62x with qwen3-8b. A document that needs 8k of context in English can need 13k in German, which moves the cache, which moves what fits.
What that does to the shortlist is visible on any machine that is not oversized. With a 12,000-word contract, a PC with an RTX 3060 12 GB fits 6 of the 18 models in English and 3 in German; a workstation with an RTX 4090 fits 14 and 10. Both figures come from the same catalogue and the same fit rule, changing only the language of the document.
The token cost by language page has the full table and lets you run your own text through it.
Check the arithmetic yourself
Every number above is reproducible without trusting us. The layer counts, the head dimensions and the sliding window come from the config.json each vendor publishes; the file sizes come from the GGUF repositories; the token ratios come from a versioned parallel corpus, sha256[:16] 2f6c96b6ea0b161f. The methodology page lists what is measured, what is derived and what is estimated, one label per field.
The speeds shown alongside a fit are estimated, from manufacturer bandwidth and assumed efficiencies. They never decide whether something fits; they do influence the recommendation when a machine is borderline, and every line where that happens says so.
Run it against your machine
Pick your machine, your document and your language, and the Model Finder computes the three-part budget above for all 18 models in the catalogue. It runs entirely in your browser: the document you describe never leaves it.
If you would rather start from the hardware, the machine profiles list what fits at 8k and at 32k for ten common configurations, including the ones where the answer is that nothing worth running fits. The runtime profiles cover the program that loads the file, which sets the margin this page budgets for.