Home›Guides›What AI your computer can run, and what you give up to run it

Choose · Guide

What AI your computer can run, and what you give up to run it

On an 8 GB card, 7 of the 28 models we measure fit alongside an eight-page document, and the biggest is 5.29 GiB. The largest that fits on any machine we list is 433.83 GiB. That is 82x, and it is the honest starting point.

Measured and written by Local AI ScopePublished Figures measured by Local AI Scope

Let’s start with the good news: your computer can almost certainly run a local AI, and the first time one answers you from your own desk, it feels a little bit like magic. The other half: what it cannot run is the one you have been using. On an 8 GB graphics card, 7 of the 28 models we measure fit alongside an eight-page document, and the biggest of them is 5.29 GiB; the largest model that fits on any machine we list is 433.83 GiB. Everything else on this page follows from that gap.

That is not a reason to skip local AI, though. It is the reason to know what you are trading before you spend a weekend installing something.

What actually changes when the model runs on your machine

Four things change, and only four.

Nothing you type leaves the machine. Not the document, not the prompt, not the draft you decided not to send after all. There is no processor agreement to sign, no international-transfer clause to read, no training opt-out to take on faith. For anything covered by GDPR, that removes the paperwork rather than mitigating it.

The cost stops being per use. You pay once for hardware and electricity. No taxi meter ticking away while you paste a long contract in and ask for a summary five times.

It works with the network unplugged. Once the file is on disk, a flight, a rural office or a locked-down network stops being a blocker.

And it is smaller and slower than what you are used to. That is exactly the part nobody puts next to the price, so the rest of this page is about putting numbers on it.

Mac mini with M6 connected to a display
Mac mini with M6 connected to a display Photo: Apple

How we count, before any number

Every figure here uses one compression per model: Q4_K_M where the author publishes it, the nearest neighbour where they do not. Q4_K_M is our quality floor, because below it the loss shows in the answers.

How much smaller, exactly

A local model is a file, and it has to fit in the memory left after the operating system takes its share — together with the memory the conversation itself occupies and the runtime’s own working margin. We reserve 0.8 GiB on dedicated graphics cards and 3 GiB on unified or system memory. On PCs with Ryzen AI Max+ a different limit applies: AMD lets the GPU use at most 96 GiB of 128 GB and 160 GiB of 192 GB.

For a 4,000-word document in English:

Machine Memory for the model Models that fit Biggest that fits Size Distance to glm-5.2
PC with an RTX 4060 8 GB 7.20 GiB 7 of 28 qwen3.5-9b 5.29 GiB 82x
PC with an RTX 3060 12 GB 11.20 GiB 9 of 28 phi-4 8.43 GiB 51x
PC with an RTX 4070 12 GB 11.20 GiB 9 of 28 phi-4 8.43 GiB 51x
Mac with M4 and 16 GB unified 13.00 GiB 10 of 28 gpt-oss-20b 10.83 GiB 40x
Laptop, 16 GB, no graphics card 13.00 GiB 10 of 28 gpt-oss-20b 10.83 GiB 40x
Mac mini with M6 and 16 GB 13.00 GiB 10 of 28 gpt-oss-20b 10.83 GiB 40x
Mac with M4 Pro and 24 GB 21.00 GiB 17 of 28 qwen3-coder-30b 17.28 GiB 25x
Workstation with an RTX 4090 24 GB 23.20 GiB 18 of 28 qwen3.6-35b-a3b 20.61 GiB 21x
Mini-PC with an NPU and 32 GB 29.00 GiB 20 of 28 nemotron-3.5-lightning-30b 23.73 GiB 18x
Mac mini with M6 and 32 GB 29.00 GiB 20 of 28 nemotron-3.5-lightning-30b 23.73 GiB 18x
Server with 2× RTX 3090 (48 GB) 47.20 GiB 20 of 28 nemotron-3.5-lightning-30b 23.73 GiB 18x
Mac with M4 Max and 64 GB 61.00 GiB 21 of 28 gpt-oss-120b 58.46 GiB 7x
Mac mini with M5 Pro and 64 GB 61.00 GiB 21 of 28 gpt-oss-120b 58.46 GiB 7x
Xiaomi AI Cube, 80 GB (prototype) 77.00 GiB 23 of 28 nemotron-3.5-super-120b 65.29 GiB 7x
PC with Ryzen AI Max+ 395 and 128 GB 96.00 GiB 24 of 28 qwen3.8-flash-next 87.25 GiB 5x
Mac Studio with M5 Max and 128 GB 125.00 GiB 24 of 28 qwen3.8-flash-next 87.25 GiB 5x
PC with Ryzen AI Max+ PRO 495 and 192 GB 160.00 GiB 25 of 28 deepseek-v4-flash 127.28 GiB 3x
Mac Studio with M5 Ultra and 512 GB 509.00 GiB 27 of 28 glm-5.2 433.83 GiB 1x

Calculated by Local AI Scope from published file sizes and each model’s real attention pattern, every row at the same reference compression so the sizes are comparable. Sizes come from the authors’ own published builds: glm-5.2, phi-4, gpt-oss-20b. glm-5.2 is 433.83 GiB in Q4_K_M; the least compressed build we catalogue (Q8_0) is 746.32 GiB.

What matters most in that table is not the ratio; it is two things you read between the lines. The first: the ladder only reaches parity at the very top. A 64 GB Mac, which is hardly a cheap machine, is still 7 times short of the largest weights that fit on any machine here, 128 GB only brings that down to 5, 192 GB to 3, and the only machine here that reaches 27 of 28 is the Mac Studio with M5 Ultra and 512 GB. mistral-large-4 (591.21 GiB) fits on none of them.

Mac Studio with M5 Ultra and 512 GBMac Studio with M5 Ultra and 512 GBHardware sheet
Mac Studio with LM Studio on screen
Mac Studio with LM Studio on screen Photo: Apple

How slow is slow

Speed splits into two numbers that behave differently, and mixing them up is why people are surprised.

Writing the answer is limited by memory bandwidth. Reading your document first — before a single word comes back — is limited by raw compute. In a short chat the second number goes unnoticed; in a long document it is the only number left.

Same 4,000-word English document, best fitting model for that job on each machine:

Machine Model Writes at Waits before the first word
Laptop, 16 GB, no graphics card phi-4 ~3.2 tokens/s ~9 min 42 s
Mac with M4 and 16 GB unified phi-4 ~7.3 tokens/s ~1 min 48 s
PC with an RTX 3060 12 GB phi-4 ~26 tokens/s ~31 s
PC with an RTX 4060 8 GB qwen3.5-9b ~31 tokens/s ~16 s
Workstation with an RTX 4090 24 GB qwen3.6-35b-a3b ~213 tokens/s ~2 s

Estimated by Local AI Scope, not measured: the formula uses each machine’s declared memory bandwidth and compute, with conservative efficiency assumptions. It is published so you can check it, and it never decides whether a model fits.

Apple M4 chip
Apple M4 chip Image: Apple

Now the same machines with a normal chat message of about 1,000 words. The RTX 4060 waits around 4 seconds and writes at roughly 31 tokens/s. The 16 GB laptop with no graphics card waits about 16 seconds and writes at roughly 17 tokens/s.

PC with an RTX 4060 8 GBHardware sheet

Is a local AI good enough to replace ChatGPT for my day-to-day work?

Here it pays to split the question in two, because each half has its own answer.

Short jobs that fit in a page or two — drafting, rewriting, summarising, questions about a document, working offline: this is exactly the kind of job a model on a normal machine is built for, and the privacy and the flat cost are real wins.

Long-document reasoning, code across a whole repository, anything where you were leaning on a frontier model: short of a 512 GB Mac Studio, the file on your disk is 3 to 82 times smaller than the largest one that fits on any machine here, depending on your machine, and no setting closes that gap.

Opens on YouTube (no cookies) only when you press play. More videos on its sheet →

What that size difference costs you in answer quality, we have not measured, and we are not going to put a number on it.

Anyone who tells you their local model is “as good as GPT in German” without publishing how they measured it is guessing.

One thing that does change the answer: the language you work in

This is one of our favourite parts to explain. The same text is not the same number of tokens in every language, and context is reserved in tokens. Measured with each model’s own tokenizer over a parallel corpus of ours (sha256[:16] 2f6c96b6ea0b161f), a contract costs up to 1.77x more tokens in German than in English, up to 1.47x in French and up to 1.40x in Spanish, depending on the model.

That is no rounding error, because the runtime reserves context in powers of two: cross a step and memory jumps. A bit like checked luggage: everything is fine until the scale tips past the limit. On an RTX 4060, a German document starts costing you a model at 2,800 words, a Spanish one at 4,200 and a French one at 4,200. On the 16 GB machines the same crossing happens at 5,700 words in German, 8,400 in Spanish and 8,400 in French. The 12 GB cards cross at the same points as the RTX 4060. From 16 GB up, every machine crosses at 5,700, 8,400 and 8,400, the 512 GB one included, and the model that drops there is phi-4: at those lengths the document already asks for more than the 16k of context phi-4 accepts. No memory is missing there; its context runs out. The token calculator has the per-model figures.

If you are going to buy something, buy memory

Two rules hold up in every scenario we have calculated.

  • Memory decides what fits; bandwidth decides how fast it runs. An RTX 4070 12 GB and an RTX 3060 12 GB fit exactly the same models. The 4070 runs them faster and adds none.
  • The file size is the specification, not the parameter count. gemma4-12b is 6.63 GiB and qwen3-coder-30b is 17.28 GiB: the second will not fit on any 12 GB card, whatever its name promises.
PC with an RTX 4070 12 GBHardware sheet
PC with an NVIDIA GeForce RTX 3060
PC with an NVIDIA GeForce RTX 3060 Photo: NVIDIA

The hardware pages carry the two lists for each machine — what fits and what does not — with the memory and bandwidth each one actually has, counted the same way as this page. We calculate that list before we look at what is on sale, and we say so in our editorial policy: a machine never ranks higher here because it is easier to buy.

Three questions, and the tool answers them

The model finder needs three answers and runs entirely in your browser — what you type into it never reaches us.

  1. What machine do you have? Memory and type, not brand. Start from your own: RTX 4060 8 GB · RTX 3060 12 GB · Mac M4 16 GB · laptop with no graphics card · RTX 4090 24 GB.
  2. What are you feeding it, how long, and in what language? A 600-word email and a 12,000-word contract are different machines’ problems, and the language moves the answer.
  3. What is in that data? Nothing sensitive, company-internal, personal data, or must-not-leave. That last one is the only answer that makes the verdict a requirement rather than a preference.
Mac with M4 and 16 GB unified memoryMac with M4 and 16 GB unified memoryHardware sheet Workstation with an RTX 4090 24 GBWorkstation with an RTX 4090 24 GBHardware sheet

Opens on YouTube (no cookies) only when you press play. More videos on its sheet →

One difference worth knowing: the tool will also show you what fits if you allow harsher compression than Q4_K_M, and it labels those clearly. The counts on this page never do.