Home›Models›nemotron-3.5-lightning-30b
nemotron-3.5-lightning-30b
Architecture mixture of experts52 LayersLicense openmdw-1.1Maximum context 256kVocabulary 131,072
At Q4_K_M it is a 23.73 GiB file, and it fits on 10 of the 18 machines we measure (those with the least memory: Mini-PC with an NPU and 32 GB LPDDR5X · Mac mini with M6 and 32 GB). Here the file is what fills the memory: even at its maximum context of 256k tokens, the context adds 1.50 GiB — and the same content in German takes ×1.40 the tokens of English.
Model size at Q4_K_M
Measured by Local AI Scope
Context memory at 8k tokens
Measured by Local AI Scope
What German costs versus English
Measured by Local AI Scope
Videos: nemotron-3.5-lightning-30b 1
The official videos we found most useful. Press play and it opens right here.
The videos live on YouTube, and nothing loads from there until you press play (no-cookie mode). The thumbnail is served by us.
Sizes of the real published files
These are the sizes of the files actually published by the author, added up when the model ships split into parts. Not an estimate from the parameter count.
| Compression | Model size |
|---|---|
Q8_0 |
32.60 GiB |
Q6_K |
31.95 GiB |
Q5_K_M |
25.11 GiB |
Q4_K_M |
23.73 GiB |
Q5_K_S |
23.06 GiB |
Q4_K_S |
21.61 GiB |
Q3_K_M |
18.46 GiB |
IQ4_XS |
17.62 GiB |
Context memory
Computed from the attention pattern this model declares layer by layer, not from the generic formula. That is why the figure is often far smaller than other calculators tell you.
- 8k tokens → 0.05 GiB
- 32k tokens → 0.19 GiB
- 128k tokens → 0.75 GiB
What it costs in each language
Measured by running this model’s own tokenizer over the same content in four languages. Tokens are what you pay for in context, in memory and in your cloud bill.
| Kind of text | EN | ES | FR | DE |
|---|---|---|---|---|
| Code | — | ×1.23 | ×1.31 | ×1.51 |
| Contracts | — | ×1.21 | ×1.25 | ×1.37 |
| Business email | — | ×1.19 | ×1.22 | ×1.31 |
| Support tickets | — | ×1.28 | ×1.36 | ×1.37 |
English is the baseline: every figure is how many times more tokens the same content costs in that language. Average across the whole corpus: DE ×1.40.
Where it fits
Fits means the model, its context memory and the working headroom all fit at the stated context, leaving the system its share. 10 of the 18 machine profiles tested (Q4_K_M, 8k tokens).
If you are willing to compress it more, it does run where the table says no: at Q4_K_S (21.61 GiB), up to 32k tokens of context, on Workstation with an RTX 4090 24 GB; at IQ4_XS (17.62 GiB), up to 64k tokens of context, on Mac with M4 Pro and 24 GB. Below Q4_K_M the model loses a little quality, and at Q3_K_M you can tell; that is why the table above sticks to Q4_K_M.
Before you hit download: as of 19 October 2026, llama.cpp, Ollama, LM Studio, mlx-lm and vLLM all open this model, with one catch that surprised us: none of the three “Q4_K_M” files we looked at fits on a 24 GB graphics card. llama.cpp opens it in its most recent stable releases, v0.6.0 (from 5 October 2026) and v0.7.0 (from 15 October 2026); heads-up: since v0.7.0 its server listens on port 9931 by default, not 8080 (the Docker images keep 8080). Ollama has offered it since 0.32.9 (ollama run nemotron-3.5-lightning), and its q4_K_M weighs 23.68 GiB, text only; LM Studio offers it with the GGUF from its community program (lmstudio-community), whose Q4_K_M weighs 22.83 GiB (we have not checked which version of its engine you need, so update it before you try). With the table’s file (23.73 GiB), Ollama’s or LM Studio’s, the RTX 4090 (23.20 GiB usable by our count) falls short even at 4k. Why does a “30B” weigh this much at 4 bits? In this model part of the expert tensors cannot use the Q4_K format and the file falls back to Q5_0 and Q8_0, as we read in its header: that is why the Q5_K_S (23.06 GiB) weighs less than the Q4_K_M. The good news: the RTX 4090 does hold the Q4_K_S (21.61 GiB) up to 32k and the IQ4_XS (17.62 GiB) up to 128k. On a Mac, mlx-lm runs it with the 4-bit version from mlx-community (16.55 GiB, prepared with mlx-lm 0.31.3): by our count, with it the Mac with M4 Pro and 24 GB reaches 128k. vLLM supports it too: with the original BF16 weights (61.31 GiB) it fits on no PC with a graphics card on our list; NVIDIA also publishes an NVFP4 version (20.08 GiB) with vLLM recipes, but we have not run the numbers for that one on our machines. And the context: the table stops at 256k, which is what its config.json declares; NVIDIA and Ollama announce up to 1M.