llama.cpp not working? 8 errors, the exact message and the fix

When llama.cpp fails, the log usually names the culprit: a busy port (8080 or 9931, by build), a build without GPU support, a model newer than your build, a file that is not a GGUF, memory running out, a context too big, an unreadable chat template, or an old CMake flag. Here are the 8, with the fix.

llama.cpp is the engine under many local AI programs, and running it directly feels like taking the lid off the machine: total control, every flag in your hands. The price is that when something goes wrong, nobody translates the message for you. Every message below was checked on 15 and 16 October 2026 against the code of llama.cpp v0.7.0, the stable release of 15 October (build b11669), and of the daily build b11658. We did not trigger these errors on a machine today: they come straight from the source, line by line. Installing it and loading the first model are covered in our llama.cpp setup guide; this page picks up where that one ends.

llama.cppRuntime sheet→

“couldn’t bind HTTP server socket”: the port is taken (and which port is it?)

You start the server and it stops right away with:

… start: couldn't bind HTTP server socket, hostname: 127.0.0.1, port: 9931

Something else is already listening on that address and port: a second copy of llama.cpp you forgot in another terminal, a program that uses the same port, or the previous run that has not finished closing. The fix is either to stop the other process or to pick another port with --port 8081 (any free number works).

Here comes the twist that catches people this month. The default port changed. Up to v0.6.0, the version Homebrew installed until 16 October 2026, the server listens on 8080. Since daily build b11521 (9 October 2026), and in the stable v0.7.0 (15 October), it listens on 9931 by default. If you installed a daily build or v0.7.0 (and that includes brew install or brew upgrade since 16 October), your browser bookmark and any app pointing at http://127.0.0.1:8080 now knock on an empty door, and nothing in the error tells you so. Two ways out: start it with --port 8080 to keep the old address, or update your apps to 9931. Once the model has loaded, the log prints a line like listening on http://127.0.0.1:9931: trust that line, not your memory.

On Linux, ss -ltnp | grep 9931 (or 8080) shows who holds the port; on macOS, lsof -i :9931; on Windows, netstat -ano | findstr 9931.

“warning: no usable GPU found, --gpu-layers option will be ignored”

You asked for layers on the card with -ngl and llama.cpp answers:

warning: no usable GPU found, --gpu-layers option will be ignored
warning: one possible reason is that llama.cpp was compiled without GPU support
warning: consult docs/build.md for compilation instructions

The message is honest: most of the time you are running a CPU-only build. llama.cpp publishes a different package per accelerator: CPU, CUDA, Vulkan, ROCm, SYCL, OpenVINO on Windows and Linux, and Metal on Apple Silicon Macs. If you downloaded the “cpu” zip, the card is invisible by design. Download the package that matches your card (on NVIDIA, the CUDA one together with its matching cudart package), or rebuild with the right option (see the CMake section below).

When the card is seen, the log says so early, with a line like ggml_cuda_init: found 1 CUDA devices (Total VRAM: …). No line like that and no warning either? Then you did not pass -ngl (its default is auto), so the warning never had a chance to show up, and you are most likely on a CPU-only build. Run it once with -ngl all: if the build has no GPU support, the warning above will say so.

Two cases where the right package is still not enough. On an Intel Mac, the official binary is built without Metal, so it runs on the CPU only. And on an AMD card, the CUDA package runs on the CPU only: you need the ROCm or the Vulkan package.

“unknown model architecture”: the model is newer than your build

The model downloads, then loading stops with:

llama_model_load: error loading model: unknown model architecture: 'xxxx'

followed by failed to load model. The GGUF file says which family of model it is, and your build does not know that family yet. It is not a broken file: it is a calendar problem. llama.cpp adds support for new families almost every week, and a model released last Tuesday needs a build from after last Tuesday.

The fix is to update. With llama.cpp publishing about 16 builds a day on average (481 in the 30 days to 23 October 2026), “the latest” is a moving target, which is good news here: the build that knows your model probably already exists. On Homebrew you get the stable channel, which can lag a few weeks behind the daily builds; if the model is very new, take a daily build from the releases page or the installer. If the newest build still does not know the name, the model is not supported yet: pick another quantization of a supported family in the meantime.

“invalid magic characters … expected ‘GGUF'” or “failed to load model”

This one is short:

gguf_init_from_reader: invalid magic characters: 'xxxx', expected 'GGUF'

Every GGUF file starts with the 4 letters GGUF. If they are not there, llama.cpp is not reading a GGUF. The usual suspects: you pointed it at a .safetensors file or a whole Hugging Face folder (llama.cpp loads only GGUF), you saved the download page instead of the file, or the download stopped halfway. If you get failed to read magic instead, the file is empty or missing.

The generic failed to load model from … and failed to load model '…' always come with a more precise line just above them: read that one. A path with a typo is a common cause.

The easy way around all of this is to let llama.cpp fetch the model itself: -hf user/model:Q4_K_M downloads the GGUF from Hugging Face into the shared cache, with the right file, every time.

qwen3-8bModel sheet→

“cudaMalloc failed: out of memory” or “failed to allocate buffer for kv cache”

On an NVIDIA card, running out of memory looks like this:

ggml_backend_cuda_buffer_type_alloc_buffer: allocating … MiB on device 0: cudaMalloc failed: out of memory

and sometimes, right after the model loads:

failed to allocate buffer for kv cache

On a Mac, the same wall reads failed to allocate Metal buffer of … bytes (out of memory). The first number tells you how much it tried to reserve; the cause is one of three. The model weights do not fit on the card (a 14B model on 8 GB, for example). The context cache does not fit (more on that below). Or something else is using the card: a game, a browser with hardware acceleration, another copy of llama.cpp, Ollama in the background.

The fixes, easiest first: close the other programs; lower the context with -c 8192; keep fewer layers on the card with a smaller -ngl (the rest go to RAM, slower but working); store the cache in 8 bits with -ctk q8_0 -ctv q8_0; or switch to a smaller quantization of the same model. The Model Finder tells you in advance which model and quantization fits your card with the context you want.

“failed to fit params to free device memory”: the default context is the model’s maximum

This is the problem that does not look like a problem. llama.cpp starts, answers, and still something is off: it is slower than expected, or a long document comes back with request (… tokens) exceeds the available context size (… tokens), try increasing it. The cause is the default context. Out of the box, llama.cpp loads the full context the model was trained with (-c 0 means exactly that), and then a mechanism called --fit, on by default, adjusts things to the free memory: first it shrinks the context, down to a floor of 4,096 tokens, then it moves layers from the card to RAM, always leaving 1,024 MiB free per device. Each step is logged at trace level, so with the default log you do not see why; you only see the result, in a line like n_ctx_seq (32768) < n_ctx_train (40960) -- the full capacity of the model will not be utilized. If the fit itself fails, you get a warning:

failed to fit params to free device memory: …

The cure is to decide the context yourself instead of letting --fit decide for you. Pick the context your task needs (8,192 tokens are plenty for chat; long documents need more), write it with -c, and, if you want more of it on the same card, add -ctk q8_0 -ctv q8_0, which nearly halves the cache. Our llama.cpp setup guide and the RTX 3060 page use the same calculator.

PC with an RTX 3060 12 GBHardware sheet→

“chat template parsing error”: the model’s template does not parse

The model loads, but the server stops before it starts listening, and the log says:

… init: chat template parsing error: …
please consider disabling jinja via --no-jinja, or use a custom chat template via --chat-template
for example: --no-jinja --chat-template chatml

Right after it comes exiting due to model loading error. Each model brings its own recipe for turning a conversation into text, and llama.cpp reads it with its Jinja engine, now on by default (the old --jinja flag is no longer needed). Some GGUF files carry templates with constructs it cannot read yet. The message gives you both exits: --no-jinja falls back to the built-in templates, and --chat-template chatml (or another built-in name) forces one. If the same model works in a newer build, update instead: template support improves almost as fast as model support.

Answers that look like nonsense, repeat the role names or never stop are the same family of problem, just without the error: try a GGUF from the model’s official repository first.

To see what the template really sends to the model, the server itself can show you. In this video, one developer opens the server’s built-in web interface, turns on the detailed log and reads the exact prompt the model receives once the template has been applied:

CMake: “LLAMA_CUBLAS is deprecated, use GGML_CUDA instead”

If you build llama.cpp yourself with an old recipe, CMake stops at the first step:

LLAMA_CUBLAS is deprecated, use GGML_CUDA instead

This one is a hard error: the build does not go on. The GPU options moved from the LLAMA_ prefix to GGML_ a long time ago, and old build commands still use the old names. The modern command for NVIDIA is cmake -B build -DGGML_CUDA=ON and then cmake --build build --config Release. The other old names (LLAMA_CUDA, LLAMA_METAL, LLAMA_NATIVE, LLAMA_SYCL…) still work for now, with a warning like LLAMA_CUDA is deprecated, use GGML_CUDA instead; switch them before they stop working too.

While you are at it: older builds also called the programs main and server. Those names are gone. Today’s package has llama, with llama serve and llama cli, and still ships llama-server and llama-cli on their own. If a script fails with “command not found”, that is usually all it is.

And if your problem is not here

3 habits solve most of the rest. Write down the build number before asking anybody for help: many answers depend on it. Start again with the smallest model you have and -c 4096: if that works, the problem is memory, not llama.cpp. And try the same GGUF in a program that runs it on llama.cpp, such as Ollama or LM Studio: if it works there, compare flags. Our llama.cpp vs Ollama comparison lists the defaults Ollama picks for you, such as the context and the number of requests at once. Our guides for when Ollama does not work and when vLLM does not work cover the errors of their own.

If you would rather start again from a clean setup, in this video another developer runs llama.cpp in Docker (with one image for the CPU and another for NVIDIA cards), from the first GGUF file to the server and its web interface. It was recorded with port 8080 and the flags of its day: if a command in it differs from this page, go with the page.