llama.cpp
Version in force v0.7.0 (stable channel) / b11850 (daily build)2026-10-15 (v0.7.0) / 2026-10-24 (b11850)Tree audited master, commit eb25b7263e1604b4382295563f5a924002d6f87c (2026-08-25)Licence MIT
llama.cpp runs models on more hardware than any of the five programs we analyse: Windows, macOS, Linux and even Android and iOS. We read its code: no telemetry and 0 calls home out of the box, not even to check for updates. MIT licence, public code. Version in force: v0.7.0 (stable channel) / b11850 (daily build).
The engine. It is the C/C++ implementation almost every other consumer local-AI program is built on, and you can use it directly: a command-line tool and a server with its own web interface.
Steps to the first token
Measured by Local AI Scope
Hosts it contacts out of the box
Measured by Local AI Scope
Licence
Measured by Local AI Scope
Videos: llama.cpp 4
The best walk-throughs we found. Press play and it opens right here.
The videos live on YouTube, and nothing loads from there until you press play (no-cookie mode). The thumbnail is served by us.
How we checked this
Everything this sheet says about the network comes from reading the published source code and the official documentation on the date shown — not from watching the traffic. We have not yet sat the program in front of a network analyser, so this is static analysis, not observation. That capture is still on our list; the day we have it, these cells move up a level of evidence. Until then, no pretending we sniffed the wire: we have read the recipe, not tasted the dish.
What leaves your machine
Before a local runtime gets anywhere near your documents, we ask it four questions: what it sends on its own, who else it calls, which address it listens on, and whether you can check any of that yourself. Think of it as a job interview.
Telemetry out of the box
None at all — and here that is a fact, not a promise. Verified by us in the primary source
We swept the whole tree at commit eb25b72 looking for telemetry, analytics, posthog, mixpanel, sentry.io, google-analytics and gtag( across every .c, .cpp, .h, .hpp, .ts, .tsx, .svelte and .js file: not one real match. The only domains written anywhere in the code are documentation URLs inside comments. There is no endpoint to send usage data to. It is also the only one of the five that does not check for updates: with a local model file it does not open a single connection.
How to switch it off: Nothing to switch off: there is none.
Source · consulted 2026-08-25
Other calls home
| What | Where to | When | Can you avoid it | Evidence |
|---|---|---|---|---|
| Model download | huggingface.co |
Only if you use -hf, --model-url or the download subcommand. With a local file it opens nothing. It does send a user agent identifying the build. |
Use local files | verified by us |
| Downloading models packaged as container images | auth.docker.io · registry-1.docker.io |
Only with the --docker-repo option. |
Do not use it | verified by us |
| Prebuilt web interface, at COMPILE time | huggingface.co |
Only when building from source with the default option on. With no network it warns and builds without the interface. | LLAMA_USE_PREBUILT_UI=OFF | verified by us |
| Manual self-update | llama.app |
Only if you run the update subcommand by hand, and only on installs made with their script. It never fires on its own. | Do not run it | verified by us |
Source · consulted 2026-08-25
Which address it listens on
| Setting | Out of the box |
|---|---|
| Listens on | 127.0.0.1:9931 · 127.0.0.1:8080 in v0.6.0 and earlier |
| Authentication out of the box | none |
| CORS out of the box | Reflects any Origin back and allows credentials; the server warns on the console when there is no key either |
It listens on localhost — and that has always been true: across the whole history of the repository, of the 719 revisions of the files that declare the host, 471 declare 127.0.0.1 and none declares anything else. The price of that silence is CORS: the server reflects any Origin back with credentials allowed, so any web page you visit can talk to your llama-server while it is running. It says so itself on the console when there is no API key either. It is also the only one of the five that tightens its own CORS policy according to how dangerous what it exposes is: turning on the agent tools — which include reading, writing and editing files and running shell commands — makes it fall back to localhost origins only. Verified by us in the primary source
«By default the server reflects any Origin header back with credentials allowed. […] security: no API key is set and CORS allows all origins»
Source · consulted 2026-08-25
Can this be audited?
Yes, and it is the reason this row can say none about telemetry with a straight face. MIT, the whole engine, the server and the web interface in one public repository. It is the only one of the five where we can state, having read the code, that it neither sends telemetry nor checks for updates. Verified by us in the primary source
Source · consulted 2026-08-25
Privacy policy
There is no privacy policy, and here that is good news rather than a gap: there is no service behind the program and no data controller to ask for one. not verified
What it runs, and where
Platforms, accelerators and model formats: the three walls you can hit on that first, eager evening. Better to find them here than at midnight with a model half downloaded.
Where it runs
| Platform | Support | Evidence |
|---|---|---|
| Windows | Yes: prebuilt binaries for x64 and ARM64, in CPU, CUDA, Vulkan, ROCm, SYCL, OpenVINO and OpenCL/Adreno flavours. Installable with winget. | verified by us |
| macOS Apple Silicon | Yes: prebuilt binary plus Homebrew, MacPorts or Nix packages. | verified by us |
| macOS Intel | Yes, but CPU only. There is an Intel Mac binary, and the release workflow builds it with Metal disabled on purpose — the comment in their own file says the runners have no GPU. | verified by us |
| Linux | Yes: x64, ARM64 and even s390x (IBM Z), in CPU, CUDA, Vulkan, ROCm, SYCL and OpenVINO flavours. | verified by us |
| + | Android ARM64 and an xcframework for iOS/macOS. It also documents RISC-V builds. | verified by us |
What it accelerates
| Backend | Status | Evidence |
|---|---|---|
| CUDA | Yes, with ready-made binaries on Windows and on Linux: CUDA 12.8 (12.4 on Windows) and 13.4 for x64, 13.4 for ARM64, each with its CUDA runtime package as a separate download. When we first audited it, on 25 August 2026, Linux had no CUDA build; the stable v0.7.0 and today’s daily builds do. | verified by us |
| ROCm | Yes, with binaries: ROCm 10.0, on Windows and Linux. | verified by us |
| Metal | Yes, built into the macOS ARM64 binary. Disabled on Intel Macs. | verified by us |
| Vulkan | Yes, with binaries: x64 and ARM64 on Linux; on Windows, x64, and ARM64 too in the daily builds since 4 October 2026. | verified by us |
| CPU | Yes, with SIMD per architecture. It is the whole reason the project exists. | verified by us |
| SYCL / OpenCL / OpenVINO | Yes: SYCL for Intel, OpenCL/Adreno for Qualcomm phones, and OpenVINO. | verified by us |
| NPU | We could not verify this. It publishes OpenVINO builds, and OpenVINO is Intel’s stack for CPU, GPU and NPU — but no official statement says llama.cpp runs inference on the NPU, so we do not claim it. | not verified |
Model formats it runs
GGUF only. GGUF is its native format — this project invented it. Safetensors have to be converted first with the script the repository itself ships. verified by us
Source · consulted 2026-08-25
Installing it, and living with it
The everyday part: how many steps to the first token, what it leaves on your disk, under which licence, and at what pace it changes under your feet once it is installed.
Steps to the first token
2 steps. The quickest way in is the official installer from llama.app (curl -fsSL https://llama.app/install.sh | sh; on Windows, irm https://llama.app/install.ps1 | iex): it picks the build that matches your GPU and copies the llama program to ~/.local/bin. It installs a daily build (b11200 on 2 October 2026), not the stable one, and has no binary for Intel Macs. Then run `llama serve -hf <hugging-face-repo>`: it downloads the model and opens the web interface. What raises the difficulty is not installing it but that everything is configured through command-line flags. verified by us
If something fails: 8 errors, the exact message each one shows and the fix
What it writes to your disk
| What | Where | Evidence |
|---|---|---|
| Models downloaded with -hf — the standard Hugging Face cache, shared with every other program that uses it, so downloads are not duplicated | ~/.cache/huggingface/hub — the Hugging Face cache (LLAMA_CACHE → HF_HUB_CACHE → HF_HOME/hub → XDG_CACHE_HOME) |
verified by us |
| Other artefacts and the RPC server cache | $LLAMA_CACHE · $XDG_CACHE_HOME · ~/.cache/llama.cpp/ |
verified by us |
| Web interface conversations — in your browser, not on the server | In your browser, not on the server: IndexedDB (LlamaUi database) and localStorage with the LlamaUi prefix. Clearing the site data deletes them, and they do not sync between browsers | verified by us |
| Prompt cache — in memory, not on disk, unless you ask for it | In memory, not on disk (8192 MiB by default). It only reaches the disk with --slot-save-path or --prompt-cache |
verified by us |
| Logs — only if you ask for them | Only with --log-file; dumping prompts to disk (--log-prompts-dir) is off by default |
verified by us |
Licence
MIT. The whole engine, the server and the web interface. verified by us
Source · consulted 2026-08-25
Version and release pace
Two rhythms. The development channel publishes around 16 builds a day (481 in the 30 days to 23 October 2026). The stable channel was only born on 21 August 2026 and has published seven versions since, the latest v0.7.0 on 15 October, which adds support for new models such as NVIDIA’s Nemotron 3.5 Super. verified by us
«tag vX.Y.Z – stable, slower release cadence, recommended for downstream distribution and casual users; tag b[NUM] – bleeding edge, faster release cadence, recommended for developers and technical users.»
Source · consulted 2026-10-24
What to know before you install it
- It is the only one of the five where we can state, having read the code, that it sends no telemetry and checks for no updates. With a local model file it does not open a single connection.
- Since 21 August 2026 there is a stable channel (v0.2.0) alongside the daily builds. Any text saying «llama.cpp has no stable releases» is now out of date. Careful, though: the winget package still ships the daily build.
- The price of that silence is CORS: the server reflects any origin and allows credentials, so any web page you visit can talk to your llama-server while it is running. The server warns you on the console. Narrow it with
--cors-originslocalhost. - The default port is now 9931, in the stable release too. It changed in daily build b11521 on 9 October 2026 and reached the stable channel with v0.7.0 on 15 October. v0.6.0 and earlier builds listen on 8080 and print a notice that the port will change; since 16 October 2026 Homebrew has shipped v0.7.0, and the project’s Docker images keep 8080 inside the container. If you have scripts or apps pointed at :8080, start it with
--port8080 and no update will move it. - NVIDIA on Linux no longer means compiling. On 25 August there was no CUDA build for Linux; the stable v0.7.0 and the daily builds now ship one for x64 and ARM64. Pick the one that matches your driver (CUDA 12.8 or 13.4); its CUDA runtime package is a separate download.
- Its distributed-compute helper server carries a blunt warning in its own documentation: the RPC backend is a proof of concept, fragile and insecure, and must never be run on an open network.