vLLM
Version in force 0.27.12026-08-11Tree audited tag v0.27.1 (2026-08-11) y rama principal del 2026-08-25Licence Apache-2.0
For anyone building a service, not for anyone who wants a model on their own computer. It is here as the marker of the professional end.
An inference engine designed to serve models to many people at once, not to chat on a laptop. It is the de-facto standard on servers with NVIDIA GPUs.
Steps to the first token
Measured by Local AI Scope
Hosts it contacts out of the box
Measured by Local AI Scope
Licence
Measured by Local AI Scope
How we checked this
Everything this sheet says about the network comes from reading the published source code and the official documentation on the date shown — not from watching the traffic. We have not run the program with a network analyser in front of it, so this is static analysis, not observation. A real capture is still pending; the day we have one, these cells move up a level of evidence. We would rather say that than let you assume we sniffed the wire.
What leaves your machine
The four questions a local runtime has to answer before it deserves your documents: what it sends on its own, who else it calls, on which address it listens, and whether you can check any of it yourself.
Telemetry out of the box
Yes, on out of the box — and it is not one notice at startup, it is a heartbeat every ten minutes. Verified by us in the primary source
That vLLM collects usage statistics by default is documented. What is not in the documentation, and only shows up in the code, is that it is not a single send: the continuous-report function runs an endless loop that sleeps 600 seconds and POSTs again, for as long as the process lives. The destination is not in the documentation either — you have to go to the code to find stats.vllm.ai. Network failures are swallowed silently. What goes out: a random run identifier, the cloud provider detected by reading hardware identifiers, core count, processor brand and family/model/stepping, total memory, the full operating-system string, CUDA version, GPU count with model and memory per device, five environment variables, the architecture of the loaded model, and — on the server path — another twenty-odd startup parameters. In its favour, and it must be said: it is the only one of the five that lets you see exactly what it sends, in ~/.config/vllm/usage_stats.json, and it offers four ways to turn it off.
«Report usage every 10 minutes. This helps us to collect more data points for uptime of vLLM usages.»
How to switch it off: Any one of these four: VLLM_NO_USAGE_STATS=1, DO_NOT_TRACK=1, VLLM_DO_NOT_TRACK=1, or creating the file ~/.config/vllm/do_not_track. The documentation publishes the first three; the fourth is only in the code.
Source · consulted 2026-08-25
Other calls home
| What | Where to | When | Can you avoid it | Evidence |
|---|---|---|---|---|
| Usage statistics | stats.vllm.ai |
At startup and every 10 minutes, always, unless you turn it off. | Yes, four different ways | verified by us |
| Model download, identifying itself as vLLM | huggingface.co (or modelscope) |
When loading a model that is not cached. It is the only one of the five that tells Hugging Face which runtime and which version is downloading. | HF_HUB_OFFLINE=1 | verified by us |
| Prometheus metrics endpoint — local, nothing leaves your machine | local — nothing leaves the machine |
Always: /metrics is mounted by default. Being outside /v1, the API key does not cover it. | Not documented | verified by us |
Source · consulted 2026-08-25
Which address it listens on
| Setting | Out of the box |
|---|---|
| Listens on | 0.0.0.0:8000 |
| Authentication out of the box | none |
| CORS out of the box | Wide open: any origin, any method, any header |
This is the one that matters. In the published version, the host argument defaults to nothing and is resolved into the empty string, and binding to the empty string on IPv4 is INADDR_ANY — that is, 0.0.0.0. A plain `vllm serve model` with no other parameters leaves the model available to the whole local network, with no password. We checked the same default in eight releases: it is not a recent regression. And there is a second floor to this: their own security page admits the API key is not enough, because it only covers the /v1, /v2 and /inference prefixes while other sensitive endpoints sit unauthenticated on the same server. Their own recommendation is to put it behind a reverse proxy. A third detail: the PyTorch communication layer opens a store that listens on all interfaces even on a single machine, and the project itself calls that insecure by default. Verified by us in the primary source
«The –api-key flag (or VLLM_API_KEY environment variable) provides authentication for vLLM’s HTTP server, but only for OpenAI-compatible API endpoints under the /v1 path prefix […]. Many other sensitive endpoints are exposed on the same HTTP server without any authentication enforcement.»
Source · consulted 2026-08-25
Can this be audited?
Yes. Apache-2.0, the only one of the five that is neither MIT nor proprietary. Every statement above about the ten-minute heartbeat and about 0.0.0.0 was read in the source of the published tag, not inferred from the documentation — which in both cases says something less complete, and in one case says the opposite. Verified by us in the primary source
Source · consulted 2026-08-25
Privacy policy
There is no privacy policy as such: there is a documentation page explaining what they collect and how to turn it off, and announcing that a cleaned, aggregated subset will be released publicly for the community’s benefit. vendor declared Source
What it runs, and where
Platforms, accelerators and model formats — the three walls you hit on the first evening, and the ones nobody writes down until you have hit them.
Where it runs
| Platform | Support | Evidence |
|---|---|---|
| Windows | No native support. The official route is WSL. | vendor declared |
| macOS Apple Silicon | Experimental and CPU only: their documentation says you must build from source. The GPU does not go through vLLM — it goes through a community plugin that uses MLX underneath and needs MLX-format models. | vendor declared |
| macOS Intel | We could not verify this. There are no wheels and no documentation for Intel Macs; the Apple Silicon section is the only macOS one there is, and its wheels are arm64. We report the absence rather than a «no». | not verified |
| Linux | Yes, this is its platform. Python 3.10 to 3.13 according to the documentation. | vendor declared |
What it accelerates
| Backend | Status | Evidence |
|---|---|---|
| CUDA | Yes, and it is the first-class case — the only one their own site flags as popular. Requires compute capability 7.5 or higher, which rules out GTX 10xx and older. | vendor declared |
| ROCm | Yes, with its own package index, but narrower than CUDA: Python 3.12 and ROCm 7.0. | vendor declared |
| CPU | Yes, with limitations: x86 (AVX512 recommended, AVX2 limited), ARM with NEON, and IBM Z. | vendor declared |
| Metal | No, only through an external plugin: vLLM-Metal, community-maintained, using MLX underneath. | vendor declared |
| NPU / accelerators | Not in the main repository. Google TPU, Intel Gaudi, AWS Neuron and Ascend are separate packages since 0.27. | vendor declared |
| Vulkan | We could not verify this. It does not appear on their supported-platforms page, and there is no statement either way. | not verified |
Model formats it runs
safetensors natively, plus AWQ, GPTQ, FP8, compressed-tensors, bitsandbytes and a couple of dozen other quantisation schemes. GGUF is no longer included: since 0.27 it has been moved out to a separate plugin and is still described by them as highly experimental. Any text saying «vLLM supports GGUF» without that caveat is out of date. verified by us
Source · consulted 2026-08-25
Installing it, and living with it
How many steps to the first token, what it leaves on your disk, under which licence and at what pace it changes under you.
Steps to the first token
2 steps. Install the package and run `vllm serve <model>`. There is no interface: you talk to it over the API. The difficulty is not installing it but that there is nothing to look at and you have to understand the memory parameters. verified by us
What it writes to your disk
| What | Where | Evidence |
|---|---|---|
| A local copy of the telemetry it sends — their documentation invites you to read it, and it is the only one of the five that shows you exactly what goes out | ~/.config/vllm/usage_stats.json |
verified by us |
| Sentinel file to disable telemetry — you create it | ~/.config/vllm/do_not_track — you create it |
verified by us |
| PyTorch compilation cache | ~/.cache/vllm (VLLM_CACHE_ROOT) |
vendor declared |
| Models — the Hugging Face cache | ~/.cache/huggingface — the Hugging Face cache (HF_HOME) |
vendor declared |
| Chat history — does not exist: it is an API server with no chat interface and it persists nothing from the requests | Nowhere — nothing is written | verified by us |
Licence
Apache-2.0. The only one of the five that is neither MIT nor proprietary. verified by us
Source · consulted 2026-08-25
Version and release pace
A minor version roughly every two weeks, with patches within one to three days. It is not monthly: the last minor jumps were 14, 12, 16 and 14 days apart. Careful when dating any text about it — the v0.28.0 tag has existed since 23 August 2026 without a release, so 0.28 may appear any day. verified by us
Source · consulted 2026-08-25
What to know before you install it
- It is the only one of the five with telemetry on out of the box, and it is not a startup notice: it sends a heartbeat every ten minutes for as long as the server lives. In its favour: it documents it, it lets you read the exact payload in a local file and it offers four ways to switch it off.
- It is also the only one that listens on 0.0.0.0 by default, with no key. A `vllm serve model` with no other parameters, on a laptop connected to a hotel wifi, leaves the model available to the entire network. It is not a regression: it has been like that for at least eight releases.
- Its own security page warns that the API key only covers the /v1, /v2 and /inference routes, and that there are sensitive endpoints with no authentication on the same server. The official recommendation is to put it behind a reverse proxy.
- It reserves 92 % of the card’s memory the moment it starts, and it is pre-allocation, not on-demand use. Careful with the figure: the 90 % that circulates online comes from an example frozen in their own documentation since version 0.4.0.
- vLLM publishes no minimum video-memory figure anywhere. The only published hardware requirement is compute capability 7.5. We are not inventing a number.
- It is not a program for your computer. No interface, no native Windows, and on a Mac only an experimental CPU backend. It is in this comparison to mark the boundary.