vLLM
Version in force 0.32.02026-10-18Tree audited tag v0.32.0 (2026-10-18)Licence Apache-2.0
vLLM 0.32.0 is not meant for your own computer: it serves models from Linux servers, and on Windows it officially runs only through WSL. Usage statistics are on by default (a heartbeat every 10 minutes to stats.vllm.ai), and vLLM is the only one of the five that listens on every interface with no key (0.0.0.0:8000). Apache-2.0, public code.
An inference engine designed to serve models to many people at once, not to chat on a laptop. It is the de-facto standard on servers with NVIDIA GPUs.
Steps to the first token
Measured by Local AI Scope
Hosts it contacts out of the box
Measured by Local AI Scope
Licence
Measured by Local AI Scope
Videos: vLLM 4
Official footage and the best walk-throughs we found. Press play and it opens right here.
The videos live on YouTube, and nothing loads from there until you press play (no-cookie mode). The thumbnail is served by us.
How we checked this
Everything this sheet says about the network comes from reading the published source code and the official documentation on the date shown — not from watching the traffic. We have not yet sat the program in front of a network analyser, so this is static analysis, not observation. That capture is still on our list; the day we have it, these cells move up a level of evidence. Until then, no pretending we sniffed the wire: we have read the recipe, not tasted the dish.
What leaves your machine
Before a local runtime gets anywhere near your documents, we ask it four questions: what it sends on its own, who else it calls, which address it listens on, and whether you can check any of that yourself. Think of it as a job interview.
Telemetry out of the box
Yes, on out of the box — and it is not one notice at startup, it is a heartbeat every ten minutes. Verified by us in the primary source
That vLLM collects usage statistics by default is documented. What is not in the documentation, and only shows up in the code, is that it is not a single send: the continuous-report function runs an endless loop that sleeps 600 seconds and POSTs again, for as long as the process lives. The destination is not in the documentation either — you have to go to the code to find stats.vllm.ai. Network failures are swallowed silently. What goes out: a random run identifier, the cloud provider detected by reading hardware identifiers, core count, processor brand and family/model/stepping, total memory, the full operating-system string, CUDA version, GPU count with model and memory per device, five environment variables, the architecture of the loaded model, and — on the server path — another twenty-odd startup parameters. In its favour, and it must be said: it is the only one of the five that lets you see exactly what it sends, in ~/.config/vllm/usage_stats.json, and it offers four ways to turn it off.
«Report usage every 10 minutes. This helps us to collect more data points for uptime of vLLM usages.»
How to switch it off: Any one of these four: VLLM_NO_USAGE_STATS=1, DO_NOT_TRACK=1, VLLM_DO_NOT_TRACK=1, or creating the file ~/.config/vllm/do_not_track. The documentation publishes three of them: the two variables VLLM_NO_USAGE_STATS and DO_NOT_TRACK, and the file. VLLM_DO_NOT_TRACK is only in the code.
Source · consulted 2026-10-23
Other calls home
| What | Where to | When | Can you avoid it | Evidence |
|---|---|---|---|---|
| Usage statistics | stats.vllm.ai |
At startup and every 10 minutes, always, unless you turn it off. | Yes, four different ways | verified by us |
| Model download, identifying itself as vLLM | huggingface.co (or modelscope) |
When loading a model that is not cached. It is the only one of the five that tells Hugging Face which runtime and which version is downloading. | HF_HUB_OFFLINE=1 | verified by us |
| FlashInfer kernel check and download, from NVIDIA’s server | edge.urm.nvidia.com |
At startup, if vLLM uses FlashInfer, or just weighs it up, and its precompiled kernels are missing: vLLM usually asks first whether NVIDIA’s server answers (one request, 5-second limit) and FlashInfer then downloads the kernels it lacks. Installed with pip (the Docker image already brings them), on data-centre Blackwell cards (B200, B300) it goes out on its own with almost any model; on an H100 or an RTX 50 it can go out with models such as gpt-oss. | Run vllm download-kernels once after installing (it is a separate step from pip): with the kernels on disk, normally neither request goes out. FLASHINFER_NO_DOWNLOAD=1 stops the download, not vLLM’s check | verified by us |
| Prometheus metrics endpoint — local, nothing leaves your machine | local — nothing leaves the machine |
Always: /metrics is mounted by default. Being outside /v1, the API key does not cover it. | Not documented | verified by us |
| Images, audio or video a request passes as a web address | whatever web address the request carries |
Only when a request to the server (chat, /v1/responses, /v1/messages, embeddings, scoring…) carries an image, audio or video as an http(s) address: the server downloads it itself before answering, announcing itself as vLLM and its version, and following redirects. In chat, with a text-only model, the request fails before anything is downloaded. | --allowed-media-domains limits it to the domains you list; or embed your media (base64). VLLM_MEDIA_URL_ALLOW_REDIRECTS=0 stops following redirects |
verified by us |
Source · consulted 2026-10-23
Which address it listens on
| Setting | Out of the box |
|---|---|
| Listens on | 0.0.0.0:8000 |
| Authentication out of the box | none |
| CORS out of the box | Closed since 0.32.0: no page from another site can read its answers in your browser until you add that page’s origin to --allowed-origins (up to 0.31.0, any origin, method and header). It only binds browsers: the server still listens on 0.0.0.0 with no key |
This is the one that matters. In the published version, the host argument defaults to nothing and is resolved into the empty string, and binding to the empty string on IPv4 is INADDR_ANY — that is, 0.0.0.0. A plain `vllm serve model` with no other parameters leaves the model available to the whole local network, with no password. We checked the same default in eight releases: it is not a recent regression. And there is a second floor to this: their own security page admits the API key is not enough, because it only covers the /v1, /v2, /inference and /cohere prefixes while other sensitive endpoints sit unauthenticated on the same server. Their own recommendation is to put it behind a reverse proxy. A third detail: vLLM uses PyTorch’s torch.distributed, and when it sets it up over TCP — across several machines, with data parallelism, or on AMD cards with AITER’s all-reduce — PyTorch’s TCPStore listens on all network interfaces, which the project itself calls insecure by default. On a single machine with NVIDIA cards and no data parallelism, version 0.32.0 uses a file instead. Verified by us in the primary source
«The --api-key flag (or VLLM_API_KEY environment variable) provides authentication for vLLM’s HTTP server, but only for OpenAI-compatible API endpoints under the /v1 path prefix […]. Many other sensitive endpoints are exposed on the same HTTP server without any authentication enforcement.»
Source · consulted 2026-10-23
Can this be audited?
Yes. Apache-2.0, the only one of the five that is neither MIT nor proprietary. Every statement above about the ten-minute heartbeat and about 0.0.0.0 was read in the source of the published tag, not inferred from the documentation — which in both cases says something less complete, and in one case says the opposite. Verified by us in the primary source
Source · consulted 2026-08-25
Privacy policy
There is no privacy policy as such: there is a documentation page explaining what they collect and how to turn it off, and announcing that a cleaned, aggregated subset will be released publicly for the community’s benefit. vendor declared Source
What it runs, and where
Platforms, accelerators and model formats: the three walls you can hit on that first, eager evening. Better to find them here than at midnight with a model half downloaded.
Where it runs
| Platform | Support | Evidence |
|---|---|---|
| Windows | No native support. The official route is WSL. | vendor declared |
| macOS Apple Silicon | Experimental and CPU only: their documentation says you must build from source. The GPU does not go through vLLM — it goes through a community plugin that uses MLX underneath and needs MLX-format models. | vendor declared |
| macOS Intel | We could not verify this. There are no wheels and no documentation for Intel Macs; the Apple Silicon section is the only macOS one there is, and its wheels are arm64. We report the absence rather than a «no». | not verified |
| Linux | Yes, this is its platform. Python 3.11 to 3.14 according to the documentation (0.32.0 dropped 3.10). | vendor declared |
What it accelerates
| Backend | Status | Evidence |
|---|---|---|
| CUDA | Yes, and it is the first-class case — the only one their own site flags as popular. Requires compute capability 7.5 or higher, which rules out GTX 10xx and older. | vendor declared |
| ROCm | Yes, with its own package index, but narrower than CUDA: for 0.32.0, Python 3.12, ROCm 7.2.3 and glibc 2.39 or newer. | verified by us |
| CPU | Yes, with limitations: x86 (AVX512 recommended, AVX2 limited), ARM with NEON, and IBM Z. | vendor declared |
| Metal | No, only through an external plugin: vLLM-Metal, community-maintained, using MLX underneath. | vendor declared |
| NPU / accelerators | Not in the main repository. Google TPU, Intel Gaudi, AWS Neuron and Ascend are separate packages since 0.27. | vendor declared |
| Vulkan | We could not verify this. It does not appear on their supported-platforms page, and there is no statement either way. | not verified |
Model formats it runs
safetensors natively, plus AWQ, GPTQ, FP8, compressed-tensors, bitsandbytes and a couple of dozen other quantisation schemes. GGUF is no longer included: since 0.27 it has been moved out to a separate plugin and is still described by them as highly experimental. Any text saying «vLLM supports GGUF» without that caveat is out of date. verified by us
Source · consulted 2026-08-25
Installing it, and living with it
The everyday part: how many steps to the first token, what it leaves on your disk, under which licence, and at what pace it changes under your feet once it is installed.
Steps to the first token
2 steps. Install the package (since 0.32.0, followed by `vllm download-kernels`) and run `vllm serve <model>`. There is no interface: you talk to it over the API. The difficulty is not installing it but that there is nothing to look at and you have to understand the memory parameters. verified by us
Step by step: install it, start a first model on port 8000, and close what it opens to the network
If something fails: 8 errors, the exact message each one shows and the fix
What it writes to your disk
| What | Where | Evidence |
|---|---|---|
| A local copy of the telemetry it sends — their documentation invites you to read it, and it is the only one of the five that shows you exactly what goes out | ~/.config/vllm/usage_stats.json |
verified by us |
| Sentinel file to disable telemetry — you create it | ~/.config/vllm/do_not_track — you create it |
verified by us |
| PyTorch compilation cache | ~/.cache/vllm (VLLM_CACHE_ROOT) |
vendor declared |
| Models — the Hugging Face cache | ~/.cache/huggingface — the Hugging Face cache (HF_HOME) |
vendor declared |
| Chat history — does not exist: it is an API server with no chat interface and it persists nothing from the requests | Nowhere — nothing is written | verified by us |
Licence
Apache-2.0. The only one of the five that is neither MIT nor proprietary. verified by us
Source · consulted 2026-08-25
Version and release pace
A minor version roughly every two weeks. The last two patches (0.25.1 and 0.27.1) came within three days, but there has been none since 0.27.1 (August 2026). It is not monthly: the last minor jumps were 14, 13, 13 and 13 days apart. Careful when dating any text about it: between 26 August and 22 September 2026 it published three minor versions (0.28, 0.29 and 0.30). verified by us
Source · consulted 2026-10-22
What to know before you install it
- It is the only one of the five with telemetry on out of the box, and it is not a startup notice: it sends a heartbeat every ten minutes for as long as the server lives. In its favour: it documents it, it lets you read the exact payload in a local file and it offers four ways to switch it off.
- It is also the only one that listens on 0.0.0.0 by default, with no key. A `vllm serve model` with no other parameters, on a laptop connected to a hotel wifi, leaves the model available to the entire network. It is not a regression: it has been like that for at least eight releases.
- Its own security page warns that the API key only covers the /v1, /v2, /inference and /cohere routes, and that there are sensitive endpoints with no authentication on the same server. The official recommendation is to put it behind a reverse proxy.
- It reserves 92 % of the card’s memory the moment it starts, and it is pre-allocation, not on-demand use. Careful with the figure: the 90 % that circulates online comes from an example frozen in their own documentation since version 0.4.0.
- vLLM publishes no minimum video-memory figure anywhere. The only published hardware requirement is compute capability 7.5. We are not inventing a number.
- It is not a program for your computer. No interface, no native Windows, and on a Mac only an experimental CPU backend. It is in this comparison to mark the boundary.