An LLM VRAM calculator should show the weight, KV-cache, and runtime-overhead terms instead of returning one unexplained fit number. The calculator above does that. I validated it against five models on my 8GB M1 MacBook Air, and every prediction landed within 4.7% to 9.8% of the memory the model actually used, each one slightly above the measured figure. Use it to screen model size, quantization, context, batch, and available memory before downloading a model.
How the local LLM hardware calculator works
The calculator on this page takes a model's parameter count, its quantization level, and its attention architecture, then computes three numbers: model weight size, KV cache size at your chosen context length, and a small runtime overhead. Weights are params x bytes-per-parameter; a model quantized to Q4_K_M uses roughly 0.6 bytes per parameter, while Q8_0 uses roughly 1.07 bytes per parameter, both measured directly from five installed models, using the parameter counts Ollama's model details report, rather than a vendor spec sheet.
The same controls cover the questions people ask under other names, such as an LLM memory requirements calculator, an LLM inference memory calculator, or an LLM inference hardware calculator: weights, KV cache, context, batch, and available memory. The label changes with the hardware question, but the estimate should remain inspectable.
Other calculators in search results often return only a final number; this one prints its formula on the page. APXML's VRAM calculator, for example, accepts model, quantization, batch size, and sequence length as inputs but does not publish the equations behind its output on the page itself.
How to use the LLM VRAM calculator
Start with a preset when you want a quick answer, or choose custom inputs for a model that is not in the validation table. The six inputs below determine the estimate. Do not replace an unknown field with the model's parameter count: architecture metadata changes the KV-cache term.
| Input | Where to get it | Why it matters |
|---|---|---|
| Parameter count and quantization | ollama show <model> --verbose | Sets the weight-memory term |
| KV heads, head dimension, and layers | ollama show <model> --verbose or GGUF metadata | Sets KV-cache growth |
| Context length | Your intended num_ctx setting | Larger context requires more memory |
| Batch size | Expected concurrent requests | KV-cache demand scales with concurrent work |
| KV-cache precision | Runtime setting, f16/q8_0/q4_0 | Lower precision uses less cache memory |
| Available RAM or VRAM | The hardware specification, minus a safety margin | Decides whether the estimate is usable |
For a cautious first pass, enter the context length you actually plan to use, leave room for the operating system, and treat a result close to the hardware limit as a warning rather than a green light. The Ollama FAQ also recommends checking ollama ps to see whether the model is fully on the GPU, fully in system memory, or split between both.
For LLM agent platform sizing, use CPU, GPU, and VM formulas only after estimating this memory boundary: decide which pool holds the weights and KV cache, then add context and concurrency before comparing hardware. This page estimates memory fit; it does not forecast VM price or network latency.
VRAM vs RAM: what changes
Apple Silicon machines like the M1 MacBook Air use unified memory, meaning RAM and the GPU's usable memory are the same pool with no separate VRAM figure to budget for. On a discrete-GPU PC, VRAM is a hard, separate ceiling: whatever doesn't fit in VRAM either fails to load or falls back to slower CPU/system RAM, which this calculator does not model (see the limitations below). The arithmetic for weights and KV cache is identical either way; only which memory pool you compare the total against changes.
Quantization and bytes per parameter
Bytes-per-parameter is not a vendor estimate here. I measured it directly from five installed Ollama models by dividing each model's on-disk GGUF file size by its exact parameter count from ollama show --verbose: llama3.2:1b at Q8_0 came out to roughly 1.07 bytes/param. The same method applied to every Q4_K_M model in the preset list landed closer to 0.64 bytes/param, and those values are built into the calculator's presets so you don't have to guess.
The QLoRA paper by Tim Dettmers and co-authors describes its method as reducing memory use enough to fine-tune a 65B-parameter model on a single 48GB GPU. That training result is not an inference guarantee, but it explains why the calculator treats quantization as a first-class input instead of hiding it inside a model-size label.
To choose between the GGUF levels themselves, the Q4_K_M vs Q5_K_M comparison lists bits per weight, file size, and published quality loss for each quant from Q2_K to Q8_0.
Context length and the KV cache
The whole formula is two lines. Weights are params times bytes-per-parameter, and the KV cache is 2 x kv_heads x head_dim x layers x context x batch x bytes-per-element. That second line needs three architecture numbers most calculators hide: kv_heads (attention heads dedicated to keys/values, which can be fewer than the total attention head count under grouped-query attention), head_dim (dimension per head), and layer count. All three come from the model's own GGUF metadata as used by llama.cpp, the same fields ollama show --verbose prints, so none of them is a guess. Doubling context length exactly doubles the KV cache term; doubling batch size does the same, while model weights stay fixed regardless of context or batch.
For the full breakdown of that formula, a calculated GB table across context lengths and cache precisions, and the exact server flag that fixes an out-of-memory error, see the kv cache size guide.
Hybrid attention: why Qwen3.5+ and gpt-oss need less KV cache
Newer models do not cache every token in every layer, and the calculator now counts only the layers that do. Qwen3.5, Qwen3.6, and Qwen3.8 make three of every four layers Gated DeltaNet linear attention, which keeps a small fixed state instead of a per-token cache (Qwen3.5-9B config). gpt-oss alternates full attention with sliding-window layers that keep only the last 128 tokens (gpt-oss-20b config).
The difference is large at long context. The table comes from the calculator's own presets at 16-bit KV cache, batch 1:
| Ollama tag | Layers that cache every token | KV cache at 64K | Same model, every layer counted | Largest context on 16GB | Largest context on 24GB |
|---|---|---|---|---|---|
qwen3.5:9b | 8 of 32 | 2.00 GB | 8.00 GB | 256K (model limit) | 256K (model limit) |
qwen3.8:27b | 16 of 64 | 4.00 GB | 16.00 GB | weights do not fit | about 106K |
qwen3.6:35b-a3b | 10 of 40 | 1.25 GB | 5.00 GB | weights do not fit | about 95K |
gpt-oss:20b | 12 of 24, plus 12 sliding | 1.50 GB | 3.00 GB | about 106K | 128K (model limit) |
qwen3:8b | 36 of 36 | 9.00 GB | 9.00 GB | 40K (model limit) | 40K (model limit) |
qwen2.5-coder:7b | 28 of 28 | 3.50 GB | 3.50 GB | 32K (model limit) | 32K (model limit) |
An older formula that multiplies by every layer would tell you qwen3.5:9b cannot hold a 64K window on a 16GB machine, when it uses a quarter of that cache. For custom models, read layer_types or full_attention_interval in the config: enter only full-attention layers in the first field, and sliding-window layers and their window in the next two. The Ollama Qwen guide and best Ollama model for coding apply the same math to specific picks.
Using it with Ollama specifically
Ollama's default KV cache type is 16-bit floating point, meaning 2 bytes per cached value regardless of how aggressively the model weights themselves are quantized. The calculator defaults to that setting but lets you switch to an 8-bit or 4-bit KV cache to match a non-default Ollama configuration. Five of its presets are the models I measured in the M1/8GB test on an 8GB M1 MacBook Air, so the two pieces of work check each other. Six more cover current Ollama tags: qwen3.5:9b, qwen3.8:27b, qwen3.6:35b-a3b, gpt-oss:20b, qwen3:8b, and qwen2.5-coder:7b, with weights sized from each Ollama download and attention layout taken from each model's Hugging Face config. If you are deciding between Ollama and a GUI-first runner before sizing the machine, LM Studio vs Ollama covers how model format changes these numbers.
What this calculator can and cannot predict
The calculator is a memory estimate, not a universal “will this feel fast?” answer. It handles the weight term, KV-cache term, selected cache precision, batch input, and a measured runtime-overhead buffer. It does not model every runtime's scheduler, CPU/GPU split, mixture-of-experts residency, or the effect of background applications.
| It estimates | It does not promise |
|---|---|
| Weight memory from parameters and bytes per parameter | Tokens per second on different hardware |
| KV-cache memory from architecture and context | That a model will stay entirely in VRAM |
| A calibrated overhead buffer | Exact memory for MoE or partial offload cases |
| How context and batch changes affect the total | That a green result means a comfortable user experience |
Ollama's FAQ notes that parallel requests multiply the effective context demand, and that cache quantization changes memory use. Those runtime settings are why the calculator exposes batch size and KV precision instead of hiding them behind one hardware score.
Validated against a real machine
I ran the calculator's formula against five real models on an 8GB M1 MacBook Air and compared predicted total memory to each model's actual resident memory after load. The model records and local measurement path are reproducible with Ollama's model tooling. mistral:7b came out to a 9.8% delta between prediction and measurement and llama3.2:1b to 6.3%; the other three landed between 4.7% and 8%. Every one of the five stayed under a 10% delta, each estimate slightly above the measured figure, and the chart below shows all five, not a cherry-picked example.

Getting there took a correction. The calculator's first version used a reasonable-sounding overhead assumption before any data existed: extra runtime memory beyond weights and KV cache, guessed as 15% plus a flat 600MB. Replayed against the same field-test measurement, that first guess produced a 55.2% delta on the smallest model, llama3.2:1b, where the current version is off by 6.3%. Real runtime overhead on this machine turned out to be a small single-digit percentage of the weights-plus-KV-cache subtotal, not the double-digit figure a first guess would assume, so the published calculator uses 3% plus 75MB because that is what the validation data supports.
What LLM can I actually run locally?
A worked example: an 8GB machine, phi3:mini (3.8B parameters, Q4_0 quantization, 32 layers), and a 4096-token context. Plugging those into the calculator gives roughly 3.71GB predicted, comfortably under 8GB, and the field test's measured figure was 3,624 MB, a 4.7% delta. The same machine asked to run mistral:7b at the same context length predicts about 4.78GB, which still fits 8GB on paper, but the M1/8GB test measured real swap activity and a steep speed drop at that size that the calculator's memory math alone does not capture. The model identifiers and architecture metadata can be inspected with Ollama's local model tooling.
Once the calculator tells you which memory tier you are in, the best Ollama model for coding guide maps each tier to a specific model tag, quant, and context setting for coding agents.
Where this estimate is wrong
The calculator's automated checks passed every test in my run, including the cross-check against the M1/8GB test, but passing tests does not mean the formula is universal. Four stated failure cases remain. Mixture-of-experts models are not one of them for memory: every expert's weights must be resident, so total parameters are the right input, and MoE changes speed rather than memory. Split CPU/GPU offload is not modeled; the formula assumes one memory pool, which breaks on a discrete-GPU machine offloading only some layers. Non-default KV cache quantization changes real usage below the calculator's f16 default unless you change the input yourself. The 3%+75MB overhead default is calibrated to one machine (Apple M1, Ollama) and will not transfer exactly to Linux or Windows. Concurrent batch requests do not scale linearly the way the KV cache term here assumes, once real server activation memory is involved.
What makes this VRAM estimate useful
The interactive calculator above lets you change model size, quantization, context, and hardware instead of accepting a fixed “models per GPU” list. Treat its fit result as a planning estimate: weights, KV cache, runtime overhead, and shared versus dedicated memory all matter. Compare the estimate with the measured M1 run above, then leave headroom for the operating system and other processes. For a deployment decision, load the exact model and context length on the target machine; a formula cannot predict all backend allocations or throughput.
FAQ
How much VRAM does a 7B, 13B, or 70B model need?
The answer depends on quantization, architecture, context, and runtime overhead, so parameter count alone is not enough. Estimate weight memory from parameters times bytes per parameter, then add KV cache and overhead. The calculator accepts those inputs instead of assigning one universal number to each model class.
How does quantization reduce LLM VRAM usage?
Quantization stores each model weight with fewer bits. Moving from 16-bit weights to an effective 4-bit representation can reduce the weight component substantially, although metadata and runtime overhead prevent a perfect fourfold reduction. It does not remove KV-cache growth from longer contexts.
What model fits in 8GB, 16GB, or 24GB of VRAM?
Use the exact model metadata and intended context rather than a parameter-count shortcut. The validated presets show 1.2B to 3.8B models fitting comfortably within 8GB and a 7B-class model fitting more tightly. Larger budgets permit larger weights or context, but the calculator has not validated every 13B or 70B architecture on physical hardware.
How I tested this
I ran the validation on August 16, 2026, on an 8GB Apple M1 MacBook Air with Ollama. The measured side is the resident memory of five models after load, from llama3.2:1b up to mistral:7b, recorded in my M1/8GB field test. Bytes per parameter for each came from its installed file size divided by the parameter count Ollama reports. The calculator's test suite also checks that it rejects a zero context length and a negative bytes-per-parameter value, that more context never uses less memory, that doubling batch size doubles only the KV-cache term, and that the weights term equals parameter count times bytes per parameter exactly.
I did not measure a discrete GPU, Linux or Windows, partial CPU/GPU offload, or any 13B or 70B model. The six newer presets in the hybrid-attention table are calculated from their configs and download sizes, not measured on hardware. The raw results list every prediction, delta, bytes-per-parameter value, and the overhead calibration.
Related coverage
- Run a Local LLM on 8GB RAM: What an M1 MacBook Air Can Do
- Ollama vs llama.cpp: Speed, Overhead, Which Local LLM Runner
References
- APXML - https://apxml.com/tools/vram-calculator
- Hugging Face - https://huggingface.co/docs/hub/gguf
- llama.cpp - https://github.com/ggml-org/llama.cpp
- Ollama - https://docs.ollama.com/api-reference/show-model-details
- Ollama Context - https://docs.ollama.com/faq
- QLoRA paper - https://arxiv.org/abs/2305.14314

