Use an LLM VRAM calculator that shows its work: params x bytes-per-parameter estimates model weights, plus 2 x kv_heads x head_dim x layers x context x batch x bytes for the KV cache. This calculator shows its work instead of hiding behind a black-box score. The calculator is validated against a real 8GB machine, and it is on this page: enter a model's parameters and quantization, pick a context length, and see whether it fits in the RAM or VRAM you have.

Key takeaways

  • The whole formula is two lines: weights = params times bytes-per-parameter; KV cache = 2 times kv_heads times head_dim times layers times context times batch times bytes-per-element.
  • Every architecture field (kv_heads, head_dim, layers) comes straight from a model's own metadata, the same numbers ollama show --verbose prints, not a guess.
  • Validated against five real models on an M1/8GB field test: every prediction landed within 4.7-9.8% of measured memory.
  • A first-guess 15%+600MB runtime overhead was off by 55.2% against real measurements. The current calculator uses 3%+75MB because that is what the validation data supports.

How the local LLM hardware calculator works

The calculator on this page takes a model's parameter count, its quantization level, and its attention architecture, then computes three numbers: model weight size, KV cache size at your chosen context length, and a small runtime overhead. Weights are params x bytes-per-parameter; a model quantized to Q4_K_M uses roughly 0.6 bytes per parameter, while Q8_0 uses roughly 1.07 bytes per parameter, both measured directly from five installed models rather than a vendor spec sheet (Source: Ollama).

Unlike the calculators already ranking for this query, this one prints its formula on the page instead of returning only a final number. APXML's VRAM calculator, for example, accepts model, quantization, batch size, and sequence length as inputs but does not publish the equations behind its output on the page itself (Source: APXML).

How to use the LLM VRAM calculator

Start with a preset when you want a quick answer, or choose custom inputs for a model that is not in the validation table. The six inputs below determine the estimate. Do not replace an unknown field with the model's parameter count: architecture metadata changes the KV-cache term.

InputWhere to get itWhy it matters
Parameter count and quantizationollama show <model> --verboseSets the weight-memory term
KV heads, head dimension, and layersollama show <model> --verbose or GGUF metadataSets KV-cache growth
Context lengthYour intended num_ctx settingLarger context requires more memory
Batch sizeExpected concurrent requestsKV-cache demand scales with concurrent work
KV-cache precisionRuntime setting, f16/q8_0/q4_0Lower precision uses less cache memory
Available RAM or VRAMThe hardware specification, minus a safety marginDecides whether the estimate is usable

For a cautious first pass, enter the context length you actually plan to use, leave room for the operating system, and treat a result close to the hardware limit as a warning rather than a green light. Ollama's documentation also recommends checking ollama ps to see whether the model is fully on the GPU, fully in system memory, or split between both (Source: Ollama Context).

VRAM vs RAM: what changes

Apple Silicon machines like the M1 MacBook Air use unified memory, meaning RAM and the GPU's usable memory are the same pool with no separate VRAM figure to budget for. On a discrete-GPU PC, VRAM is a hard, separate ceiling: whatever doesn't fit in VRAM either fails to load or falls back to slower CPU/system RAM, which this calculator does not model (see Limitations). The arithmetic for weights and KV cache is identical either way; only which memory pool you compare the total against changes. (Source: Apple Support)

Quantization and bytes per parameter

Measured in our test: bytes-per-parameter is not a vendor estimate here. We measured it directly from five installed Ollama models by dividing each model's on-disk file size by its exact parameter count from ollama show --verbose: llama3.2:1b at Q8_0 came out to roughly 1.07 bytes/param. The same method applied to every Q4_K_M model in the preset list landed closer to 0.64 bytes/param, all baked into the calculator's presets so a reader doesn't have to guess (Source: Hugging Face).

Tim Dettmers and the QLoRA authors describe their method as reducing memory use enough to fine-tune a 65B-parameter model on a single 48GB GPU (Source: QLoRA paper). That training result is not an inference guarantee, but it explains why the calculator treats quantization as a first-class input instead of hiding it inside a model-size label.

Context length and the KV cache

The KV cache formula needs three architecture numbers most calculators hide: kv_heads (attention heads dedicated to keys/values, which can be fewer than the total attention head count under grouped-query attention), head_dim (dimension per head), and layer count. All three come from the model's own GGUF metadata, the same fields ollama show --verbose prints (Source: llama.cpp). Doubling context length exactly doubles the KV cache term; doubling batch size does the same, while model weights stay fixed regardless of context or batch.

Using it with Ollama specifically

Ollama's default KV cache type is 16-bit floating point, meaning 2 bytes per cached value regardless of how aggressively the model weights themselves are quantized (Source: Ollama). The calculator defaults to that setting but lets you switch to an 8-bit or 4-bit KV cache to match a non-default Ollama configuration. This calculator's presets are the same five models measured in the M1/8GB test on an 8GB M1 MacBook Air, so the two pieces of work check each other.

What this calculator can and cannot predict

The calculator is a memory estimate, not a universal “will this feel fast?” answer. It handles the weight term, KV-cache term, selected cache precision, batch input, and a measured runtime-overhead buffer. It does not model every runtime's scheduler, CPU/GPU split, mixture-of-experts residency, or the effect of background applications.

It estimatesIt does not promise
Weight memory from parameters and bytes per parameterTokens per second on different hardware
KV-cache memory from architecture and contextThat a model will stay entirely in VRAM
A calibrated overhead bufferExact memory for MoE or partial offload cases
How context and batch changes affect the totalThat a green result means a comfortable user experience

Ollama notes that parallel requests multiply the effective context demand, and that cache quantization changes memory use. Those runtime settings are why the calculator exposes batch size and KV precision instead of hiding them behind one hardware score (Source: Ollama Context).

Validated against a real machine

We ran the calculator's formula against five real models from an 8GB M1 MacBook Air and compared predicted total memory to each model's actual resident memory after load. The model records and local measurement path are reproducible with Ollama's model tooling (Source: Ollama). Measured in our test: mistral:7b came out to a 9.8% delta between prediction and measurement, llama3.2:1b to 6.3%. Every one of the five models tested landed under a 10% delta, the numbers printed on the calculator's own validation table, not a cherry-picked example.

Getting there took a real correction, and it is worth telling honestly rather than hiding. The calculator's first version used a reasonable-sounding overhead assumption before any data existed: extra runtime memory beyond weights and KV cache, guessed as a chunky percentage plus a large flat buffer. Measured in our test: replayed against the same field-test measurement, that first guess produced a 55.2% delta on the smallest model, an order of magnitude worse than the current version's largest error. Real runtime overhead on this machine turned out to be a small single-digit percentage of the weights-plus-KV-cache subtotal, not the double-digit figure a first guess would assume. The published calculator uses the measured, much smaller overhead instead.

What LLM can I actually run locally?

A worked example: an 8GB machine, phi3:mini (3.8B parameters, Q4_0 quantization, 32 layers), and a 4096-token context. Plugging those into the calculator gives roughly 3.71GB predicted, comfortably under 8GB, and the field test's measured figure was 3,624 MB, an 4.7% delta. The same machine asked to run mistral:7b at the same context length predicts about 4.78GB, which still fits 8GB on paper, but a the M1/8GB test measured real swap activity and a steep speed drop at that size that the calculator's memory math alone does not capture. The model identifiers and architecture metadata can be inspected with Ollama's local model tooling (Source: Ollama).

Where this estimate is wrong

Measured in our test: the calculator's automated checks passed every check in this run, including the cross-check against the M1/8GB test, but passing tests does not mean the formula is universal. Five stated failure cases: mixture-of-experts models (Mixtral, DeepSeek-MoE) use total param count here, but MoE models only activate a subset of experts per token, so the calculator overestimates them. Split CPU/GPU offload is not modeled; the formula assumes one memory pool, which breaks on a discrete-GPU machine offloading only some layers. Non-default KV cache quantization changes real usage below the calculator's f16 default unless you change the input yourself. The 3%+75MB overhead default is calibrated to one machine (Apple M1, Ollama) and will not transfer exactly to Linux or Windows. Concurrent batch requests do not scale linearly the way the KV cache term here assumes, once real server activation memory is involved (Source: llama.cpp).

FAQ

The answers below assume an Ollama-style local runner and the measured boundary described above (Source: Ollama).

How to calculate LLM VRAM usage?

Multiply parameter count by bytes-per-parameter for the model weights, then add the KV cache: 2 times kv_heads times head_dim times layers times context length times batch size times bytes per cached value. Add a small runtime overhead, typically single-digit percent on top of that subtotal based on the validation measurements.

How much VRAM for LLM models?

It depends on size, quantization, architecture, and context. In our validated test set, models from 1.2B to 7.2B parameters needed roughly 1.1GB to 4.9GB predicted, matching measured memory within 4.7-9.8% across all five models tested. Use the calculator with the target model's metadata before choosing hardware.

What LLM can you run with 16 GB of VRAM?

With headroom to spare, 16GB covers every model class in our validated range, from 1.2B to 7.2B parameters, at the 4096-token test context. It may support larger models or longer contexts, but those cases are outside this validation set. Enter their architecture metadata instead of treating 16GB as a universal guarantee.

What specs do I need to run a local LLM?

Enough RAM or VRAM for the model's weights plus its KV cache at your intended context length, with a small overhead margin. Our calculator's presets show that 1.2B-3.8B models fit comfortably in 8GB; 7B-class models fit in principle but push closer to that ceiling.

Which local LLM is best for 24GB of VRAM?

24GB opens up meaningfully larger models and longer context windows than this calculator's validated 1.2B-7.2B test range. Use the calculator's custom-input fields with a larger model's architecture numbers (available from ollama show --verbose) to check a specific model before downloading it.

Related coverage

References