Methodology

Every result is an estimate. Actual memory use depends on runtime, version and settings (batch sizes, flash attention, CUDA/Metal buffers, other apps). This page lists every formula, number and source the calculator uses, so you can check or change them.

1. Total memory

total = weights + KV cache + recurrent state + overhead

Weights

If a real GGUF file exists on Hugging Face for the selected model and quantization, its size is used (split files are summed). Otherwise: weights = parameters × bits per weight ÷ 8. Mixture-of-experts models use total parameters, because all experts must be in memory.

Parameter counts come from the unquantized (BF16/F16) GGUF file size ÷ 2 where available (text model only), otherwise from the model card or the Hugging Face safetensors count. Each model shows its source.

KV cache

Computed per group of layers from the model’s real config.json:

• Full attention: layers × 2 × kv_heads × head_dim × tokens × sequences × bytes. Grouped-query attention uses the real number of KV heads.
• Sliding-window / chunked layers (Gemma 2/3/4, gpt-oss, Llama 4): same formula, but tokens are capped at the window size.
• MLA (DeepSeek R1, GLM-4.7 Flash): layers × (kv_lora_rank + qk_rope_head_dim) × tokens × sequences × bytes — one compressed latent per token.
• Hybrid linear attention (Qwen3.5 / 3.6 / 3.8): only full-attention layers have a KV cache; linear-attention layers add a fixed FP32 state of v_heads × k_dim × v_dim × 4 plus the convolution state, per sequence.

KV precision (bytes per element), from llama.cpp’s block formats (ggml-common.h): F16 = 2 (16-bit float = 2 bytes); Q8_0 = 1.0625 (block_q8_0: 32 int8 values + one FP16 scale = 34 bytes / 32 values); Q4_0 = 0.5625 (block_q4_0: 32 4-bit values + one FP16 scale = 18 bytes / 32 values).

Overhead

Default 10% of weights. Source: Flavio Copes, “How much VRAM do you need to run an LLM locally?”: “A 1.1× multiplier on the weights is a reasonable estimate.” It covers runtime compute buffers, which llama.cpp logs as “compute buffer size”; compare with your own log. Adjustable under “Assumptions”.

2. Will it fit?

• Usable memory, discrete GPUs: 90% of VRAM. Source: Flavio Copes, “How much VRAM do you need to run an LLM locally?”: “Count on about 90% usable” — the driver and the desktop keep part of the VRAM. Check what is actually free with nvidia-smi (or rocm-smi). (nvidia-smi documentation)
• Usable memory, Apple Silicon: 72% of unified memory. Source: Flavio Copes, “How much VRAM do you need to run an LLM locally?”: “about 70–75% is realistically available for the model”. macOS caps the GPU’s share of unified memory (Metal’s recommendedMaxWorkingSetSize); llama.cpp prints this value at startup. (Apple Metal documentation)
• Multiple GPUs: memory is summed, minus 5% split loss. Assumption, not a published standard: layer splitting across GPUs leaves some memory unused on each card. Adjustable.
• Partial offload (discrete GPUs only): the model is treated as equal-sized layers, as llama.cpp’s -ngl splits it. GPU layers = ⌊usable ÷ (total ÷ layers)⌋; the rest must fit in system RAM (80% of the RAM you enter — an assumption: the OS and apps need the rest; adjustable). Apple Silicon has no partial offload because CPU and GPU share the same memory.
• Borderline: results within 5% of total device memory from the limit are flagged; small differences in runtime buffers decide those cases.

The defaults are conservative (overhead and reserve are both counted). Users with a headless GPU can raise usable memory; results near the limit should be tested before buying hardware.

3. Quantization formats

Bits per weight (bpw) are effective averages. GGUF values were measured by llama.cpp on Llama-3.1-8B; other architectures differ slightly because embeddings and output layers are quantized differently. When a real GGUF file exists for the selected model, the calculator uses the real file size instead of this value. Last checked 2026-10-18.

4. Validation

The formula (parameters × bits ÷ 8) is compared with published reference sizes: real GGUF files on Hugging Face and figures from the llama.cpp documentation. Required tolerance: ±10% for every case. Current mean absolute error: 1.6%, max 8.1%. These cases run as automated tests on every build. When a real file exists, the calculator uses it directly instead of the formula.

* Decimal GB (10⁹ bytes) to match the published figures. Everywhere else the calculator uses binary GB (1024³ bytes), the unit GPU vendors use for VRAM.

The KV-cache formula itself is exact arithmetic over the model config; for example Llama 3.1 8B at 8,192 tokens in FP16 is exactly 1 GiB (32 × 2 × 8 × 128 × 8192 × 2 bytes). What is not validated yet: runtime buffers on specific GPUs. If you measure real usage, the numbers from your runtime log are the best reference.

5. Data sources and freshness

• Models (44): config.json, parameter counts and GGUF file listings from Hugging Face, synced 2026-10-18 by an automated job that opens a pull request for human review when anything changes. Ungated mirrors (Unsloth) are used where the original repo is gated; architectures are identical.
• Hardware (54): Memory sizes, memory types and bandwidth were checked against the linked manufacturer pages on 2026-10-18. ‘GB’ is the vendors’ unit, which for VRAM and unified memory equals GiB (1024³ bytes). Bandwidth is left blank where the manufacturer does not publish it (NVIDIA GeForce and RTX A6000 pages list only size, type and bus width; the MI300X page shows no bandwidth figure). Bandwidth is not used in any calculation yet. Checked 2026-10-18.
• No live prices. Cloud providers are linked without prices; we do not show hardware prices.

6. Known limitations

• Runtime compute buffers grow with batch/ubatch size and differ between CUDA, ROCm, Vulkan and Metal; the overhead percentage approximates them.
• Vision projectors (mmproj files) for multimodal models are not included.
• For Gemma 4 the KV cache is shown as an upper bound: the architecture can share keys and values.
• Partial offload assumes equal layers; embedding and output layers differ in size.
• Speed (tokens per second) is not estimated yet.
• AWQ/GPTQ/MLX sizes use the formula only; embeddings often stay 16-bit, so real files are somewhat larger.

7. Licensing and trademarks

This site describes models and links to them; it does not host or redistribute model weights. Model and hardware names are used descriptively; no endorsement is implied.