Model assumptions
Free GPU planning tool
Free LLM VRAM Calculator
An LLM VRAM calculator estimates the GPU memory required to load model weights and serve a context window. This tool separates weights, KV cache, and runtime overhead so you can plan hardware with transparent assumptions.
Quantization-awareKV-cache estimateGPU fit guidePrivate browser calculation
Estimate VRAM How the VRAM estimate works
1. Model weights
Parameters x bits per weight / 8. Quantization lowers this largest fixed component.
2. KV cache
2 x layers x context tokens x hidden size x concurrent sequences x cache bytes.
3. Runtime overhead
A planning allowance for framework buffers, kernels, metadata, and memory fragmentation.
This is a standard multi-head inference approximation. Grouped-query or multi-query attention can use a smaller KV cache, while training, activations, temporary workspaces, and some quantization formats can use substantially more memory.
Straight answers
LLM VRAM calculator FAQ
What is an LLM VRAM calculator?
An LLM VRAM calculator estimates the GPU memory needed to load model weights and serve a context window. It combines weight precision, KV cache, concurrency, and runtime overhead into one planning estimate.
How much VRAM does a 7B or 8B model need?
Weights alone are about 14-16 GB at FP16, 7-8 GB at 8-bit, and 3.5-4 GB at 4-bit. KV cache and runtime overhead increase the real requirement, especially with long contexts or concurrent requests.
Why does context length increase VRAM use?
The KV cache stores attention keys and values for previous tokens. Its memory grows roughly linearly with context length, layer count, hidden size, precision, and the number of concurrent sequences.
Does 4-bit quantization make every model fit in one GPU?
No. Quantization reduces weight memory, but KV cache, activations, framework buffers, and quantization metadata still consume VRAM. Large models may still need multiple GPUs or CPU offloading.
Why can actual VRAM use differ from this estimate?
Grouped-query attention, flash-attention kernels, framework allocation, training versus inference, quantization format, fragmentation, and offloading all change actual memory use. Treat the result as a capacity-planning baseline.
Keep planning your AI workload
Compare runtime costs, count prompt tokens, or estimate the energy footprint of your next deployment.