← All tools

Local AI Hardware Sizer

Whether a model runs on your machine comes down to VRAM, and VRAM comes down to two choices: how hard you quantize the weights, and how much context you keep. This is an honest rule of thumb — the linear KV estimate approximates a typical dense transformer, and real usage varies by architecture — but it gets the decision right: which class of card, if any, holds the model you want.

A rule of thumb, not a benchmark — the KV estimate varies by architecture, and runtimes differ. Leave a few GB of headroom.

Weights at this quantization

16.00 GB

KV cache at this context

0.02 GB

Total VRAM needed · ×1.1 overhead included

17.62 GB

Fits a 24 GB card.

Weights are just parameters × bytes — halve the bytes per parameter and the same card holds twice the model, which is why quantization is the whole ballgame for local AI. The KV cache then grows with every token of context you keep, quietly eating headroom people budget only for weights.

How to read it: quantization is the whole ballgame — halving bytes per parameter doubles what fits, which is how a model that demands an 80 GB tier at full precision lands on a 24 GB card at int4, usually with modest quality loss. Context length is the budget line people forget: the KV cache grows with every token you keep, so a long-context session can evict a model that fit on paper. Size for the tier above your total when you can — runtimes, drivers, and your desktop all want a slice of the same memory.

Embed this tool

Paste this snippet into any page to run the calculator there.