Decode tok/s — writing speed
How many tokens the model writes per second. A token is about three-quarters of a word. You read at roughly 5–8 tok/s, so anything above 15 feels quick in a chat. Coding agents want 30 or more, because they write a lot and you wait on them.
Prefill tok/s — reading speed
How fast the model reads what you give it before it starts answering: your question, your files, the conversation so far. Agents re-read everything after each step, so on long jobs this matters as much as writing speed. Graphics cards beat Macs by far more here than on writing. It's also the least precise column.
Weights, KV and Total — memory needed
Weights is the model itself. KV is its working memory for the current conversation, and it grows as the conversation gets longer. Total is both plus about 1.6 GB of overhead. If Total is more than your machine has free, the row turns grey.
Weight quant — Q4, Q5, Q6, Q8
How much the model file is compressed. Each model lists every version on its own line, smallest first. Q4 is the smallest and fastest, Q8 is almost identical to the original but twice the size and roughly half the speed. Models that only ship in one format show one line.
Perplexity vs Q8
How much quality the compression costs. Give a model real text and have it guess each next word: perplexity is roughly how many words it's torn between at each step, so lower means surer and more often right. The percentage is how much that uncertainty rises against the uncompressed model — Q8 is the reference, since it's within a rounding error of the original.
It doesn't mean that share of words comes out wrong. Most output is identical; the loss shows as an occasionally worse word choice, and those slips compounding across a long chain of reasoning or a multi-step coding job. Short answers rarely show it, long agent runs do.
These are estimates, not measurements: published llama.cpp quant deltas measured on a 7B model, scaled by size, since bigger models lose less. Mixture-of-experts models are sized by their experts rather than their total, which is why a 35B model with 3B active loses more than a 27B dense one.
KV cache precision
Compression for the working memory, not the model. 8-bit halves it with almost no downside — twice the context, or twice the agents, in the same memory. 4-bit saves more but starts losing details in long conversations. It's chosen when you start the server, separately from the weight quant.
External GPUs
Pick a laptop card and you can attach one or more desktop cards in Thunderbolt enclosures. Both are seen as ordinary GPUs, so the model is simply split across them — you get their memory added together, at the speed of the slower one. Tensor parallel isn't offered here: it needs constant traffic between the cards every layer, and Thunderbolt is nowhere near fast enough. Apple Silicon can't do this at all — eGPU support ended with the Intel Macs.
Laptops with integrated graphics — Intel Arc, Ryzen AI Max — can take an enclosure too, but nothing pools: one process can only drive one kind of backend, so the external card works alone and the laptop's own memory sits idle. The figures then are simply the card's.
CPU offload
When a model is too big for the card, llama.cpp can keep some layers on the GPU and read the rest from ordinary system RAM. Rows that only run this way are tinted and tagged with how much stayed on the card.
It works, but system RAM is three to ten times slower than a graphics card's, so writing speed drops towards the slower of the two. Mixture-of-experts models cope far better than dense ones, because only a small slice of the weights is read per word. Prefill suffers worst of all. And it costs you agents: whatever VRAM the model doesn't use is what's left for working memory, so offloaded setups usually run one.
VRAM limit on graphics cards
The driver always keeps a little of each card for itself. OS default assumes the card is also running your monitor, which takes close to 1 GB. Raised is a headless server — nothing plugged in, nothing else using the card — so only the driver's own share is kept, about half a gigabyte. Pushed uses almost the whole card. It's fine on a dedicated inference box, but there's no swapping on a graphics card: run out and the job crashes rather than slowing down. Full counts every byte with nothing held back — the theoretical ceiling, useful for comparing, but slightly more than any card really gives you.
GPU memory limit — default, raised, pushed
Macs and other unified-memory machines don't let the graphics side use all the memory by default. macOS allows about two-thirds on smaller machines and three-quarters on larger ones; Linux on the Ryzen AI Max allows about half. Raised is the usual one-line tweak, leaving about 15% for the system. Pushed leaves only the bare minimum — more room for the model, but close everything else, because running out makes the machine swap or freeze. Full counts all of the memory for the model; no machine actually gets there, since the OS needs some to run, so it's there as a ceiling to compare against. The page shows the exact command for whichever real setting you pick.
Compare settings
Pick a model and it shows every combination of weight quant and KV cache precision side by side, for the machine you chose in Pick hardware — plus how each KV setting holds up as context grows. It's the quickest way to see what a setting actually costs or buys you before changing it.
Context
How much text the model keeps in mind at once. 32K is roughly 25,000 words — a good chunk of a codebase. More context costs more memory and slows writing, because the model looks back over all of it for every word.
Agents and per agent
How many AI workers can run on the machine at the same time, and how fast each one writes. The model loads once and is shared; each agent only needs its own working memory. An agent counts as long as it still gets at least 15 tok/s.
Class
A rough capability tier — small, mid, capable, strong, frontier — for general smarts. It's a band, not a score, because no single test covers all of these models.
SWE V and SWE Pro — coding tests
Each test is a real bug from a real software project. The model gets the code and the bug report and has to write the fix; hidden tests then check it. The number is the percentage of bugs it fixed.
V (Verified) is the older, easier test: top models now score around 95%, and its answers may have leaked into training. Pro is the current, harder one, and the same model scores 15–25 points lower on it. Read Pro when it's shown — around 60 is today's best open models. A dash means never tested, not a bad score. Never compare a V number with a Pro number.
Badges
Worked out for the machine and settings you've selected, so they move when you change anything. Top class that fits is the most capable model still writing at 15 tok/s or more. Best at speed is the most capable one that manages 30. Fastest usable is the quickest model above the small class.
Several graphics cards
Layer split is the easy default: the model is divided between the cards and each word passes through them one after another. You get all their memory, but the speed of one card. Tensor parallel makes every card work on every word at once — close to their combined speed, but harder to set up, and the cards need to match. NVLink is a fast direct connection between cards that makes tensor parallel work better.
Linking machines together
Several Macs or servers can pool their memory to run a model none of them could hold alone. That adds memory and lets more agents run at once, but each answer is no faster than the slowest machine in the group. A 10 Gb network is enough; faster cables buy almost nothing for this.
Asterisks and prices
Memory sizes marked with an asterisk aren't sold by Apple for that chip — they're there so you can compare. Prices are rough and moved a lot during 2026, so check before buying.