Most advice about local AI assumes you have a big card. This is about the other situation, which is far more common: a gaming laptop with 8 GB of video memory, already bought, not being replaced.
We borrowed one for a night. An Asus TUF with an RTX 3070 Laptop GPU, 8 GB of video memory, 16 GB of system memory, running Ollama 0.33.3. We drove it over Tailscale from another room and left it working. Eight models, ten tasks, and a separate sweep of five models across five context lengths.
Every row is free to download, including the runs that went wrong and the harness that produced them. No email.
The thing we did not expect
A model that does not fit is not refused. Ollama serves the part that does not fit from system memory, across the PCIe bus, and that is where the speed normally goes.
So we expected the ranking to follow the file size. It did not.
| model | size on disk | share NOT on the card | tokens/sec |
|---|---|---|---|
| gemma3 4B | 2.7 GB | 0% | 72 |
| qwen2.5-coder 7B | 4.4 GB | 0% | 49 |
| mistral 7B | 4.6 GB | 0% | 48 |
| llama3.1 8B | 4.9 GB | 0% | 46 |
| qwen3 8B | 5.2 GB | 0% | 43 |
| qwen3 30B (a3b) | 17.9 GB | 67% | 15 |
| phi4 14B | 9.5 GB | 37% | 4.9 |
| qwen3 14B | 9.5 GB | 37% | 3.9 |
Read the bottom three rows again. The 30 billion parameter model is nearly twice the size of the 14 billion parameter models, spills nearly twice as much off the card, and runs three to four times faster.
Why, and it is not magic
qwen3:30b-a3b is a mixture of experts. The name carries the answer: a3b means roughly 3 billion active
parameters per token. All 30 billion are loaded, but only a small slice does work on any given token.
So the number that predicts speed is not how big the file is. It is how much of the model has to be touched to produce one token, and whether that part is on the card.
A dense 14B has to touch all 14 billion parameters every time, and 37% of them are sitting in system memory. That is the worst combination available on this hardware: too big to fit, too dense to spill gracefully. On an 8 GB card the 14B tier is the one size to skip.
What fits completely
Five models never touched system memory at all, at the default context:
gemma3 4B at 2.7 GB is the fastest thing on the card by a wide margin. The two 7B models land around 48 tokens a second. llama3.1 8B and qwen3 8B are the biggest of the comfortable ones and still clear 43.
For scale, and to be fair to expensive hardware: the same test on our RTX 5090 desktop runs a 26 billion parameter model at 237 tokens a second. The laptop is not competitive with a two thousand dollar card and nobody should pretend otherwise. What it is, is usable. Forty tokens a second is faster than you read.
Context is what actually breaks it
This is the part most benchmarks skip, and it is the one that will bite you, because the context window comes out of the same 8 GB. A model that fits perfectly in a short chat starts spilling in a long one.
We loaded each model at five context lengths and measured what stayed on the card.
| model | 2k | 4k | 8k | 16k | 32k |
|---|---|---|---|---|---|
| gemma3 4B | 0% | 0% | 0% | 0% | 0% |
| qwen2.5-coder 7B | 0% | 0% | 0% | 0% | 8% |
| mistral 7B | 0% | 0% | 0% | 8% | 31% |
| llama3.1 8B | 0% | 0% | 0% | 13% | 34% |
| qwen3 8B | 0% | 0% | 0% | 20% | 38% |
gemma3 4B never leaves the card, even at 32k. That is a genuinely different proposition from the others and it is why it is our pick for long conversations rather than for raw quality.
Everything in the 7B to 8B range is comfortable to 8k and starts sliding after that. If you hold long chats with qwen3 8B at 32k, more than a third of it is being served from system memory and you will feel it.
Speed is not the ranking
We gave every model the same instruction: rewrite this as five bullets and keep every number. Six numbers were in the source.
qwen3 8B kept all six. mistral 7B kept five. llama3.1 8B kept four. gemma3 4B, the fastest model on the card by a mile, kept two. The Qwen coder kept one.
Which tells you something obvious in hindsight. A model tuned for code is the wrong tool for ordinary writing, and it is the worst on the list at it. The fastest model on this card is the worst at following an instruction you actually gave it. If you are picking one model to live on an 8 GB laptop and you mostly write and summarise rather than code, the speed column is not the column you should be reading.
What to actually install
| if you mostly | install |
|---|---|
| write, summarise, ask questions | qwen3 8B |
| hold long conversations | gemma3 4B, the only one that never leaves the card |
| write code | the qwen coder, and nothing else |
| want to punch above the card | qwen3 30B a3b, at about 15 tokens a second |
| were considering a 14B | do not, it is the worst of both |
If your card is an RTX 3060, a 4060, or an old GTX 1080, this is the same 8 GB and the same wall.
Four things we got wrong
These are in the data too, which is the point of publishing the failures.
Seven empty answers looked like model failures. They were our own token budget running out. The tell
is in the file: eval_count equals the budget exactly on those rows.
We wrote up a fix before testing it. A reasoning-effort setting we expected to shorten thinking moved it by about 4%. It is not in the recommendations because it is not a recommendation.
A context sweep reported that none of the models fit. The laptop had gone to sleep. Every measurement errored and the summary read like a finding. The sweep script now refuses to write a result where every cell failed.
A scoring check failed a correct answer over a hyphen. It wanted “eight gigabyte” and the model wrote “eight-gigabyte”. It also only matched word forms and graded a gradient as pass or fail.
The rule we took from all four: when a test fails everything, audit the test before you believe it.
What is not in here
Quantisation. How aggressively a model is compressed changes every number above, and doing it properly is a whole separate bench rather than a footnote. That is the next one.
And the honest limit on everything above: one laptop, one night, one run per cell. These are our numbers on our hardware, not a law of nature.
Take the data
All 103 rows, the raw JSON, and the harness, free, no email.
The harness talks to Ollama over HTTP, so the machine under test does not have to be the machine you are sitting at. Point it at your own laptop over your own network and get your own numbers, which is the only way any of this is worth anything. We would genuinely like to see yours next to ours.
Related: which AI rig do you need if you are deciding what to buy rather than what to run, and local AI for beginners if none of this is installed yet.