The reasoning tax on a local LLM, measured on a 16GB-VRAM GPU

Every recent open-weight model ships with a thinking mode, and every model card presents it the same way: turn it on for hard problems, get better answers. On a datacenter GPU that trade looks attractive, because the extra tokens arrive fast. On a 16GB consumer card the same extra tokens arrive at forty or fifty per second, and a model that thinks for ten thousand tokens before answering makes you wait several minutes for something you could have had in seconds. I wanted a number for that trade, not an impression, so I measured it on my RTX 5060 Ti with the five models I use or have evaluated seriously: Qwen3.8-27B in two quantizations, the Ornith-1.5-35B-A3B mixture of experts, and two 9B models. Two kinds of measurement go into this article: the time you wait for an answer, on eight everyday probes with reasoning off and on, and the number of programming problems solved, on fifty Python problems with executable tests. Put side by side, the two show when thinking is worth what it costs on this hardware.

What exactly is the reasoning tax?

It's the extra time you wait, and the extra energy you pay, for the tokens a model writes before its answer. Those tokens aren't shown to you, or they sit in a collapsed block you hardly ever open, yet they are generated at the same speed as the answer. A model that decodes at 57 tokens per second and thinks for 9,000 tokens makes you wait two and a half minutes before the first word of the answer appears.

Throughput benchmarks hide this completely. Tokens per second is the rate at which the model writes, and it's the same with thinking on or off. What changes is how much the model chooses to write, and that is a property of the model's habits, not of the engine.

A thinking model writes at the same speed per token, and then writes a long letter to itself before it writes to you.

If you are choosing a local model for a team and want this trade measured on the tasks your people actually do, the approach I use is described in my hub on AI infrastructure for companies.

How I measured the wait

Eight probes, two for each of four profiles a developer meets daily: a short chat question, a request for a single shell command, a code review of a small snippet, and a long prompt of about 4,300 or 8,500 tokens with a question at the end. Each probe ran three times with reasoning off and three times with reasoning on, on the same server, for each of the five models.

Reasoning off used temperature zero with the thinking switch disabled in the chat template. Reasoning on used the sampler recommended by the Qwen family for thinking (temperature 0.6, top-p 0.95, top-k 20), a different seed for each repetition, a 32,768-token budget and a 900-second timeout, so that a long thought wouldn't be cut and mistaken for a fast answer. The prompt cache was disabled, so every repetition paid its own prefill, as a fresh question does for a user.

The tax, model by model

The median wait grew 14.4 times on the GSQ-RCO 27B and 4.6 times on the Ornith 35B mixture of experts. Across all probes, the 27B went from 5.9 to 131 seconds, the 35B from 2.7 to 8.7.

With reasoning off, the same medians were 2.7 s for Ornith-1.5-35B-A3B, 3.8 s for Ornith-1.5-9B, 4.3 s for NeoHorse-1-9B, 5.9 s for the GSQ-RCO 27B and 8.7 s for the IQ4_XS 27B.

The median hides how unevenly the tax falls across tasks. On long prompts it's mild, since the model spends most of the time reading anyway. On short requests, the kind a fast model without thinking answers in a couple of seconds, it's huge.

The shell command row is the one I remember. Asked for a single command, the 27B thought for almost nine thousand tokens, about 98% of everything it wrote by my estimate, and answered after three minutes and twenty seconds. Without thinking, the same model gave a correct command in under three seconds. The tax is paid in tokens written: the 27B decodes at about half the speed of the 35B, but with reasoning on it writes between 1.8 and 6.4 times as many tokens.

Nothing was truncated at the 32,768-token budget. One probe of the IQ4_XS 27B, a code review, ran past the 900-second timeout and produced no answer at all.

Does thinking at least buy better code?

The latency probes don't grade the answers. For quality I used fifty Python problems written for the purpose, with 560 executable assertions: text encoding traps, date arithmetic around daylight saving changes, Italian holidays computed from Easter, payment and tax code validation, SQL window functions, bug fixes with misleading comments. A problem counts as solved only when every assertion passes. Each model got a time limit: two minutes of usable time and four of hard cap per problem with reasoning off, five minutes of usable time and ten of hard cap with reasoning on.

With reasoning off, the GSQ-RCO 27B solved 34 problems and the Ornith 35B 19. With reasoning on, within five minutes, the order reversed: 26 for the 35B and 12 for the 27B. Both differences are confirmed by a paired test on the same problems, and both were declared as the questions of the campaign before any data existed: 17 problems to 2 with reasoning off (p = 0.0007, 0.004 after the Holm correction), 16 to 2 with reasoning on (p = 0.0013, 0.007 after Holm).

The reversal looks like a capability finding and it's mostly a time finding. All 28 failures of the 27B with reasoning on were answers still being written when the ten-minute cap arrived; of the 22 answers it did finish, all 22 were correct. The 35B mixture of experts thinks at twice the speed and finishes in time. At ten minutes the solved counts were 31 for the 35B and 22 for the 27B, a difference the paired test doesn't confirm (p = 0.078).

The 9B models tell the opposite story to the 27B. Both solved only 8 problems without thinking, and with thinking NeoHorse reached 18 within five minutes. For a small model, reasoning buys real problem-solving; for the dense 27B on this card, it mostly buys waiting.

What 20 minutes of thinking bought

The obvious follow-up was to give the 27B the time it was asking for. I reran every problem that had been cut at ten minutes, with the same model, seed and settings, and a 20-minute cap.

With 20 minutes, the GSQ-RCO 27B solved 11 of its 28 cut problems, left 10 still running, and delivered 7 wrong answers. The uniform IQ4_XS build solved 11 of its 36 cut problems, left 25 running and delivered no wrong answers. Because the same seed reproduced the start of the reasoning in 25 of 28 and 33 of 36 cases, the 20-minute run is the continuation of the ten-minute one, and the totals can be added; two of the 27B's thirty-three solved problems came from a different sample, which I count anyway and flag here.

With 20 minutes to think, the 27B arrived where it already was in thirty seconds without thinking. On this card, for this kind of problem, its reasoning buys time, not solved problems.

There is a second, less comfortable detail. Within ten minutes, the GSQ-RCO 27B never handed back a wrong answer: every failure was an unfinished one. Given 20 minutes it started to deliver wrong answers, seven of them. Longer thinking didn't only finish more problems, it also finished some of them badly. "This model doesn't give wrong answers when it reasons" is true only inside the ten-minute window.

The energy side of the same tax

The tax is paid in electricity as well as in time. On the same coding problems, the marginal energy per solved problem with reasoning off ranged from 0.5 Wh on the 35B to 4.6 Wh on a 9B. With reasoning on it ranged from 17.9 Wh on the 35B to 118 Wh on the IQ4_XS 27B. The cost per problem is still a fraction of a cent at my electricity price, so for one developer it decides nothing. It does matter at scale, and it's another view of the same fact: thinking multiplies the energy per solved problem by ten to sixty for this kind of problem, depending on the model.

Switching reasoning per request instead of per server

These numbers make reasoning a per-request choice, decided for each request instead of once for the server. With llama.cpp and the Qwen-family chat templates, you can start the server with reasoning enabled and disable it on individual requests through the template arguments. That way the same loaded model serves both modes and you pay no reload.

# Request without reasoning: the chat template receives enable_thinking=false.
curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "Find files larger than 100 MB under /var"}],
    "temperature": 0,
    "chat_template_kwargs": {"enable_thinking": false},
    "max_tokens": 512
  }'

# Same request with reasoning: recommended sampler and a large token budget.
curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "Find files larger than 100 MB under /var"}],
    "temperature": 0.6, "top_p": 0.95, "top_k": 20,
    "max_tokens": 32768
  }'

On the 27B, the first request answered this kind of question in under three seconds; the second took a median of more than three minutes on my shell command probes. Expose the switch to the user, or send short requests to the first form automatically, and most of the tax goes away without giving up reasoning where it helps.

Where the time goes inside a thinking answer

The latency harness estimates, from the characters in the reasoning channel, how many of the generated tokens were thinking. The share is striking: on the chat, command and review probes of the GSQ-RCO 27B, between 79% and 98% of everything generated was reasoning, the top of the range on the command probes. On the MoE the share was 78% to 95%. On the long-prompt probes the estimate is too coarse to quote. So the visible answer is a small tail at the end of a long internal monologue, and the length of the monologue sets nearly all of the time it takes to get there.

Two consequences follow. Streaming the reasoning to the user doesn't shorten the wait for the answer, it only makes the wait visible, which some people prefer and others find worse. And a model's speed advantage only translates into a shorter wait if its monologue isn't proportionally longer: the MoE wins on waiting time not only because it decodes twice as fast, but because on most probes it also thinks for fewer tokens than the 27B.

When thinking is worth it on a 16GB card

The data suggests a few rules that I now follow myself.

Leave reasoning off for short requests. A shell command, a regular expression, a quick explanation: with thinking off, all five models answered in a few seconds. With thinking, the 27B took minutes on the same requests.

Turn it on for small models on hard problems. With reasoning the 9B models solved more problems; NeoHorse more than doubled its count. Their thinking is short enough to fit in a reasonable wait.

Prefer the fast thinker when thinking is needed. The 35B mixture of experts reasons at twice the speed of the dense 27B and with fewer tokens. When a task needs reasoning and you are waiting for it interactively, it's the better tool on this card, and the paired test on coding agrees.

Treat the dense 27B as a non-reasoning model for interactive use. Its thinking-off answers were the best of the five on coding, and they arrived in seconds. Its thinking-on answers arrived minutes later and were not more often correct.

Cap the time explicitly. A reasoning model without a time or token budget can run for 20 minutes on one problem. Decide in advance how long an answer is worth to you, and enforce it.

These rules are specific to this hardware. On a card that decodes three times faster, the same tokens cost a third of the wait, and the balance shifts toward thinking. For this reason I measure on the hardware I own and don't rely on model cards. The configuration behind every number here is in the recommended strategies for local LLMs on a 16GB card, the measurement traps I had to fix first, including a prompt cache that faked fast repetitions, are in what llama.cpp benchmarks get wrong, and the energy figures come from the same campaign as the real cost of a dedicated 16GB card. The thinking sampler I used is the one recommended on the Qwen3.8-27B model card, and the engine is upstream llama.cpp.

Reasoning is a budget you spend in seconds and tokens, and on a 16GB card the exchange rate is poor for a dense 27B and good for a fast mixture of experts or a small model on a hard problem. The ranking of two models can reverse when you flip the switch, which means that any comparison that doesn't say whether reasoning was on is comparing two different experiments. If you are setting up local models for a team and want the reasoning policy to come from measured waits, not from a model card, you can describe the case in the free quote form: two minutes, and I tell you whether it's something I can help with.