The Real Cost of a Self-Hosted Coding LLM: Energy, ROI and Concurrency on a 16GB GPU

In the companion benchmark I put two 9-billion-parameter coding agents through a full agent loop on the same 16GB NVIDIA RTX 5060 Ti and concluded that both are genuinely usable, and that the choice between them is a question of path economy, not raw capability. That answers which model. This piece answers the question a decision-maker asks next, and the one that actually decides whether a sovereign coding stack is a good idea or an expensive hobby: what does it cost to run, and how far does one card stretch? I measured the power draw on the card rather than guessing from the TDP, priced the energy against the 2026 list rates of the APIs and GitHub Copilot, and mapped the VRAM budget to see how many developers a single 16GB GPU can actually serve. It sits alongside the two earlier parts of the series, the strategic case for local inference and the 35B MoE teardown.

Full disclosure, unchanged across the series: I use Claude and Fable in production every day, and I am quoting Anthropic's own published prices below. This is not an anti-vendor argument, it is a cost model. The frontier APIs remain superior in raw capability; the question here is narrower and purely economic.

TL;DR

• Measured, not guessed: a 9B coding model draws about 131 W under sustained decode on this 16GB card, roughly 0.83 kWh per million output tokens.
• Marginal cost is one to two orders of magnitude below the APIs: about a quarter-euro of energy per million output tokens against $5 (Haiku 4.5) up to $50 (Fable 5). You do not need a euro-dollar conversion to see a gap that size.
• The card pays off on token volume, not autocomplete: a heavy agentic developer or a small team recovers a ~€500 card in months; a solo developer who only wants inline completion never beats Copilot's flat $10 unlimited plan.
• 16GB is a single-active-stream device: roughly 580,000 tokens of total KV-cache budget, and the fast path (-np 1, speculative decoding) is single-slot by design. You scale by adding cards, not sessions.
• The real case is utilization plus sovereignty: self-hosting wins for sustained multi-seat agentic work and for data that legally cannot leave, not as a blanket "cheaper for everyone".

What does a self-hosted coding model cost to run?

Almost nothing per token in energy, but the honest number has to be measured, not derived from the card's rating. The RTX 5060 Ti carries a 180 W TDP, and the lazy way to model cost is to assume it runs there. It does not: decode on a quantized LLM is memory-bandwidth bound, so the compute units are often idle waiting on VRAM, and the card never approaches its power ceiling during generation. Sampling the board power with nvidia-smi during sustained decode gives the real figures.

From there the arithmetic is direct: at ~131 W and ~44 output tokens per second, one million output tokens takes about 6.3 hours of decode and burns roughly 0.83 kWh. Prefill (reading the prompt) is an order of magnitude cheaper per token, so output tokens dominate the energy bill, exactly as they dominate the API bill. Converting that to money needs a price for electricity, and here I use a verifiable, dated source rather than a round guess: the Eurostat household electricity statistics put the EU average at €0.29/kWh in the second half of 2025, ranging from €0.11/kWh in Hungary to €0.40/kWh in Ireland (Germany sits near the top at €0.39). So a million output tokens costs about €0.24 of electricity at the EU average, and between roughly €0.09 and €0.34 depending on which member state you plug the card into.

Is a self-hosted model actually cheaper than the API?

On the marginal cost of a token, yes, by one to two orders of magnitude, and the gap is so wide that the euro-versus-dollar question does not even change the answer. Anthropic's published API prices (checked July 2026) bill output tokens as follows, per million:

I am deliberately not converting euros to dollars in that table, because inventing an exchange rate to compare two published prices is exactly the kind of false precision I avoid. I do not need to: a quarter of a euro against five to fifty dollars is one to two orders of magnitude whichever way the euro-dollar rate swings, and OpenAI's frontier tier (its own pricing page) sits in the same upper band. The self-hosted marginal cost is energy plus nothing, because the weights are already on your disk and the model does not meter you.

That table also understates the local advantage, in two ways. It counts only output tokens, but the APIs bill input as well, every token of your system prompt, your tool schemas and the files you paste into context, at their own per-million rate, whereas locally that prefill is a handful of extra watt-seconds. And it ignores that prompt caching, the main lever for cutting an API bill, discounts input reads but never touches output, which is the expensive half. Add up the whole invoice on a token-heavy agentic workload and the gap widens rather than narrows.

That marginal picture is only half the ROI, though. The card is capex, roughly €450 to €500 at 2026 EU retail for a 16GB 5060 Ti, and whether it pays back is a pure utilization question. Against a mid-tier API at ten to fifteen dollars per million output tokens, a ~€500 card recovers its cost somewhere in the low tens of millions of output tokens. In practice that means a heavy agentic developer, the kind generating a few million output tokens a month across coding sessions, breaks even in months, not years; a five-person team sharing one card gets there in a couple of months and runs near-free afterwards. An under-used card, by contrast, is pure loss: the economics reward sustained volume, not occasional prompts.

The self-hosting case is a utilization-and-sovereignty case, not a "cheaper for everyone" case. On heavy, sustained, multi-seat agentic workloads, or on data that legally cannot leave the building, the card wins comfortably. For a hobbyist's occasional question, the flat-rate cloud plan is both cheaper and less work.

If cost governance for an AI-assisted dev team is the actual problem you are trying to solve, that trade-off (which workloads to keep local, which to route to an API, where the break-even really sits for your volume) is the kind of sizing I do in my hub on AI automation for companies, where the methodology matters more than the demo.

Does that beat GitHub Copilot's price, not just the token APIs?

For agentic chat it wins the same way; for plain autocomplete it does not, and pretending otherwise would be dishonest. Inline completion is where local inference shines on latency (the companion benchmark measured a dedicated 7B model completing at 22 ms warm), but latency is not the whole cost. GitHub's Copilot plans put the Pro tier at $10 a month flat, with unlimited code completions that are not metered and do not consume the credit pool, a status the June 2026 billing changes left untouched (those touched code review, not completion). Against an unlimited flat $10, a €500 card bought only to autocomplete for a single developer essentially never pays for itself.

The point is that autocomplete is the wrong thing to justify the hardware with. The card earns its keep on agentic and chat token volume, where the APIs charge those 19-to-192-times multiples over self-hosted energy, and where a heavy user burns millions of output tokens a month. Buy the GPU for the agent loop; the fact that it also serves inline completion (with the co-residency swap tax the benchmark documented) is a bonus, not the business case.

How many developers can one 16GB card serve?

One at a time, effectively, and this is the limit people underestimate. The constraint is the KV cache, the per-token memory every attention step reads, and I measured its growth directly on the card. On the 9B model at a q4_0 cache the base cost (weights, CUDA context, compute buffer) is about 8.3 GiB, and the cache then grows dead-linearly at roughly 13 KiB per token. Of the ~15.75 GiB usable on a 16 GB board, that leaves a fixed KV budget of about 7.4 GiB, or ~580,000 tokens total, and, crucially, that budget is shared across every concurrent session, because llama.cpp splits one context window across its sequence slots.

So several developers co-reside only at modest per-user context, and a single long-horizon agentic run, the kind that reaches tens of thousands of tokens, already claims a large slice of the budget on its own. The deeper limit is not even memory: the throughput-optimal serving configuration is a single sequence slot (-np 1), and the speculative-decoding speedup that makes decode fast requires it, so the fast path is single-stream by construction. Serving many users concurrently means giving up that fast path and sharing one card's fixed ~448 GB/s of bandwidth between them.

A 16 GB card is a single-active-stream workstation device. It has no headroom to give several developers a full agentic context at once, and it cannot both run the fast speculative path and fan out across users. You scale this architecture by adding cards, not slots.

That is the honest ceiling, and it reframes the ROI. The "team of five shares one card and breaks even in two months" math holds only if those five are not all running long agentic sessions at the same instant; the moment they are, you are buying a second and third GPU, and the per-developer capex climbs back toward the point where a metered API, or a mix of local and cloud, is the pragmatic answer.

So when does self-hosting a coding model make sense?

When you have sustained token volume, or data that cannot legally leave, or both, and not otherwise. The wrong question to ask of a €500 local setup is whether it matches today's frontier: it plainly does not, and I am not going to dress it up with a benchmark score, because leaderboard numbers rot in weeks and I do not trust them as evidence. The right question is whether it is seriously usable, and my own measurements answer that directly. On a real cross-file debugging task it finds every root cause with minimal, idiomatic fixes, in Python, PHP and TypeScript alike; it resists a planted red herring; it never regresses working code; and it holds perfect tool-call protocol reliability across hundreds of calls. The capability that not long ago lived only behind a frontier cloud API now runs on a card that costs less than a phone, with the data never leaving the room. That compression is the story, and it does not need a scoreboard to be true.

What the economics add is discipline. Self-hosting is not a way to make AI free; it is a way to move a variable, metered, jurisdiction-dependent cost into a fixed, owned, private one, which is a good trade precisely when your volume is high and your data is sensitive, and a bad one when your usage is light and your data is ordinary. The card, the energy bill, the single-stream ceiling and the swap tax are all real, and a serious plan accounts for them instead of waving at "local is cheaper". If you are weighing a sovereign inference tier for a development team and want the break-even worked out against your real token volume and your real compliance constraints rather than a vendor's slide, the free scoping form takes about two minutes and tells you honestly whether the numbers favour a card under the desk, an API, or the boring, correct mix of both.