The Real Cost of a Self-Hosted Coding LLM: Energy, ROI and Concurrency on a 16GB GPU
The companion to the Ornith-vs-Qwable benchmark turns from which model to what it costs. I measured the power a 9B coding model draws on a 16GB GPU, then set the energy bill against Claude's and Copilot's 2026 list prices: the marginal cost per million tokens is one to two orders of magnitude below any API tier. But the card only pays off on real token volume, autocomplete alone loses to Copilot's flat plan, and 16GB is a single-stream device: you scale by adding cards, not developers. Continua a leggere
Ultima modifica:
Running a serious LLM on your own hardware is no longer a lab exercise. I put a 16GB consumer GPU through a 35-billion-parameter Mixture-of-Experts model with 262,000 tokens of context, and the agentic tool-calling came out 100% reliable. This is the strategic half of the story: why local inference turned from a hobby into architectural insurance in 2026, after a frontier model was suspended worldwide by government order. The hard numbers live in the companion deep-dive. 
Mistral Large 3 MoE (2 dicembre 2025) è il primo open-weight frontier-class deployabile on-prem in Europa - 41B attivi / 675B totali, Apache 2.0, addestrato su 3000 H200 francesi. Confronto con Claude Sonnet 4.6 via API: accuracy, latenza P95, costi totali per 1M chiamate, compliance GDPR. Include configurazione Scaleway H100 SXM ($2,73/hr) vs managed Bedrock. 
Gemini 3.1 Pro integra Computer Use nativo (niente modello separato) con 1M context standard. Claude Computer Use è stabile ma richiede Sonnet 4.6/Opus 4.7 dedicati. Ho benchmarkato entrambi su OSWorld-V e su tre workflow reali (SAP login, estrazione dati gestionale, onboarding cliente) nella mia sandbox. Tabella pricing, latenza P95, accuracy per tipo di task, e considerazioni data sovereignty per aziende europee.