MTP speculative decoding in llama.cpp, faster but not identical
Some recent GGUF files ship with a multi-token prediction head that llama.cpp can use as a built-in draft model. On Qwen3.8-27B it lifted decoding on code from 29.3 to 56.8 tokens per second, every single problem faster, for about 1.3 GB of extra VRAM. What the release notes do not say is that the answers change: at temperature zero only 7 of 50 were identical with and without the head, while the number of correct answers stayed the same. Speculation here is fast, not neutral. Continua a leggere
Ultima modifica:
Ternary-Bonsai-2-27B is Qwen3.8-27B with its weights reduced to three values, packed at 1.75 or 2.13 bits per weight, and it needs a fork of llama.cpp to run. On a 16GB card the result is striking on size and speed: a 27B model in six to seven gigabytes that decodes almost twice as fast as the 4-bit build. On 134 deterministic items it passed 105 and 107 against 113 for the conventional quants, a gap that this comparison cannot test, for a reason worth understanding. 
The 35B-A3B shape, 35 billion parameters with about three billion active per token, is the sweet spot of a 16GB card, and two models compete for it. At the same IQ3_XXS quantization, Qwen3.6-35B-A3B and Ornith-1.5-35B-A3B tie overall on 134 deterministic items, but not in every category: Qwen follows precise instructions in eight cases where Ornith fails, and Ornith is better at arithmetic. Ornith decodes 19% faster, and Qwen loses more ground in Italian. 
The Ornith 1.5 family offers a dense 9B and a 35B mixture of experts with about three billion active parameters, and on a 16GB card the MoE is both bigger and faster. With reasoning on it solved 26 Python problems out of 50 within five minutes against 10 for the 9B, a difference confirmed by a paired test, and it answered sooner on every probe. The 9B keeps two real advantages: half the VRAM and a context about twice as long as the MoE can load. 
The GSQ-RCO build of Qwen3.8-27B is a smaller file than the uniform IQ4_XS and comes with a speculative decoding head, so on a 16GB card it is twice as fast and holds twice the context. The question everyone asks next is whether the cheaper quantization costs intelligence. On fifty coding problems it solved 34 against 33, and this measurement cannot tell them apart: the design can only see differences larger than twenty points, and a non-difference is not an equivalence. 
These are the two models I serve every day on a 16GB card, and they are not ranked by a single number. Without reasoning the dense Qwen3.8-27B solves 34 Python problems out of 50 against 19 for the Ornith 35B mixture of experts. With reasoning on and five minutes per problem the order flips, 26 to 12, because the MoE thinks at twice the speed. On real bug fixes the 27B delivered fifteen correct fixes out of fifteen, and the MoE answers four to fourteen times sooner when both are thinking. 
Turning on reasoning in a local model is sold as a free upgrade in quality, and on a 16GB card it is neither free nor always an upgrade. On five models the median wait grew between 4.6 and 14 times, a request for one shell command went from three seconds to more than three minutes, and the coding ranking between a dense 27B and a 35B mixture of experts reversed completely. Given twenty minutes, the 27B with reasoning on reached what it solved in thirty seconds without it. 
A 16GB consumer card runs a 27B dense model or a 35B mixture of experts only when every server parameter is chosen on purpose. The defaults of llama.cpp are friendly to a first try and hostile to a stable setup: an automatic fit that rewrites what you did not pin, a context budget split across slots, a KV cache pair that silently leaves the fast kernel, and a reasoning mode under which a three-second answer takes three minutes. 
Doctrine è comodo finché il database è piccolo; su tabelle grandi le scelte di default costano. Come ottimizzare le query in Symfony 7: controllo dell'hydration, fetch join contro lazy loading, batch processing per le operazioni massive, e quando conviene scendere a SQL nativo senza buttare via la manutenibilità. Pattern da applicazioni con milioni di righe, dove un findAll innocuo saturava la memoria. 
Le query N+1 non rompono nulla in sviluppo e affondano la produzione: un gestionale con molte relazioni le genera senza accorgersene. Come individuarle con gli strumenti giusti, l'eager loading fatto bene, e i pattern Eloquent avanzati per tenere il numero di query sotto controllo anche con relazioni annidate. Esempi da gestionali reali, dove una lista innocua generava centinaia di query a ogni caricamento.