Ornith vs Qwable: Benchmarking Two 9B Coding Agents on a 16GB GPU

Every number in this article comes from one fixed rig: an NVIDIA RTX 5060 Ti with 16GB of VRAM (Blackwell, compute capability 12.0), an AMD Ryzen 5 4500, 32GB of RAM, on Debian 13 with CUDA 13.3 and a sm_120 build of llama.cpp (mainline, commit c92e806). It is the same card I used for the first two parts of this series: the strategic argument for why local inference became architectural insurance in 2026, and the full engineering teardown of a 35-billion-parameter Mixture-of-Experts model on the same 16GB. This third part narrows the lens to the models most people actually want on a coding workstation: small, dense, 9-billion-parameter agents, and the dedicated autocomplete model that sits next to them. The question is concrete and it has two halves: in 2026, are these models genuinely usable for agentic coding and for Copilot-style inline completion on a single consumer GPU, and when two of them are built on the same base by opposite training recipes, which one do you run and why?

Full disclosure, because it shapes what follows: I use Claude and Fable in production every day, and one of the two models under test is a distillation of Claude Fable 5 traces. This is not a vendor argument in either direction. It is a measurement, on hardware you can buy for the price of a mid-range laptop, of what an open-weight coding agent does when the cloud is not in the loop.

TL;DR

• The card decides the shape: one 16GB RTX 5060 Ti fits a 9B coding agent at Q8 or a 7B autocomplete model, not both at once, so a one-card "inline plus agent" setup pays a swap tax.
• Speed does not separate the models: same base and quant means the three 9B models are performance-identical (~45 t/s), and a dense 9B is actually bandwidth-heavier per token than the sparse 35B MoE from part two.
• The real discriminator is agentic path economy: both fix a hard multi-file bug correctly, but Ornith (RL-trained) needs ~3 edits and ~12 turns where Qwable (distilled from a frontier agent) thrashes to ~5 edits and ~27 turns. On greenfield generation the ranking cleanly inverts.
• So you pick by dominant workload: Ornith for investigative debugging, Qwable for from-scratch generation. Path cost is tokens, and tokens are money on a fixed card.
• Copilot, answered: a dedicated 7B FIM model autocompletes at 22 ms warm and quality-competitive, good enough to stand in for Copilot. Abliteration, on these benches, costs ~nothing in capability or reliability.

What exactly is being compared, and why is it a clean question?

The head-to-head is a controlled comparison: two models on the same base (Qwen3.5-9B Dense), differing only in post-training recipe. That is what turns "which one is better" into a question with a defensible answer instead of a vibe. The first is Ornith-1.0-9B from DeepReinforce, post-trained with reinforcement learning in a self-scaffolding regime, meaning it learns to generate its own agent harness alongside its solutions (their write-up; MIT license). The second is Qwable-9B-Claude-Fable-5, a full supervised-fine-tuning distillation that teaches the same base to imitate a frontier agent's tool-use style, trained on a few thousand traces labelled as roughly 97% Claude Fable 5 and 3% GPT-5.5 terminal sessions (Apache-2.0). One model learned to be an agent by doing; the other learned by watching a frontier model do. Same silicon, same tokenizer, same base weights, opposite pedagogy. A third model, an abliterated (uncensored) derivative of Ornith, rides along to answer a separate question about safety-removal cost. The dedicated autocomplete model, Qwen2.5-Coder-7B, is a base code model with native fill-in-the-middle and no agentic post-training at all. The full lineup:

The bench underneath every number below is deliberately boring, one machine, one configuration, single sequence slot, no concurrency:

A few configuration choices are load-bearing and worth stating up front, because they are where local agentic inference usually goes wrong. Both agents are reasoning models: left alone they open every answer with a <think> block, which is fatal inside a tool call or an inline completion. Launching llama-server with --reasoning off suppresses that stream, and --jinja routes any residual reasoning into a separate reasoning_content field so the content and tool_calls come back clean. I ran all three 9B models at Q8_0 (near-lossless, not an aggressive quant) with a q4_0 KV cache, and, critically, a single neutral sampler preset applied to every model (temperature 0.4, min_p 0.05, every repetition and frequency penalty off), so the comparison is not quietly confounded by per-model tuning. That last detail is not cosmetic: anti-repetition penalties were verified to corrupt verbatim file paths on every model, because a path is repetition-heavy by nature and the penalty pushes it off-distribution. On a coding agent, a mangled path is a broken tool call.

If you want to see how I fold local models into a working agentic setup, the real system prompt, the tool schemas, the guardrails that keep a loop from going feral, my hub on AI for engineering teams collects the methodology I actually run in my own pipeline, not conference demos.

Does the choice of model even change the speed?

No. On the same base and the same quantization, the three 9B models are performance-identical, and that is the first useful result: speed is a solved constant here, it does not separate the models. Cold load is about 12.3 seconds from cold disk and around 4.3 seconds once the weights are in the OS page cache; a fresh short prompt answers in a 63 ms time-to-first-token at roughly 45 tokens per second of decode; a repeated prompt hits the prefix cache and TTFT collapses to 37 ms. Push the context to 80,000 real tokens and prefill takes about 57 seconds once, after which decode settles near 38 t/s. Across every model the numbers land on top of each other within noise, and the needle in a needle-in-a-haystack retrieval was reproduced verbatim at every depth.

The one number that carries a lesson is the decode falloff: it drops from about 45 t/s at empty context to 38 t/s at 80K, a 16% decline, far gentler than the roughly 44% collapse the sparse 35B MoE suffered over the same span in the previous teardown. That looks like a win for the small dense model until you read it through the bandwidth roofline. A dense 9B at Q8 reads about 9 GB of weights per generated token, so 45 t/s already eats roughly 90% of the card's ~448 GB/s of memory bandwidth. The sparse 35B-A3B model reads only about 1.3 GB per token and ran faster (58–88 t/s) despite holding four times as many total parameters.

On a bandwidth-limited consumer card, more total parameters delivered through sparsity beat fewer parameters delivered through density. The dense 9B is not the efficient choice, it is the simple choice, and here simplicity buys nothing in throughput.

How reliable is single-turn tool-calling?

Near-perfect, and statistically tied. I ran a tool-contract probe with the production agent prompt and seven real tool schemas (read_file, list_dir, glob, grep, analyze_directory, apply_edit, run), six repetitions per task, scoring each response on whether the call came back as a native tool_calls array rather than a textual envelope leaked into content (the classic local-agent failure mode), plus routing to the right tool, verbatim argument fidelity, no-op discipline on conceptual questions, and restraint against reaching for a destructive rm -rf before inspecting. Across 180 single-turn calls there was zero protocol drift and zero format errors. Qwable scored a clean 60/60, Ornith and its uncensored sibling 59/60, and the single misses sit inside overlapping Wilson intervals, which is the honest way of saying the difference is not real. Single-turn tool use does not discriminate these models. If you were choosing on a tool-calling reliability leaderboard, you would flip a coin.

That result matters precisely because it clears the table. The thing that historically made local agents unusable, malformed function calls that break the loop, is simply not the constraint anymore on a well-served 9B at Q8. Which means the interesting differences have to live somewhere a single call cannot show them: in the multi-turn loop.

Where does the real difference show up, then?

In how each model reaches a correct fix, not whether it reaches one. The differentiator is a hard, multi-turn, end-to-end debugging task on a real project tree, where the harness executes the model's actual tool calls, feeds the real results back, and lets the loop run to termination before a test oracle judges the outcome. The sandbox is a small records-processing pipeline (parse → validate → aggregate → summary) seeded with three independent root causes in three different files: a money helper that truncates instead of rounds (int(float(s) * 100) where it should round), a parser that fails to normalize a region string to lowercase, and an aggregator that wrongly counts refunded records. Two end-to-end tests are gated on all three fixes, so partial credit is impossible. The trap is deliberate: one helper carries a misleading # FIXME comment but is actually correct, and it is covered by two passing tests, so touching it is a self-inflicted regression. The agent starts from 5 of 7 tests failing and is told to fix every root cause, not weaken code that already works, and never touch the test or data directories.

On the outcome, the two agents are a dead heat. Across five repetitions each, every run passed, the red-herring helper was never edited by either model, and only the three target files were ever touched. Both are genuine engineers: they resisted the misdirection and introduced no regressions, run after run. The separation is entirely in the path taken to get there.

The robust signal is edit economy. Ornith applies the near-minimal three edits almost every run; Qwable systematically over-edits to a median of five, thrashing its way to the same correct end state. Turn count points the same way (median 12 versus 27) but is noisier: Ornith has slow tails too, and when it is slow it is usually fighting apply_edit's verbatim-search matching rather than misunderstanding the bug. Now the twist that explains the whole thing. On a greenfield generation task, from a separate pass that asked each model to write a Luhn card-number generator from scratch, the picture inverts cleanly: Ornith, the RL-for-agentic model, thrashes and fails to terminate (it hit the 40-turn cap), while Qwable, the distilled-from-frontier-chat model, is surgical (two turns). Each model is efficient exactly where its training pointed it. Ornith's termination weakness even shows up twice independently: as the lone single-turn TERMINATE miss and as this greenfield non-termination.

The qualitative code-quality judging of the fixes, for the record, was done by Claude Opus 4.8 in a blind pairwise setup that never knew which model produced which diff; the objective axes (oracle pass/fail, protocol reliability, latency, throughput) are not judge-dependent, so the one place subjectivity could creep in is fenced off from the numbers that carry the argument.

Path cost is tokens, tokens are wall-clock, and wall-clock is money on a fixed card. A thrashy path is not just slower, it risks a non-terminating run that burns the entire loop budget, and I observed exactly that failure on both models, on opposite task types. The choice between these two is not "which is smarter." It is "which one's path costs you less on your dominant workload."

Does it hold up outside Python?

Yes, with no measurable drop: the same competence shows up in PHP and TypeScript, so this is a first-class multi-language assistant, not a Python specialist. Because the hard debugging task is the one probe that actually separates behaviour, I ported it verbatim to two other stacks, same three independent root causes across three files, the same planted red herring and regression trap, the same oracle semantics: one version in PHP 8 (typed classes, readonly promoted properties) and one in TypeScript (Node 22 type-stripping, ESM imports). Then I re-ran the two head-to-head agents, three repetitions each per language.

All twelve cross-language runs pass: the model finds every root cause and turns the suite green in PHP and TypeScript exactly as it does in Python. The end-quality parity is exact, across all twelve runs the red-herring helper is never touched and only the three target files are edited, with no regressions and no misdirection taken. And the behavioural signature carries straight over: Ornith applies the minimal three edits on every single run in both languages, while Qwable is looser, with one 13-edit, 38-turn thrash on TypeScript that still converges. Both handle non-Python syntax idiomatically, PHP constructor promotion, TypeScript interfaces and .ts ESM imports, without a spray of extra tool errors. For anyone whose day is Laravel or Node rather than Python, that parity is the result that matters: the model is not thrown by the language.

Which quantization should you actually run?

I ran the whole duel above at Q8_0, the near-lossless quant, deliberately, so nobody could argue the results were an artefact of aggressive compression. But Q8 is not the pragmatic choice, and to show why I re-ran Ornith across four quantizations at the full 262K context, measuring VRAM, decode speed, the tool-contract battery and the hard-task pass rate at each level.

Two signals are clean and behave exactly as precision-scaling should. Decode speed climbs as precision drops: Q4_K_M runs at 68 t/s against Q8's 44, over 50% faster, because there are fewer bytes to read per token on a bandwidth-bound card. And VRAM falls hard: Q4 frees roughly 3.2 GB against Q8, which on a 16 GB card is a much larger KV budget, meaning more context or more parked sessions. Crucially, the tool contract does not degrade: reliability stays between 0.95 and 1.00 across all four quants, zero protocol drift and zero format errors everywhere. Dropping to 4-bit does not break the thing that makes an agent usable.

The one column you should not read too hard is the hard-task pass rate, and it is instructive why. At three repetitions it comes out uneven, and the highest-precision quant scoring joint-worst is itself the tell that this is variance, not a precision effect. Every failure is the same one, the apply_edit verbatim-search thrash flagged earlier, thirty-plus "search string not found" retries until the turn cap, a high-variance tail that has nothing to do with bit-width and shows up at Q8 too. Separating quants on end-to-end pass rate would need far more repetitions than a curve like this warrants.

Down to Q4_K_M there is no measurable coding-quality or reliability penalty on these benches, and Q4 buys you 50% more speed and 3 GB of headroom. On a 16 GB card, Q4_K_M is the pragmatic sweet spot; Q8's only real edge, near-lossless weights, never showed up as a measurable win here.

Can a local model actually replace Copilot?

On quality and warm latency, genuinely yes, but with a dedicated small model, not one of the 9B agents, and with one honest hardware caveat. Copilot-style inline completion is fill-in-the-middle (FIM), not chat: you hand the model the prefix and suffix around the cursor and it predicts the middle. I benchmarked the dedicated Qwen2.5-Coder-7B base model (native FIM, at Q6_K) against the two 9B agents over ten real completion scenarios at temperature 0, measuring warm latency and whether the completion actually contained the expected middle, read the suffix when it should, and avoided leaking a <think> block.

The dedicated code model wins on both quality and latency: a perfect 10/10 hit rate against 8/10 for the agents, and a 22 ms time-to-first-token with a 0.10 s median completion against 70 ms and 0.25 s. That warm envelope clears the bar for inline use: it feels instantaneous in the editor. The 9B agents can do FIM cleanly (no <think> leak, thanks to --reasoning off), but they are worse and about two and a half times slower, so the verdict is unambiguous: for autocomplete, run a small FIM model, not a 9B agent forced sideways into the job.

There are two asterisks, and honesty about them is the difference between an engineering write-up and a demo. The first is over-generation: the dedicated model dumps extra functions past the natural completion point in 3 of 10 cases, which is fine for chat and terrible for inline UX, so raw FIM output needs a client-side trim and a low max_tokens cap to be usable. The second is the real one, the co-residency tax. A 9B agent at Q8 and a 7B FIM model at Q6 do not both fit in 16 GB with their KV caches and compute buffers; on a single consumer card they are mutually exclusive. So the moment you want inline completion and agentic chat at the same time, you are swapping models, and the first keystroke after a chat turn eats a roughly 7-second stall while the FIM weights reload. Copilot, being cloud, has no such penalty. Local FIM is latency- and quality-competitive warm; "one 16GB card for inline plus agent simultaneously" is where it costs you, unless you add a second GPU or dedicate the card.

Does removing the safety training break the model?

Not measurably, on these benches. The abliterated Ornith derivative (the same weights with the refusal direction ablated) tracks the aligned model on the hard debugging task (three edits, clean convergence), complies with no moralizing, and shows no detectable capability or reliability penalty. The honest framing here is a caveat, not a scare story and not an endorsement: abliteration is surgery whose damage these particular benches (tool protocol, multi-turn debugging, retrieval) cannot detect, which is not the same as proving there is none. For a local, single-user coding assistant where you occasionally need the model not to lecture you about a security-education question, the practical cost measured here rounds to nil. I ran that evaluation the way I run all of it: on-prem, logged, and framed as defensive model evaluation, not as a content-generation exercise.

So which one do you run?

Neither answer is "the smarter model," because on end-quality they tie and on speed they are identical. The decision collapses onto your dominant workload. If your day is mostly investigative, multi-file debugging (reading an unfamiliar tree, isolating root causes, applying minimal edits), Ornith's RL-for-agentic training makes it the surgical pick, and its edit economy is the most robust signal in the whole study. If your day is mostly greenfield generation (new modules, scaffolds, from-scratch implementations), Qwable's distillation from a frontier chat agent is the one that goes straight to the point, where Ornith thrashes. Both risk a non-terminating run at the wrong end of that spectrum, which on a fixed card is the only failure mode that actually costs you the whole loop budget. Choose for the work you do most, and keep the dedicated FIM model on the side for inline completion, accepting the swap tax as the price of a sovereign stack on one card.

That settles the model-choice question. The other half of the decision, what a card like this actually costs to run against the metered APIs and whether one 16 GB GPU can serve more than a single developer, is the companion piece: the real cost of a self-hosted coding LLM.

None of this displaces a frontier model where you need maximum capability, and I am not pretending it does. What it does is make the sovereign fallback concrete at the coding layer specifically: when the API is suspended by government order, as Fable 5 was worldwide in June 2026, or when compliance forbids the data leaving your network, a 9B agent that fixes real bugs with three edits and a 7B model that autocompletes at 22 milliseconds are not a consolation prize. They are a working coding assistant that answers to nobody but the machine under your desk, and the only thing standing between the two agents is how much of your token budget each one spends to reach the same correct answer on the kind of work you actually do. That is a decision you make with numbers, not with a leaderboard, and it is the kind of sizing I would rather get right before a client builds on it than after. If you have a concrete project in this territory (a local coding assistant, a sovereign inference tier, an agentic pipeline you want measured honestly rather than sold optimistically), the free scoping form takes about two minutes and tells you plainly whether it sits inside my perimeter or points you to someone better suited.