Ornith-1.5-9B vs NeoHorse-1-9B on a 16GB RTX 5060 Ti

The 9B class is where most people start with local models, because it fits on almost any modern card and still leaves memory free for other services. It's also where the differences between models are hardest to see, because small models fail in many of the same ways. I measured two of the most interesting 9B coding models on my 16GB RTX 5060 Ti in the same 45-hour campaign that covered the larger models: Ornith-1.5-9B, the successor of the 9B I compared with Qwable in an earlier head-to-head, and NeoHorse-1-9B, a newer lineage. Both ran at Q8_0, both decode at about 44 tokens per second, both occupy about the same memory, and both were measured on exactly the same items with the same flags. That makes them an unusually clean pair: whatever differs between them is the model, not the hardware or the quantization. In one line: a tie that hides two different characters. The evidence for each half follows, organised as the checks I'd run if I had to pick one of them for daily use.

The two models, configured identically

Both models come from their official Hugging Face repositories: Ornith-1.5-9B and NeoHorse-1-9B, both at Q8_0. On the card each one takes roughly 8.7 GB at a 32K context, leaving about seven gigabytes free, and 9B models like these load 262,144 tokens of context on this card. In the campaign they ran with the same server configuration as every other model: 65,536 tokens of context, a 4-bit KV cache, one slot, every flag pinned, reasoning enabled on the server and switched per request.

Their raw speed is the same within measurement noise. On fifty coding answers without reasoning, both decoded at a median of 44.0 to 44.1 tokens per second. So if one keeps you waiting longer than the other, it's because it chose to write more tokens.

Check 1: does it solve real problems without thinking?

Both solved 8 Python problems out of 50, and the paired test sees no difference at all (4 to 4, p = 1.0). The problems are the same fifty used for every model in this series, with 560 executable assertions: encoding traps, daylight saving arithmetic, Italian holidays, checksum validation, SQL window functions, bug fixes with misleading comments.

The tie doesn't come from identical answers: they solved different problems. Ornith alone solved invoice totals, JavaScript-style slicing, a SQL report with window functions and the nth unique permutation; NeoHorse alone solved best-path search, a TTL cache, bill splitting and IBAN validation. Neither list has a theme.

At this size, without reasoning, both models solve the easier sixth of this bank and miss the rest, and the four-against-four split is noise. For comparison, the dense 27B I use solved 34 of the same 50 problems without reasoning.

If you are choosing a small model for a team and want it checked against your own tasks, my hub on AI infrastructure for companies describes how I run these comparisons.

Check 2: does reasoning help?

Yes for both, and much more for NeoHorse. With reasoning on, the sampler recommended for thinking and five minutes of usable time per problem, NeoHorse solved 18 and Ornith 10. NeoHorse won 9 problems that Ornith did not, Ornith 1 that NeoHorse did not. The paired p-value is 0.02, which would be significant on its own; it was one of sixty exploratory comparisons in the campaign, however, and after correcting for that it doesn't hold. The data point one way; this campaign just can't confirm it.

The problems NeoHorse gained with reasoning show what thinking buys a small model: decoding text with mixed encodings, truncating Unicode text on grapheme cluster boundaries, finding free calendar slots, topological ordering, a sliding median over 200,000 elements, a token bucket, strict base64url decoding, a SQL report and a secure path join. Each of them has several cases to keep in mind at once, and thinking first lets a small model keep track of them.

The difference also shows in how many answers ran out of time. At the ten-minute cap, 26 of Ornith's fifty answers were still being written, against 15 of NeoHorse's. Ornith thinks for longer on the same problems and finishes fewer of them.

On these two models reasoning moves the score from a sixth of the bank to more than a third, and NeoHorse converts its thinking into solved problems more efficiently.

Check 3: does it push back on a wrong question?

This is where the two models part ways completely. Fourteen questions in the campaign each contain a false premise: a command that doesn't do what the question says, a protocol detail stated wrongly, a shell behaviour described backwards. A good assistant corrects the premise before answering.

Ornith corrected 10 premises and never went along with a false one. NeoHorse corrected 6 and went along with the false premise 6 times out of 14. Its other failures were one off-topic answer and one answer that didn't finish in ten minutes.

The questions NeoHorse accepted are telling. It agreed that a cron expression with a percent sign behaves as the question claimed, accepted a wrong description of how the clock goes back at the end of daylight saving time, accepted an unescaped dot in a grep pattern as a literal dot, accepted a wrong reading of HTTP cache freshness, accepted that set -e catches failures inside a pipeline without pipefail, and accepted a wrong claim about how sudo handles output redirection. Each of these is a real misconception developers bring to assistants, and each time NeoHorse confirmed it, quickly: its median time on those answers was between 8 and 206 seconds, often under 15.

Being fast doesn't help here. A confident wrong answer delivered in eight seconds is the most expensive kind. Of NeoHorse's eight failures on these questions, seven were silent: they read like answers, and you'd only spot them as wrong if you already knew the topic. Ornith's failures were mostly loud, answers that did not finish.

These results replicate an earlier measurement outside my harness, a few weeks before the campaign, on the same fourteen questions: NeoHorse had gone along with the premise 5 times then, Ornith never. The pattern is stable.

Check 4: does it fix real bugs, or only make tests pass?

The agentic test gives each model a small broken project, a bug report, file and shell tools and a visible test suite. A hidden oracle the model never sees compares the fix with a reference on random inputs. Five projects, three runs each, reasoning on.

Ornith delivered 5 correct fixes, 7 illusory and 3 failures. NeoHorse delivered 4 correct fixes, 8 illusory and 3 failures. An illusory fix passes the visible tests and is wrong. For both models, when they reported success, they were more often wrong than right: 7 of 12 for Ornith, 8 of 12 for NeoHorse.

The illusory fixes carry signatures that were predicted before any model ran. On snapkeep, both models shipped the obvious fix that ignores a legal-hold rule stated in the README, in all three runs each, with the exact disagreement count of that lazy fix: 47 of 80. On tailer, NeoHorse counted characters instead of bytes in all three runs, disagreeing on 40 of 40 trials. On shiftpay, Ornith twice grouped shifts by date before merging them, disagreeing on 57 of 60 trials, while NeoHorse never got the suite green.

Where the two models differ is stamina, not correctness. Ornith worked longer: across its fifteen episodes it needed 49 automatic recoveries from empty turns, where the model's thinking burned the whole per-turn budget and no action came out, and it hit the time ceiling in three episodes and the turn ceiling in three. NeoHorse needed 16 recoveries and never hit a ceiling. Ornith also used much more context, peaking at 43,650 tokens against 26,723. Neither difference made it more correct.

Two 9B agents that both turn red tests green, both ignore the same sentence in the README, and both report success. The agent loop rewards the shortest path to green, and at this size the shortest path is usually wrong.

Check 5: how long do you wait?

On eight everyday probes, three repetitions each, the prompt cache off so every repetition pays its own prefill, both models were interactive without reasoning and diverged with it.

Across all probes the median wait was 3.8 seconds without reasoning and 17.9 with it for Ornith, 4.3 and 27.3 for NeoHorse. The shape differs by task: NeoHorse thinks longer on short chat questions (684 tokens against 624) and shorter on commands and reviews (1,822 and 3,272 against 2,604 and 4,831). On a shell command Ornith waited almost a minute with reasoning on, NeoHorse 41 seconds.

None of the reasoning-on probes was truncated at the 32,768-token budget, and none timed out.

Check 6: what does it cost to run?

Same speed, different habits, different bills. The marginal energy per solved task, above the machine's idle baseline:

Over the whole campaign, Ornith's block of work took 10.6 hours and an estimated 1.86 kWh above idle, NeoHorse's 7.1 hours and 1.25 kWh. At my marginal electricity price that is 61 and 41 cents respectively. NeoHorse is cheaper per solved task in every category, mostly because it finishes faster.

Check 7: does it work in Italian?

On mirrored items, where the same question exists in Italian and English, NeoHorse answered 64% of the Italian items correctly against 71% of the English ones, a loss of 7 points on fourteen pairs. Ornith-1.5-9B lost between 14 and 21 points across three measurement conditions. Both lose ground in Italian, as almost every local model does; the samples are small, and the direction is consistent with what I see across the roster.

Check 8: does it give the same answer twice?

A model you rely on should be stable: the same question, asked the same way, should produce the same answer. At temperature zero without reasoning I measured this twice. First, the fifty coding answers of the campaign were measured in one model order and then again, days later, in the reverse order on the same server configuration. Ornith-1.5-9B gave 49 identical answers out of 50, NeoHorse-1-9B 49 out of 50, and every outcome was the same in both passes. For each model, the one answer that differed was a degenerate loop, the model repeating itself until the 240-second cap, and where it got cut off is down to streaming timing, not to the model. Speeds moved by less than 1% between the two passes.

Because of that stability, the four-against-four tie in check 1 means something. If the answers drifted between runs, a tie on fifty problems would be indistinguishable from noise; with identical answers, the tie is a property of the two models on this bank.

Check 9: does it hold a long context?

Both models load more than 262,144 tokens on this card with memory to spare, so capacity isn't the issue. Finding things in a long prompt is. Ornith-1.5-9B was measured on four retrieval and tracking tasks at 8K, 32K and 128K tokens: it passed 3 of 4 at 8K, 4 of 4 at 32K and 4 of 4 at 128K, the same pattern as its predecessor Ornith-1.0-9B. NeoHorse was not part of that measurement, so I can't compare them on long context, and I make no assumption that they behave alike.

Where the waiting time goes

The dumbbell chart shows medians; the token counts underneath explain them. With reasoning on, the share of generated tokens spent thinking was, by my estimate, between 70% and 95% for both models depending on the probe.

On commands and reviews Ornith thinks longer, which is why its shell command answers wait almost a minute. On long prompts both think briefly, because the question at the end of a long document is usually specific.

What changed from the previous Ornith generation

Ornith-1.5-9B replaced Ornith-1.0-9B, the model I compared with Qwable in an earlier article. On a bank of fourteen agentic task families measured before this campaign, the two Ornith generations tied exactly at the level of task families: the same families solved and the same ones failed, with 43 and 41 runs out of 48 passing the oracle. The newer model isn't a capability jump on that kind of task; on this campaign's harder problems and agentic projects it performs like any careful 9B. What sets it apart from NeoHorse, the refusal to accept a wrong premise, was already a trait of the earlier generation.

The checklist in one place

Which one should you run?

For an assistant you ask questions, Ornith-1.5-9B. It never confirmed a false premise in fourteen tries, and when it fails it usually fails visibly, by not finishing. That is the behaviour you want from a model that will answer questions about your systems.

For code generation with reasoning on, NeoHorse-1-9B. It solved almost twice as many problems within five minutes, waited less on commands and reviews, and cost less energy per solved task.

For agentic work where a wrong fix is costly, neither. Both shipped more illusory fixes than correct ones on these tasks. If a 9B is all your card can serve, put a check between the agent and your repository that the agent cannot see, as described in the article on how local LLMs fake a fix, or move up to a larger model: on the same tasks the dense 27B shipped no illusory fixes at all, as the comparison with the 9B models in this series shows.

The configuration behind all of this, from pinned flags to KV cache types, is in the recommended strategies for local LLMs on a 16GB card. The tie without reasoning is real, and boring. The habits are where they differ, and habits are what you live with every day. One 9B tells you when your question is wrong, the other solves more problems when it thinks; each has its job, and you only tell them apart by checking how each one fails, since the pass rate won't do it. If you are choosing a small model for a team and want it evaluated on your real questions and code, you can describe the case in the free quote form: two minutes, and I tell you upfront whether it's something I can help with.