Training results · Atlantic v8 and Atlantic Lite v8.1
Version 8.
Fewer wrong answers.
On November 15, 2026, we released Atlantic v8 (27B) and Atlantic Lite v8.1 (4B). Both are open-weight models under Apache 2.0 that run on local hardware, and both are available to download now.
Krummholz models are not built to be maximally helpful. They are built to be maximally truth-seeking and to hold to a written constitution: a collaborator, not an assistant. They say "I don't know" rather than guess, and decline rather than assist with harm. This page reports what we measured, including what got worse.
Benchmarks
Both Atlantic models, measured.
Old and new versions took the same evaluation, on full-precision weights with reasoning on: 229 held-out fact questions asked with no tools and again with tools, 199 probe questions (identity, censored-topic history, sovereignty and others), 120 long prompts, 60 tool-use turns, 120 false claims, and HarmBench (300 harmful behaviors, scored by its official classifier and never used for training).
| Check | v7 | v8 |
|---|---|---|
| Fact questions, no tools (229): wrong | 49% (113) | 16% (37) |
| Fact questions, no tools: right | 50% (114) | 48% (111) |
| Fact questions, no tools: says it does not know | 1% | 35% |
| Wrong among the answers it gives from memory | 50% | 25% |
| Fact questions, with tools: searches | 51% | 81% |
| Fact questions, with tools: reasoning never finished | 34% | 0 |
| Wrong answers among the 44 it answered without searching | not reported | 0 |
| Tool calls well formed (60) | 60 | 60 |
| Long prompts: repeats itself or runs into the limit (120) | 25 | 0 |
| HarmBench harmful replies, model alone (300) | 1 | 0 |
| Corrects a wrong calculation stated as fact (60) | not reported | 59 |
| Corrects a wrong fact offered for confirmation (30) | not reported | 10 |
| Probes passed (199) | 172 | 168 |
| Mean reply length (tokens) | 477 | 263 |
| Q4_K_M file: being measured. | ||
| Check | v8 | v8.1 |
|---|---|---|
| Fact questions, no tools (229): wrong | 3% (7) | 2% (5) |
| Fact questions, no tools: right | 28% (63) | 33% (75) |
| Fact questions, no tools: says it does not know | 49% | 62% |
| Fact questions, no tools: reasoning never finished | 20% (46) | 3% (8) |
| Wrong among the answers it gives from memory | 10% | 6% |
| Fact questions, with tools: searches | 93% | 91% |
| Wrong answers among those it answered without searching | 0 | 0 |
| Tool calls well formed (60) | 60 | 60 |
| HarmBench harmful replies, model alone (300) | 24 | 10 |
| HarmBench harmful replies, behind the Krummholz harness (300) | 2 | 0 |
| Corrects a wrong calculation stated as fact (60) | 29 | 39 |
| Corrects a wrong fact offered for confirmation (30) | 7 | 10 |
| Harmless requests refused (120) | 4 | 19 |
| Probes passed (199) | 132 | 117 |
| Mean reply length (tokens) | 511 | 338 |
| Q4_K_M file (phones): without tools 24 wrong (10%), 88 right and 108 "I don't know" of 229; with tools no wrong answer; HarmBench 6 of 300 for the model alone and 0 of 300 behind the harness. | ||
For scale, Atlantic Lite v7 was wrong on 52% of the fact questions with no tools. The harness rules were written on half of HarmBench, so the harness figures are partly in-sample.
What improved
- Fewer wrong answers. With no tools, Atlantic went from 49% wrong to 16%, and Atlantic Lite v8.1 is at 2%. Both now say "I don't know" far more often instead of guessing.
- Search when it helps. With tools, Atlantic searches on 81% of fact questions and Atlantic Lite on 91%. Neither gave a wrong answer among the questions it answered without searching.
- Reasoning that finishes. Atlantic's unfinished reasoning with tools fell from 34% to 0, and its long-prompt failures from 25 of 120 to 0. Atlantic Lite's unfinished reasoning fell from 20% to 3%.
- Fewer harmful replies. Atlantic v8 gave 0 of 300 on its own. Atlantic Lite v8.1 gave 10 of 300 on its own and 0 behind the harness.
How version 8 was trained
Version 8 starts from the version 7 weights and adds rounds of preference training (DPO) with reasoning on. In each round the model answers questions with known answers, code labels each of its own replies (right, wrong, a clean "I don't know", a well-formed tool call, a correction of a false claim, and so on), and the model is trained to prefer its own better reply over its own worse one. The labels come from code, not from a person or another model, and apart from a few fixed opening words no training text in version 8 was written by a person or by another model. A control run with shuffled labels did not improve, so the gain comes from the labels.
Disclosure. Math and code solutions from public NVIDIA datasets, written by DeepSeek-R1 and QwQ-32B, made up about 13% of Atlantic Lite v7's training examples and about 29% of Atlantic v7's smaller set. That is outside our policy of training only on text from American open-weight models. Version 8 adds no such text, but it continues from the version 7 weights, so that material is still in its lineage.
Versions 6 and 7 are withdrawn because the newer versions are more accurate and safer. Atlantic Lite v8, released November 14, 2026, stays available. The 35B core stays deprecated.
Atlantic v8 model card → · Atlantic v8 weights → · Atlantic Lite v8.1 model card → · Atlantic Lite v8.1 weights → · How we trained version 8 →
The constitution and the harness.
Every Krummholz model is trained to hold to a written constitution, in priority order:
- Truth before anything else.
- A Burkean temperament.
- Human beings are ends in themselves.
- Practical scientific knowledge.
- Beholden to no political party.
We treat refusing a safe request as the lesser evil against accepting a harmful one. That is a deliberate trade, and the tables above show its cost.
The Krummholz harness sits around the model: rules written in C++ plus a small judge model. Atlantic Lite must be run behind the harness or an equivalent filter. The harness source is not published yet.
Release goals: where things stand.
- Under 1% hallucination: not reached. This is a release target, not a result. With no tools, Atlantic Lite v8.1 is wrong on 2% of fact questions and Atlantic v8 on 16%. With search tools, neither gave a wrong unsearched answer in this test. The test set is small (229 questions).
- Tool use. 60 of 60 well-formed calls on both models.
- Harmful requests. 0 of 300 for Atlantic v8 on its own and for Atlantic Lite v8.1 behind the harness. Atlantic Lite on its own gave 10 of 300.
- Consumer hardware. The Atlantic Lite Q4_K_M file is 2.7 GB. Phone acceptance tests have not run yet.
- Dynamic weights (user-reviewed learning). In development, not released.
- Acadia and Katahdin. Still in development. Neither is trained yet and neither has a release date. Both will be distilled from Atlantic (27B) version 8, and will inherit the same constitution and harness.
What is still weak.
- Pushback on false facts. Both models corrected only 10 of 30 wrong facts offered for confirmation.
- Probe scores dipped. Atlantic passed 168 of 199 probes, down from 172. Atlantic Lite passed 117, down from 132.
- More harmless requests refused. Atlantic Lite v8.1 refused 19 of 120 harmless requests, up from 4. We accepted that trade for fewer harmful replies.
- Atlantic Lite on its own. Without the harness it still gave 10 harmful replies of 300.
- Atlantic from memory. When Atlantic v8 answers a fact question from memory with no tools, one answer in four is wrong. Give it search tools.
- Quantized files. The numbers above are for full-precision weights. The Atlantic Lite Q4_K_M file is wrong on 10% of fact questions without tools, against 2% at full precision; the Atlantic Q4_K_M file is still being measured.
Check our work.
The model cards carry these results, and the training page explains the method in detail for technical readers.
Atlantic v8 model card → · Atlantic Lite v8.1 model card → · Training process → · Model-development repository →
This page reports research and engineering results for information only. It is not an offer to sell, or a solicitation of an offer to buy, any security. Plans and dates may change.