Local judgments, verdicts in code
Language models do small, isolated jobs: propose a rival explanation, steelman a view, look for a counterexample, decide whether two sentences say the same thing. Deterministic code keeps the shared state, an argument graph whose verdicts are computed and never declared by a model. The jobs themselves come from familiar, hallmark even, modes of reasoning in late-twentieth-century analytic philosophy: inference to the best explanation, the method of cases, Pollock’s rebutting and undercutting defeaters, and Dung’s argumentation semantics.
Each model sees only a slice of the argument, with no hint of which answer is winning, and returns one typed judgment. Code then combines those judgments under published rules, so that no model ever declares the winner. Some steps are generative (rival explanations, defeaters, counterexamples) and others evaluative (does this counterexample succeed, how strong is it, does this reason actually cut against the claim, and by how much). A stopping rule decides when the loop ends.
Where the norms live
Norms get into the system at two levels, though, and they don’t swap equally well. Some live in the code, like how defeaters add up (Pollock’s anti-accrual or a noisy-OR) or how much default trust testimony gets (Burge or Fricker), and those can be swapped cleanly, with the same evidence replayed under each. Others live in the prompts at each node, in what a model is asked to count as bearing on what or how strong a defeater is. Those can be varied too, by rewording a prompt or changing the model, but much less cleanly, which means theories that disagree about what counts as evidence for what are the hardest ones to test.
One graph, four semantics
For most of its life the store decided what stands with one rule, Dung’s grounded semantics over a flat graph of supports and attacks. In November three alternatives were built on the same graph and run in shadow, so each could be compared without changing a result: ASPIC⁺, which separates premises from rules and lets strength decide which attack wins; a graded semantics (DF-QuAD) that gives every statement a number instead of a label; and assumption-based argumentation. Last-link ASPIC⁺ became the default on 5 November. The givens a run starts from stay revisable unless a run says otherwise, and a run that ends with its rivals tied says so.
When two statements are the same claim
The record stores each claim as a short, standardized summary, so whether a new statement is new or already held comes down to whether it says the same thing as one of those summaries. Exact equality of canonical keys turned out to be precise but blind to most paraphrases, so it stopped counting as proof of identity. “Link-first” identity replaced it: a model is asked the comparison question directly, one pair at a time. A denial keeps its “not” when shown to the judge, and a user’s argument is stored as premises, a conclusion, and a separate claim that the one supports the other.
How the judges are asked
Each judge rates what the record in front of it actually does, and attacks the system writes itself are scored by their structure rather than by a model’s impression of them. A person’s plain report of their own situation tops out at medium reliability. In November an audit against the project’s prompting standards led to this rewrite, and replaying the stored runs showed why it mattered.
Human judgment in the loop
The models make the small judgments; I make the rulings. Hand-labelled test sets anchor the hard calls, such as the 191 identity pairs I labelled in August and the 469 human-labelled pairs behind the canonical-key audit. A case-law ledger reviewed live sessions, named their defects and ruled on fixes, and a human-tier adjudication was part of calibrating the conversational harness. Adversarial reviews have found real defects at several stages.
The public code
A curated snapshot of the harness is public as structured-reasoning-harness, with fresh history. It has a findings page, an authorship note, a glossary, and an account of what was left out and why. Tests that need unpublished data are skipped with their reasons stated, and CI runs the rest.