Matt Priselac

Research · The harness

The harness: local judgments, verdicts in code

So far: a plain prompt beat my epistemic harness on held-out arguments, and the stronger the model, the more a hand-coded harness got in its way. The harness’s clearest win is observation over authority, at 97%.

Spending time working with AI agents every day for work, it’s impossible to miss how capable they are. And yet it’s also impossible to miss that there’s something distinct in the patterns of their reasoning, in where they excel and where they’re weak. Given the way the models are trained and then post-trained, it’s not surprising that they operate in a distinctly associationist manner. That’s no knock on these models, or on associative models of thought in general. But it does raise a question about how well these models can reason, especially as the reasoning ascends to levels of abstraction or generality beyond their training and post-training material. Structure in thought and judgment may be what explains abstract reasoning about concepts, judgments or thoughts themselves, and it’s exactly what these models may well be missing, given their associationist foundations.

4 Eric Mandelbaum, “Associationist Theories of Thought”, Stanford Encyclopedia of Philosophy.

The goal of this line of inquiry is to explore exactly this issue. As a first pass, the project evaluated how well models reason by comparing the direct output of models to a harness that keeps model judgments local and then pulls together the results of those local judgments with deterministic code. The idea is that the local judgment of models is good, but the weighing of reasons by an associationist system may be weak. Each model sees only a slice of the argument, with no hint of which answer is winning, and returns one typed judgment; code then combines those judgments under published rules, so that no model ever declares the winner. That project set the basic line of research, which has since taken several twists and turns.

The laboratory: a two-sided coin

What falls out of this project is a two-sided view of these language models: as subjects of investigation, and as measurement tools for doing philosophy. As subjects of investigation, we can set up a ladder of evaluation to see how well models do or do not express different epistemic norms. Are the norms innately in them? Are the models able to follow them as instructions? Or do they need a harness to leverage their local judgments into better reasoning? And of course, once a harness is available, can it serve as a bootstrap for creating a model that can either follow the norms as instructions or internalize them?

As measurement tools, these lines of investigation can be used in several directions for philosophical investigation. They can help turn claims about epistemic norms into predictions and draw contrasts between theories on cases. For those who want to reason from claims about folk epistemology, or about folk intuitive judgments on cases, we can see how well the best models, which best capture such folk data from their training inputs, align with some philosophical theories rather than others.

How the harness does its work, from the division of labour between models and code to where the norms live, is on the Method page.

How it moved

1. A harness for one question (June to July). As an initial plan, I thought we could take those hallmark modes of reasoning and try to represent them in a harness, where each node is a particular operation in which a model makes a judgment, and those judgments are then integrated and cycled back on each other. The harness wasn’t conversational. A question comes in, is parsed into its assumptions and a core question, and then the harness loops until a stopping rule fires. Some stages are generative, coming up with rival explanations, defeaters and counterexamples, and others are evaluative: is this counterexample successful, how strong is it, does the putative reason actually cut against the claim, and by how much? More norms were then modeled for summing those judgments up. While the results were interesting and the outputs compelling, we quickly started to see the limits that the project is grappling with to this day. When you have generators coming up with dozens of counterexamples or defeaters or supporters, and you are trying to integrate them all through the harness, you need to be able to tell whether something is the same reason as something else. Another issue that arises quickly is that how the harness reasons through a question depends strongly on what the model at each node brings to the table. Worse, and this is really a theme, there was no good way to distinguish information flowing through the harness from what the system was actually committed to, which meant there was no principled way to balance how an input ought to shape the current judgment against prior commitments, because there was no real sense of a commitment in the system at all.

2. A conversational layer (July to August). Still, we had some interesting foundations, and a natural way to extend a fundamentally epistemological project was with a conversational arm. Adding that layer brings its own complexity, because giving every incoming statement the full twentieth-century analytic epistemology level of scrutiny is, on its face, absurd. So the goal was to take theoretical understanding of conversational context and speech acts from philosophy of language and use it as a second harness, one that keeps a structured record of the conversation: each party’s commitments and obligations, the live question, the context. As a conversation unfolds the system now has commitments to use as priors, takes information in and revises (or doesn’t) those commitments, and uses them as a basis for replies, while the full analytic treatment is only convened when the conversational harness decides the context demands it. Again, though, we quickly ran into issues around handling a record that is fundamentally about propositions that can be expressed in endless linguistic forms. When updating the record, how can we tell whether some claim is new or the same one as before? And as the context grows, which parts of the commitment record does a new input actually bear on, and to what degree? We tried various information retrieval structures as well as simply having a fast decision model make those calls, and each helped in some ways, but each also failed in ways that snowball quickly. The record stores each claim as a short, standardized summary, so the identity question is really whether a new statement says the same thing as one of those stored summaries. After a lot of trial and error, the best approach we found was to ask a model that question directly, one pair at a time, and on a hand-labelled test set it worked well. One model correctly matched 67 of 106 pairs that really were the same claim while wrongly linking only one of 363 pairs that weren’t, and a second model did about as well. But there was a catch in the test itself. Almost every new statement in it (467 of 469 pairs) had already been tidied into a summary before the comparison was made, so what we had shown was that two clean summaries can be matched reliably. What a person actually types in a conversation is messier: it leans on context, uses “that” and “it” for things said earlier, and words things loosely. Whether matching still works when one side is that kind of everyday language is the next thing to test, and it needs real conversations with human-checked labels rather than another synthetic set.

3. A benchmark that said no. The first real sign that something was wrong came in August, when I ran the epistemic harness against a plain prompt on Redwood Research’s argument-critique ratings (LMCA), and the plain prompt won on held-out arguments. Later work in October sharpened the point: the stronger the model, the more a hand-coded harness got in its way. Decomposing commitments, claims and reasons into separately stored propositions and then trying to reassemble them through code began to feel like an ultimate limiting block. Imposing structure in that way seemed to lose too much of the conceptual relationships already built into the models, and not buy enough epistemic performance back. So the next phase aimed to do better at leveraging the models’ ability to make conceptual connections at a broad scale, and to make the enforcement of epistemic norms less rigid.

From there the work branched. The store of commitments moved outside the model in the commitment store, then back inside the model’s context in Strength, while the plain-prompt challenge kept testing the benchmark that said no. The harness itself kept going: in October a curated copy went public, and in November it gained three new ways of deciding what stands.