Matt Priselac

Research · The commitment store

The commitment store: what the system holds, kept outside the model

So far: one mid-size model placed 91.8% of new statements where they belong in the store, but reading the strength of an input back out of its judgments failed.

Description

The guiding idea here was that we needed to stop trying to discern propositional identity and relevance one proposition at a time, and instead use the power of the models to work in bigger chunks, themes and arguments rather than sentences. To do that, the DoxasticLayer starts from an explicit commitment store, a list of what the system should treat as its priors (the system’s, not the model’s anymore). Code groups that store into families of related commitments, which we called “towers”, and the model is then asked how a given input bears on each tower. This did help alleviate the difficulties of the more granular approach, since questions about the identity of propositions largely fell away, and moving the store outside the model gave us a clear, explicit and inspectable account of the priors belonging to the system. Nevertheless, we ran into a similar problem when it came to reading strength and degree of relevance back out of those judgments; even once we had verdicts on how an input bears on a tower, we had the same trouble judging the bearing and strength of those verdicts against the store in order to update it. A detour into the models’ own probabilities said why. Models read the direction of evidence reliably and its strength badly, and credences are in any case relative to priors, which a model’s weights are not.

The work is paused. It began in September 2026 and its latest entry is from 27 October.

Method

A card of commitments. The store starts from an explicit card of what the system treats as its priors. Incoming text can occasion revising a commitment but never overwrite it, updating is kept separate from responding, and model judgment is channelled into typed, auditable tasks.

Towers. Code groups the store into families of related commitments, called towers, and a model is asked how a given input bears on each tower. The step that decides what an input bears on went through staged versions, ending in a hybrid: a model reads, a trained classifier decides yes-or-no questions, and code assembles the structure.

Rulings and replay. It was built as a supervised build, with workers on separate parts and a growing book of my binding rulings, and a hand-written toy engine kept byte-for-byte as a replay target.

A frozen test. The judge that decides where a new statement attaches was tuned on a development battery, then tested on a held-out battery under a pre-registration frozen first: three new cases in new domains and 216 statements, with keys a blind audit agreed with on 92% of statements before adjudication.

Diary

Two entries so far. The reading of what belief needs that came out of this work is in Locating belief.

The commitment store

September – October 2026
Dev diary
September 14–21

The DoxasticLayer, rebuilt from a toy

After a two-week pause I opened a third branch. The idea was to stop trying to judge identity and relevance one sentence at a time, and instead to start from an explicit store of the system’s commitments and judge each input against families of them. I started from a toy engine I’d written by hand (“game night” and “export cause” scenarios) and kept it byte-for-byte as a replay target. The v2 design charter laid out a card of commitments that incoming text can occasion revising but never overwrite, update kept separate from response, and model judgment channelled into typed, auditable tasks.

September 14 alone had 87 commits. I ran it as a supervised build: worker “chips” for substrate, spine, transaction and fixtures, each with its own log, and a growing book of binding rulings from me (R0 to R61). By the end of the day the log reads STOPPING POINT: one full turn clean. On September 15–16 my own adversarial review reversed one of my rulings. The “tower judge”, the model step that decides what an input bears on, went through staged versions v1 to v6.1, ending in a hybrid in which an LLM reads, a trained classifier decides yes/no questions, and code assembles the structure.

Experiment log
October 25–27

Where a new statement attaches

Back in the DoxasticLayer, the question was the one the tower judge was built for: when a new statement arrives, where does it connect to the store? I tried prompts and models on a development battery, then froze a pre-registration before a held-out run. That battery had three new cases in new domains (a fish kill below a dairy, a fall in a school’s maths marks, and who wrote an anonymous 1887 letter to a newspaper) and 216 statements, with keys that a blind audit agreed with on 92% of statements before adjudication.

A cascade of two small models, calling a mid-size model only when they disagreed, got 90.3% of the clean-core statements right, inside the predicted 85 to 94.25%. Two of the four predictions failed, and they’re reported as failures. The more useful finding came after. The mid-size model alone got 91.8%, so the cascade’s lead in development didn’t survive: the small models’ development figures had been optimistic, because the prompts were chosen with their results in view. That made the case for one mid-size model as the default judge, with the cascade kept as the cheaper option. The four statements every model missed all bear on the question through a step nobody says out loud: an analogy, a motive, or a doubt about an argument’s step.

Findings

Where a new statement attaches. A cascade of two small models, calling a mid-size model only when they disagreed, got 90.3% of the clean-core statements right, inside its predicted 85 to 94.25%. Two of its four predictions failed, and they are reported as failures. The mid-size model alone got 91.8%, so the cascade’s lead in development didn’t survive: its prompts had been chosen with the small models’ development results in view. That made one mid-size model the default judge. The four statements every model missed all bear on the question through a step nobody says out loud: an analogy, a motive, or a doubt about an argument’s step.

Strength doesn’t read back out. Moving the store outside the model made questions about the identity of propositions largely fall away, and gave a clear, inspectable account of the system’s priors. But even with verdicts on how an input bears on a tower, judging the strength of those verdicts against the store, in order to update it, ran into the same trouble as before. Locating belief says why: models read the direction of evidence reliably and its strength badly.