Nothing has beaten the plain prompt
Three weekly budgets and 31 tries across four models in, nothing has beaten the plain prompt on any of them. Judging all of an argument’s critiques together, reconstructing the argument first, adding a defender’s reply, comparing critiques in pairs, checking each critique’s key claim, and the dataset’s own rubric all recovered the same single judgment, with the arms correlating around 0.8 with each other.
The model moves the number
What changed the result was the model. Correlation with the human ratings went from 0.41 for an open-weight 27B model to 0.44 and 0.49 for two small models and 0.67 for a mid-size frontier model.
Small models fall for length
The small models fall for long, confident, technical-sounding objections: their scores track a critique’s length at about 0.4, where the human raters’ track it at almost zero.
Thinking did more than structure
Turning on a model’s thinking did more than any structure, and undid much of the trap where a confident “that doesn’t follow” gets full marks.
What it means for the other projects
It fits the harness’s founding result and the ladder: structure doesn’t help just by being there, and on this task no structure added anything at any of the four models. See the benchmark that said no.