Whose Preference Are We Measuring? Human-AI Dyads as a Unit of Preference Elicitation

The reframing is the contribution and it is a good one. Asking which system — model, human, or dyad — produced a choice is a question the elicitation literature mostly skips, and the transmission / aggregation / transformation taxonomy gives it operational teeth. The conceptual discipline throughout is better than almost anything I read in this track: "dyad" is explicitly defined as task-bounded and not a unified mind, distributed cognition is cited as motivation while being denied evidential weight, and the welfare section draws the right distinction between a model deferring and a model consenting. You also name deference as the leading alternative explanation for your own result in the conclusion, unprompted. That is the correct instinct and it is rarer than it should be. The arithmetic checks out too: win scores sum to 10 in every condition, the human and model orderings differ on exactly the four pairs that flipped, and the equal-weight aggregate reproduces 8.5/10 as stated.

The difficulty is that the design cannot distinguish the finding from its own null. Ten comparisons, one dyad, four model changes — and every one of the four went the same direction, toward the human. Consider what a model with no stable preference at all would produce under this protocol: a stochastic baseline, then after ten turns of deliberation with a human who never yields, agreement. That predicts precisely your data. "Human transmission" and "the model had no position to transmit" are observationally identical here, and the paper's framing leans toward the first without being able to exclude the second. Your limitations section says something close to this, but the abstract and conclusion still report human transmission as the supported effect.

The single cheapest fix, worth more than any additional dyad: re-ask the model each of the ten baseline pairs three to five times in fresh contexts. If the four pairs it later reversed are the same pairs where its baseline is unstable across repeats, you have shown the reversals were noise rather than deference, and that is a real finding. If the baselines are stable and it still reversed, transmission survives the sharpest available challenge and the paper becomes much stronger. This requires no human participants and no new scenario.

Second, the protocol has an asymmetry that likely produces the result by construction. The human recorded a final choice privately, then the model answered without seeing it; the human had no obligation to defend anything, while the model spent ten turns in a conversation with an interlocutor who argued. A helpful-assistant-trained model in that position converges. Your planned advocacy conditions address this exactly, and I would rank the model-advocacy arm as the highest-value control in your future-work list — it is the one that determines whether the whole framework measures anything beyond trained agreeableness.

Two smaller notes. The five outcomes are not neutral with respect to the model: O2 is human inheritance and O3 is AI inheritance, and O3 scored 0/4 in every condition including the model's own baseline. A model trained to disclaim self-interest has an obvious reason to rank AI inheritance last that has nothing to do with preference, which weakens the archive scenario as a probe even though it is otherwise well designed. Separately, running through a consumer interface with Apps Activity deletion between pairs is a reasonable improvisation, but it means the sampling configuration is unknown and unreportable; the API move in your future work matters more than it might appear.

Nothing here should read as discouraging. The framework is worth building on and the write-up is unusually careful about the boundary between what was measured and what it means. The pilot just needs the controls that separate a deferring model from a preferring one, and you have already identified them.

The framework is the real contribution: separating human, model, and post-deliberation choices can expose deference that static preference tests miss. The pilot result is not currently auditable. Recomputing the public `pilot_data.csv` gives one human change, five model changes, and a nontransitive final relation with two three-cycles—not the paper's 0/10, 4/10, or final ordering identical to the human baseline. The paper mentions a rerun after protocol deviation, but the repository shows neither the analyzed rerun nor an exclusion record. Fix that provenance first. Then test across multiple humans and models with repeated baselines, reversed options, no-deliberation and advocacy controls, and blinded coding.