Deliberative aggregation

A body of affected people is assembled. It can be an open crowd on a voting platform, a stratified sample, a citizens' assembly, or a panel drawn by lot. This body goes through a procedure, either proposing and voting on statements or discussing them until a common position forms. Whatever the body decides becomes the governing input to the model. This input can be a constitution to fine-tune on, a set of policy targets, or the pool of raters whose comparisons become the training data. The approach claims legitimacy through procedure. The rule is binding because of who decided it and how it was decided, not because of what it says.

A model takes over tasks that a human facilitator, chair, or annotator would perform when a group disagrees. It drafts a candidate group statement from everyone's opinions and critiques, and revises it as participants rank the versions. It selects sets of statements that represent the full range of opinion under formal fairness guarantees. It chairs a live session by managing the speaking queue and intervening when participants are uncivil. It speaks for stakeholders who are not present. It also rates each contribution for justification, novelty, and openness, in place of the human coders who used to do this. Some of these tasks run live inside a session, and some work offline from a collected set of opinions. What the papers share is that the model takes a role in the deliberation rather than using its output afterward.

The standard pipeline, reinforcement learning from human feedback (RLHF), pools everyone's comparisons into one reward model. This model implicitly averages people who disagree into a single substitute judge. In effect, it applies a voting rule that nobody chose, and it serves as the baseline for this approach. The other papers instead make an explicit collective decision rule the system's objective. Depending on the paper, the system maximizes the reward of the worst-off group, finds the policy that beats every alternative head to head, draws from a maximal lottery, aggregates whole policies rather than reward values, or matches the population's actual distribution of views. In the limiting case, the designed object is an economic redistribution rule rather than a language model policy, and it is optimized directly to win an election among the people it governs. The same tools also show what any such rule can achieve. This includes the axioms it satisfies, how far it can fall short of the ideal, whether a single labeler who misreports can break it, and impossibility results showing that no protocol satisfies everyone.

These systems collect many evaluations after training and combine them with a voting rule rather than an average, instead of building the aggregation into training. Several models from different model families, or one compact reward model fitted to each annotator, each score the candidate outputs and vote. The same idea treats existing benchmark and tournament results as ballots and searches for the ranking of agents that contradicts the fewest of them. Because the voting happens at evaluation time, who votes can be changed without retraining anything. This is what separates it from the approach that writes a voting rule into the training objective.

The system designs the question each person is asked, instead of aggregating whatever judgments people happen to offer, because the question is where the normative work happens. A model runs an open-ended interview, generates the edge case that would be most informative, guides someone through critiquing an agent's behavior until their own preferences take shape, or asks what value they actually applied to a hard case and then which value is wiser in that case. The designs differ in how the question is chosen, and the difference matters. Some choose the next comparison adaptively, based on information gain. One applies a single fixed framing (choosing principles without knowing which position one would occupy), which pushes answers toward protecting the worst-off. What separates all of these designs from voting platforms is that the system, not the participants, sets the agenda.

Concrete judgments about particular cases and candidate general principles are adjusted against each other until the two cohere. The collective answer is therefore not a tally at all. This process is modeled formally under measurable criteria of account, systematicity, and faithfulness. It is also read as a description of what constitutional AI already does, and shown to be the same search as coherence-driven self-improvement. Alongside this formal work, the papers argue that the target should be justified reasons rather than counts. They argue that preferences do not represent values, and that alignment should be settled by the claims people can fairly make on each other.