Constitutionalism

This approach treats the written rules as an artifact in their own right. Someone decides which principles go in, how each one is worded, and whose values they encode. The text is then published so that outsiders can read it and hold the developer to it. Claude's Constitution and OpenAI's Model Spec are the two best-known published texts. The other papers here study the document rather than any model. They choose and frame the principles before training begins, force pairs of principles into conflict to reveal contradictions in the text itself, and compare the text with cross-cultural value surveys to see which culture's commitments it implicitly carries.

The model is shown its own answers together with the written principles and asked to critique and rewrite them. It is then trained on the improved versions. A second model, reading the same principles, supplies the preference labels that human raters would otherwise give. The document and a model that reads it thus replace almost all of the human labeling. Once training is finished, the text no longer needs to be present when the model answers. This is also why the adherence benchmarks belong to this approach. Once the rules have been absorbed into the weights, there is no rule left to inspect. The only available audit is to break the published text into atomic claims and measure how often trained models violate them.

Instead of one overall verdict on whether an answer was good, the constitution is broken into separate rules, each graded on its own. The per-rule grades are combined with explicit weights into the reward that drives training. This differs from the approach that trains the constitution into the weights in the shape of the signal, not in who produces it. That approach uses one overall preference over whole responses, and this one a grade for each rule with a visible weighting. Because each rule has its own score and weight, one can see which rule is being broken and retune it without rebuilding the reward.

The governing principles are present in the context when the answer is produced, and the model reasons through them explicitly. As a result, how a rule is applied is visible in the reasoning trace instead of hidden inside a preference. The two papers differ in where the principles come from, and the difference matters. In deliberative alignment, the model recalls a standing specification that the developer wrote and published, and it is trained to reason over that specification. SPRI, by contrast, has no constitution at all. It writes new principles for each query and scores its own answer against them, so nobody outside can read the rules in advance. This second design gains coverage where no rulebook exists. However, it gives up the one property that makes the rest of this regime constitutional, namely a text that others can hold the system to.

Rules are enforced by a component outside the model being governed. A second model inspects prompts as they come in and answers as they go out, and it blocks whatever the standard forbids. The standard itself varies. It can be a hand-written risk taxonomy, a statement of permitted and restricted content turned into synthetic training data for a classifier, or a rule tree extracted from a company's own product and design documents, with every node linked back to its source. It can also be an explicit weighting of harms and benefits rather than a list of rules. In every case, the standard is written down separately from the model it guards, so it can be rewritten or replaced without retraining anything. The source of the policy text is a variation within this approach, not a separate approach. What the papers share is that the rule and its enforcer can be detached from the model they govern.

This approach fixes an order of authority among the sources of text, instead of stating what the model may say. The developer's system prompt outranks the user, and the user outranks content pulled in from web pages, tool outputs, and other untrusted sources. The model is trained on examples that teach it to obey the higher-ranked source and to selectively ignore conflicting instructions from lower-ranked ones. This prevents an instruction hidden in a retrieved document from overriding the rules the operator set. The evaluation in this approach tests exactly that failure. Rules are given to the model in its prompt. A user then tries to talk the model out of them, and adversarial optimization pushes harder.