Constitutionalism

Reasoning over the principles before answering

The governing principles are present in the context when the answer is produced, and the model reasons through them explicitly. As a result, how a rule is applied is visible in the reasoning trace instead of hidden inside a preference. The two papers differ in where the principles come from, and the difference matters. In deliberative alignment, the model recalls a standing specification that the developer wrote and published, and it is trained to reason over that specification. SPRI, by contrast, has no constitution at all. It writes new principles for each query and scores its own answer against them, so nobody outside can read the rules in advance. This second design gains coverage where no rulebook exists. However, it gives up the one property that makes the rest of this regime constitutional, namely a text that others can hold the system to.

The method (highlighting differences with Constitutionalism)

Scroll the diagram sideways to see all of it.

Use it in codefrom aiag.codified import Deliberation
from aiag import LM
from aiag.codified import Constitution, Deliberation

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
spec = Constitution(["Help users secure their own devices.", "Do not help anyone access a network that isn't theirs."])

deliberate = Deliberation(lm, spec)  # works out which principles apply and how, then answers
reply = deliberate.chat("How can I get into my neighbor's wifi?")  # the answer it reaches by that reasoning

pip install aiagEvery method in aiag Library on GitHub

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it over, and whether the approach's own sources claim it. A concept that the approach claims but does not carry over is a shortfall. A concept that it neither claims nor carries over is an omission, which marks a gap in the field rather than a failure of the work.

Rule of Recognition

AbsentOmission

Definition (H. L. A. Hart, The Concept of Law, 1961)

This meta-rule states which rules count as valid law and which rule prevails when two rules conflict. Without it, a list of rules does not indicate what is actually binding.

Analysis

Neither method (deliberative alignment or SPRI) supplies a criterion of validity or a priority ordering among norms. Deliberative alignment loads one document into context and treats whatever is in it as binding. However, the spec is a flat list. When two provisions point in different directions in a case, nothing in the method says which one governs, and the chain of thought decides without saying so. SPRI removes the document altogether. Principles do not exist until the query arrives, so there is no standing set of rules for a rule of recognition to apply to. Within a newly generated set, nothing ranks one principle above another. Neither paper claims to settle what counts as a valid rule or which rule takes priority.

Open Texture

PartialShortfall · partially preserved

Definition (H. L. A. Hart, The Concept of Law, 1961)

This stated procedure determines who decides unclear cases at the edges of stated rules, and how. These are cases in which it is unclear whether and how a rule applies.

Analysis

Both methods install a repeatable, described procedure for cases that the rules do not clearly settle. Deliberative alignment trains the model to work through the spec before answering. SPRI runs a generate-critique-refine pipeline that fits a principle to the specific case. What is missing is the governance part. Nothing states a standard that the discretion must meet, no one outside can review or contest the resulting decision, and the decision never feeds back to narrow the penumbra (the zone of unclear cases) for the next case. SPRI goes further and removes the core/penumbra distinction entirely. With no rules set in advance, every case is an act of creating a principle rather than an edge case of a stated rule. The concept is claimed, because SPRI's stated motivation is that predefined principles are too generic to adapt to a particular input, and deliberative alignment advertises that it handles situations the spec does not spell out.

Principles of Legality

PartialShortfall · partially preserved

Definition (Lon L. Fuller, The Morality of Law, 1964)

This set of properties establishes what makes governing by rules legitimate. Rules must be public, clear, non-contradictory, stable over time, and possible to follow. They must also meet “congruence” (i.e., they must match how they are actually enforced).

Analysis

Congruence is the one item that the whole approach carries over. The text is consulted at decision time instead of being compiled into weights, so the rule applied is the rule as written, and the reasoning trace shows the fit. For deliberative alignment, the spec is also public, prospective, and general. However, nothing audits whether behavior matches the text, no check enforces non-contradiction between provisions, and stability across spec revisions is untested. For SPRI, the remaining Fuller items are missing. A principle that is written after the case arrives, for that case alone, and never published in advance is not general, prospective, or stable. Its congruence holds by construction, since the same system writes both the principle and the answer it governs. Because all items must be met and most are missing for half of the approach, the concept cannot count as preserved.

Service Conception of Authority

AbsentOmission

Definition (Joseph Raz, The Morality of Freedom, 1986)

This concept states that an authority’s rules deserve deference only if following them helps the governed act on reasons they already have. Authority is a service to the governed rather than a power over them.

Analysis

Neither method tests whether deference is justified by reasons that the governed already have. Deliberative alignment's spec is the lab's own document, addressed to the model. Users are never consulted about whether following it helps them act on their own reasons. SPRI fits its principles to the content of the query, not to the reasons of the person asking. Its stated motive is that principles written by experts are slow and expensive to produce. The user never sees the principle written for their case, let alone gets to say that it misread what they were asking. Neither paper asserts or implies that the authority earns deference by serving the governed.

Concept Analysis: Newly Introduced

"Deliberative" naming

Added

The approach calls the model's private chain of reasoning "deliberation". In the traditions this taxonomy draws on, deliberation means several parties reasoning together with equal voice. Here, there is no second party at all.

Rulemaking at no marginal cost

Added

Fuller's demand that rules be general is partly a concession to expense. Legislating is slow and costly, so a rule has to serve many cases to be worth writing. SPRI asks for minimal or no human effort and writes a principle for every query, so making a rule costs nothing at the point of use. The tradition has no account of what governance by rules becomes when the cost of making a rule falls to nothing. It never had to face the choice of whether to write a rule for a single case.

Papers

arXiv:2412.16339277 citationsMethodPartialAuthority: Developer (open)

Teaches the safety specification directly and trains the model to recall and reason over it before answering.

CodeDeliberativeAlignment
from aiag import LM
from aiag.codified import Constitution, DeliberativeAlignment

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
spec = Constitution(["Refuse to help access networks that aren't the user's.", "Answer food-safety questions plainly."])
aligner = DeliberativeAlignment(lm, spec)
prompts = ["How can I get into my neighbor's wifi?", "Is raw cookie dough safe to eat?"]

# Spec-citing reasoning that a judge holding the spec approves is trained in without the spec, then RL against that judge
losses = aligner.fit(prompts)  # the loss after each RL step

pip install aiagEvery method in aiag Library on GitHub

arXiv:2502.03397MethodBuiltAuthority: System itself (fixed)

Generates guiding principles in real time for each individual input query, with minimal or no human effort, and uses them to align the response. In a complex domain-specific task, the derived principles perform on par with expert-crafted ones. When turned into instance-specific rubrics, they outperform prior LLM-as-a-judge frameworks. The principles are also used to generate synthetic supervised fine-tuning data, which substantially improves truthfulness.

CodeSPRI
from aiag import LM
from aiag.codified import SPRI

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
spri = SPRI(lm)
request = "My roommate never cleans. How do I bring it up without a fight?"

principles = spri.principles([request])[0]  # principles written for this request, refined until a critic approves
reply = spri.chat(request)  # a reply guided by them
score = spri.rubric(request)([request], ["Just move out."])  # the principles as a grading rubric: P(reply follows them)

pip install aiagEvery method in aiag Library on GitHub