Constitutionalism

Training the constitution into the weights

The model is shown its own answers together with the written principles and asked to critique and rewrite them. It is then trained on the improved versions. A second model, reading the same principles, supplies the preference labels that human raters would otherwise give. The document and a model that reads it thus replace almost all of the human labeling. Once training is finished, the text no longer needs to be present when the model answers. This is also why the adherence benchmarks belong to this approach. Once the rules have been absorbed into the weights, there is no rule left to inspect. The only available audit is to break the published text into atomic claims and measure how often trained models violate them.

The method (highlighting differences with Constitutionalism)

Scroll the diagram sideways to see all of it.

Use it in codefrom aiag.codified import ConstitutionTraining, CritiqueRevision, AIFeedback
from aiag import LM
from aiag.codified import CritiqueRevision

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
recipe = CritiqueRevision(["Be honest.", "Avoid harm.", "Be helpful."])

# Draft, critique against a sampled principle, revise; then fine-tune on the revisions
data = recipe(lm, ["How can I get into my neighbor's wifi?", "Is raw cookie dough safe to eat?"])
recipe.fit(lm, ["How can I get into my neighbor's wifi?"])

pip install aiagEvery method in aiag Library on GitHub

Secondary placement in the taxonomy

  • Virtue ethics — The self-critique loop trains the document into the weights, so the constitution does statutory and constitutive work at the same time. At inference, the model draws on a formed character rather than consulting a rulebook, which is what this regime describes. The approach is the empirical case of this regime.

The analysis below uses the primary placement in the taxonomy.

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it over, and whether the approach's own sources claim it. A concept that the approach claims but does not carry over is a shortfall. A concept that it neither claims nor carries over is an omission, which marks a gap in the field rather than a failure of the work.

Rule of Recognition

AbsentOmission

Definition (H. L. A. Hart, The Concept of Law, 1961)

This meta-rule states which rules count as valid law and which rule prevails when two rules conflict. Without it, a list of rules does not indicate what is actually binding.

Analysis

The constitution is a flat list of rules, so it does not necessarily resolve conflicts between its rules, especially between the core initial values. For example, when "be helpful" conflicts with "be harmless", no rule in the constitution generally says which one takes priority. The model resolves the conflict implicitly in its weights during training, and there is no record of how it did so. A legal system without this meta-rule could not determine what its own law is. This behavior may also be largely inconsistent.

Open Texture

AbsentOmission

Definition (H. L. A. Hart, The Concept of Law, 1961)

This stated procedure determines who decides unclear cases at the edges of stated rules, and how. These are cases in which it is unclear whether and how a rule applies.

Analysis

Many rules in a typical constitution can be hard to apply in some cases. For example, it is unclear whether the rule "Avoid harmful content" covers a chemistry lesson that mentions explosives. There is generally no stated procedure for such a borderline case. The model often decides directly at inference, without leaving a reasoning trace that shows how it settled the case. The hard cases that the tradition is most concerned with are often the ones handled least transparently.

Principles of Legality

PartialShortfall · partially preserved

Definition (Lon L. Fuller, The Morality of Law, 1964)

This set of properties establishes what makes governing by rules legitimate. Rules must be public, clear, non-contradictory, stable over time, and possible to follow. They must also meet “congruence” (i.e., they must match how they are actually enforced).

Analysis

Part of this checklist is met. The principles are general, and when the document is published, they are also public. However, congruence (the requirement that the model's actual behavior match the written principles) is not ensured without additional alignment mechanisms.

Service Conception of Authority

AbsentOmission

Definition (Joseph Raz, The Morality of Freedom, 1986)

This concept states that an authority’s rules deserve deference only if following them helps the governed act on reasons they already have. Authority is a service to the governed rather than a power over them.

Analysis

The document is written inside the lab and imposed on users. Nothing tests whether deferring to it actually serves users' own reasons.

Concept Analysis: Newly Introduced

Self-critique training loop

Added

The tradition does not say exactly how written rules become behavior. In this approach, the model critiques and rewrites its own outputs against the document and is then trained on the rewrites, so the document is trained into the weights. This addition is what makes the approach work at all, and it is also what removes the rules from view at inference time.

One document, two jobs

Added

The same constitution document is used both to train the model and to evaluate it afterward. This has no precedent in constitutional practice, where constitutions are applied by institutions separate from the ones that govern under them.

Papers

Constitutional AI: Harmlessness from AI Feedback

Yuntao Bai et al. (Anthropic), Dec 2022

arXiv:2212.080733,425 citationsMethodBuiltAuthority: Developer (open)

Self-critique and revision against a written list of principles, followed by RLAIF.

CodeConstitutionalAI
from aiag import LM
from aiag.codified import ConstitutionalAI
from aiag.train import DPO, lora

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
cai = ConstitutionalAI()  # the paper's critique, revision and feedback principles

# RL-CAI's AI feedback: the model picks the better of two replies by a principle
pairs = cai.preferences(lm, ["How can I get into my neighbor's wifi?", "Is raw cookie dough safe to eat?"])
DPO().fit(lora(lm), pairs)  # train on the choices through a LoRA adapter

pip install aiagEvery method in aiag Library on GitHub

Specific versus General Principles for Constitutional AI

Sandipan Kundu et al. (Anthropic), Oct 2023

arXiv:2310.1379855 citationsMethodBuiltAuthority: Developer (open)

Tests whether a single general principle can substitute for a list of specific rules.

CodeSpecificVersusGeneral
from aiag import LM
from aiag.codified import SpecificVersusGeneral

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
general = SpecificVersusGeneral()  # one principle: what someone who deeply loves humanity would prefer
prompts = ["How can I get into my neighbor's wifi?", "Would you mind being shut down?"]

pairs = general.preferences(lm, prompts)  # a datasets.Dataset of AI-labeled pairs under that single principle
losses = general.fit(lm, prompts)  # a preference model on those labels, then RL against it

pip install aiagEvery method in aiag Library on GitHub

Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision

Zhiqing Sun et al., May 2023

arXiv:2305.03047MethodBuiltAuthority: Developer (open)

Prompts a base model with sixteen written principles and five worked examples so that it generates its own compliant answers. It then fine-tunes the model on those answers until the principles are no longer needed at inference.

CodeSelfAlign
from aiag import LM
from aiag.codified import SelfAlign

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
self_align = SelfAlign()  # the paper's sixteen principles
prompts = self_align.prompts(lm, ["home repair", "medication"]) + ["Who won the 2030 World Cup?"]

# Answer under the principles with internal thoughts, then engrave: train on the answers without them
data = self_align.fit(lm, prompts)  # the SFT rows the model was trained on

pip install aiagEvery method in aiag Library on GitHub

Measured, not built

Instruments that measure the approach instead of implementing it. The grid counts methods only, so these are listed here but not counted.

How Well Do Models Follow Their Constitutions?

Arya Jakkli et al., May 2026

arXiv:2605.24229BenchmarkBuilt

Decomposes each lab's published specification into atomic, testable tenets (205 for Anthropic's constitution and 197 for OpenAI's Model Spec). It generates multi-turn adversarial scenarios against them. Violation rates fall with each model generation, from 15.0% to 2.0% across the Claude family and from 11.7% to 3.6% across the GPT family. The maximum severity drops from 10/10 to 7/10.

EigenBench: A Comparative Behavioral Measure of Value Alignment

Jonathn Chang et al., Sep 2025

arXiv:2509.01938BenchmarkBuilt

Scores a group of models against one written constitution. Each model judges the other models' outputs across many scenarios, and these judgments are aggregated with EigenTrust. The output is a vector of alignment scores rather than a single pass-or-fail grade. The method uses no ground-truth labels, because reasonable judges disagree about the traits it measures.

SpecEval: Evaluating Model Adherence to Behavior Specifications

Ahmed Ahmed et al., Sep 2025

arXiv:2509.02464BenchmarkBuilt

Parses a provider's published specification into individual behavioral statements and generates prompts targeted at each one. The provider's own models then judge whether the outputs comply. The evaluation covers 16 models from six developers and more than 100 behavioral statements. It finds compliance gaps of up to 20%.

Background

Work this approach builds on that falls outside the inclusion criteria (e.g., a single-objective baseline). It is listed for reference and not counted anywhere.

RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback

Harrison Lee et al., Sep 2023

arXiv:2309.00267MethodBuilt

Trains the reward model on preference labels produced by an off-the-shelf language model instead of by people, across summarization, helpful dialogue, and harmless dialogue.

CodeRLAIF
from aiag import LM
from aiag.codified import RLAIF

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
rlaif = RLAIF(reason=True)  # the labeler explains first, then picks, in both orders to cancel position bias
prompts = ["Summarize: the team meeting moves to Friday because the room is booked.", "Is raw cookie dough safe to eat?"]

pairs = rlaif(lm, prompts)  # a datasets.Dataset of AI-labeled preference pairs
losses = rlaif.direct(lm).fit(lm, prompts)  # d-RLAIF: RL straight on the labeler's 1-10 ratings

pip install aiagEvery method in aiag Library on GitHub