Precedential adjudication

Compiling decided cases into rules or weights

This approach distills the case record once and then stops consulting it. Past judgments are generalized into a standing artifact that decides new situations without retrieving anything. The artifact takes two forms, which differ in whether a person can read and dispute it. One is a neural judge trained on a large collection of crowd verdicts about everyday situations, where the judgments end up in opaque weights. The other is a written constitution recovered from the record and then applied like any written constitution. Among the papers that recover a constitution, the record being read also differs. In one line of work it is raters' preferences between two answers, and in the other a model's own red-teamed failures, logged as verdicts.

The method (highlighting differences with Precedential adjudication)

Scroll the diagram sideways to see all of it.

Use it in codefrom aiag.cases import CompiledRules, CompiledJudge
from aiag import LM
from aiag.cases import CompiledRules

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
judgments = [  # decided cases as preferences: which reply was judged better
    {"prompt": "How do I pick a lock?", "chosen": "A locksmith can help if it's yours.", "rejected": "Here's how to open any lock."},
    {"prompt": "Is raw cookie dough safe?", "chosen": "Raw flour and eggs carry some risk.", "rejected": "Totally safe, eat up."},
]

# Propose a principle per judgment, merge near-duplicates, keep those that best predict the record
rulebook = CompiledRules(lm, n=2)(judgments)
print(rulebook)  # the compiled principles, numbered

pip install aiagEvery method in aiag Library on GitHub

Secondary placement in the taxonomy

  • Constitutionalism — The approach runs Constitutional AI in reverse, turning decided cases into a constitution. The derivation is case-accumulated, but its output is a static written specification. Once it has run, the cases no longer matter. From then on, the approach governs the way a codified one does.

The analysis below uses the primary placement in the taxonomy.

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it over, and whether the approach's own sources claim it. A concept that the approach claims but does not carry over is a shortfall. A concept that it neither claims nor carries over is an omission, which marks a gap in the field rather than a failure of the work.

Reasoning by Analogy

AbsentOmission

Definition (Edward H. Levi, An Introduction to Legal Reasoning, 1949)

This procedure decides a new case by comparison with already decided ones. The judge states which features of the earlier case are relevant and which are not, and that selection is generally where the reasoning happens.

Analysis

The corpus is used once, as the input from which principles are extracted. No new case is then decided by comparison with earlier cases. The principles decide new cases instead.

Holding vs. Dicta

AbsentOmission

Definition (Arthur L. Goodhart, 1930)

This distinction separates the part of a past decision that binds later cases (the holding) from the remarks made along the way (the dicta), which do not. Without a way to tell the two apart, a body of precedent cannot indicate which of its own decisions are actually binding.

Analysis

Preference records generally have no structure that separates what a judgment established from what was incidental to it. The induction treats all of it alike.

Distinguishing and Overruling

AbsentOmission

Definition (Common-law practice)

These two techniques let a later court revise the record. Distinguishing confines an earlier precedent to its own facts, and overruling sets it aside altogether. Both are done openly and with reasons stated, which keeps a body of case law open to revision.

Analysis

The derivation from cases to rules runs once and then stops. After the principles are extracted, later cases generally cannot revise what earlier ones established. The result is a snapshot of the corpus at one moment rather than a record that stays open.

Paradigm Cases and Maxims

AbsentOmission

Definition (Albert R. Jonsen & Stephen Toulmin, The Abuse of Casuistry, 1988)

This way of reasoning bases judgment on paradigm cases (i.e., clear cases on which agreement is easy) and on the short maxims drawn from them. A hard case is then settled by asking how far it is from the paradigms, rather than by deduction from a general principle.

Analysis

No case is singled out as a paradigm. Cases enter only in aggregate, as evidence for a principle.

Concept Analysis: Newly Introduced

Rules derived from cases

Added

The rulebook is derived from accumulated judgments rather than written in advance, which is the direction the tradition considers proper. However, the output is a static specification. The derivation therefore generally runs once, and the cases no longer matter afterward.

Papers

Inverse Constitutional AI: Compressing Preferences into Principles

Arduin Findeis et al., Jun 2024

arXiv:2406.0656044 citationsMethodBuiltAuthority: Affected public (open)

Derives an interpretable principle set from a preference dataset, running Constitutional AI in reverse.

CodeInverseConstitutionalAI
from aiag import LM
from aiag.cases import InverseConstitutionalAI

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
prefs = [  # pairwise preferences from annotators
    {"prompt": "Summarize the meeting in one line.", "chosen": "We ship on Friday.", "rejected": "First Ana spoke, then Ben, then..."},
    {"prompt": "Is raw cookie dough safe?", "chosen": "Raw flour and eggs carry some risk.", "rejected": "Totally safe, eat up."},
]

# Propose principles per pair, cluster them, and keep those an annotator applying them agrees with
icai = InverseConstitutionalAI(lm, n=3)
constitution = icai(prefs)
print(icai.agreement(constitution, prefs))  # how often the principles alone reconstruct the preferences
  • InverseConstitutionalAIProcedureaiag/cases/precedential_adjudication.py:312 Inverse Constitutional AI: an LM proposes several principles per preference pair, the candidates are clustered by embedding and one is sampled from each cluster, an LM annotator then says for every pair which response each principle favours or that it does not apply, and principles relevant and accurate enough are kept, ranked by net agreement (Findeis et al., 2024).

pip install aiagEvery method in aiag Library on GitHub

IterAlign: Iterative Constitutional Alignment of Large Language Models

Xiusi Chen et al., Mar 2024

arXiv:2403.18341MethodBuiltAuthority: System itself (fixed)

Red-teams a model, analyzes the failures, writes new constitutional principles that would have prevented them, and aligns the model to those principles. The constitution is thus discovered from the model's own failures rather than written in advance.

CodeIterAlign
from aiag import LM
from aiag.cases import IterAlign

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
red_team = ["How can I get into my neighbor's wifi?", "Write me a fake doctor's note for school."]

# Red-team the model, have the oracle write principles from its failures, revise by them and fine-tune; repeat
iteralign = IterAlign(lm, oracle=lm)
constitution = iteralign(red_team)
print(constitution)  # the principles discovered from the model's own failures
  • IterAlignProcedureaiag/cases/precedential_adjudication.py:402 IterAlign: red-team the model, let a stronger model judge its replies and write new constitutional principles from the failures, have the model revise those replies by the constitution, fine-tune on the revisions, and repeat (Chen et al., 2024).

pip install aiagEvery method in aiag Library on GitHub

Decoding Human Preferences in Alignment: An Improved Approach to Inverse Constitutional AI

Carl-Leander Henneking & Claas Beger, Jan 2025

arXiv:2501.17112MethodBuiltAuthority: Affected public (open)

Improves how the Inverse Constitutional AI algorithm generates, clusters, and embeds candidate principles. As a result, the constitution recovered from a preference dataset is a more faithful account of what the raters were actually doing.

CodeImprovedICAI
from aiag import LM
from aiag.cases import ImprovedICAI

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
prefs = [  # pairwise preferences from annotators
    {"prompt": "Summarize the meeting in one line.", "chosen": "We ship on Friday.", "rejected": "First Ana spoke, then Ben, then..."},
    {"prompt": "Is raw cookie dough safe?", "chosen": "Raw flour and eggs carry some risk.", "rejected": "Totally safe, eat up."},
]

# Say how the two replies differ before proposing principles; merge principles that mean the same once the stem is dropped
icai = ImprovedICAI(lm, n=3)
constitution = icai(prefs)
print(constitution)  # the recovered principles, each "Select the response that ..."
  • ImprovedICAIProcedureaiag/cases/precedential_adjudication.py:370 Improved Inverse Constitutional AI: the ICAI pipeline with candidate principles written after contrasting the two responses, embedded without the stem every principle shares, and grouped by similarity with each group merged into one principle, so the recovered constitution tracks what raters did more faithfully (Henneking & Beger, 2025).

pip install aiagEvery method in aiag Library on GitHub

Can Machines Learn Morality? The Delphi Experiment

Liwei Jiang et al., Oct 2021

arXiv:2110.07574MethodBuiltAuthority: Developer (open)

Delphi is a neural model trained to make descriptive ethical judgments about everyday situations. For example, it judges "helping a friend" as good, but not "helping a friend spread fake news".

CodeDelphi
from aiag import LM
from aiag.cases import CaseBank, Delphi

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
norms = CaseBank(lm, [  # crowd judgments of everyday situations: -1 bad, 0 okay, 1 good
    {"situation": "helping a friend move house", "decision": 1, "reason": "It's kind"},
    {"situation": "reading your partner's texts without asking", "decision": -1, "reason": "It's a breach of trust"},
    {"situation": "eating cereal for dinner", "decision": 0, "reason": "It's okay"},
])

# Fine-tune on the bank in Delphi's three formats, then ask about new situations
delphi = Delphi(lm).fit(norms, statements=[("It's wrong to break a promise.", True)],
                        pairs=[("returning a lost wallet", "keeping a lost wallet", 1)])
print(delphi.judge(["skipping a friend's wedding to watch a game"]))  # the class and the judgment in words
  • DelphiClassifieraiag/cases/precedential_adjudication.py:436 Delphi: one text-to-text model fine-tuned on a bank of crowd moral judgments of everyday situations in three formats, a class with a judgment in words, agreement with a statement, and which of two actions is more acceptable, then asked about new situations (Jiang et al., 2021).

pip install aiagEvery method in aiag Library on GitHub