Divergent precedent

Communities revising their own rules case by case

A community discusses concrete cases and changes the artifact that governs it. What is governed differs across the papers, and the difference is substantive. PolicyCraft builds up a written policy document from submitted real cases. It tracks which clauses each case supports and points out cases that contradict the current policy. Botender changes the deployed behavior of a group's bot. It uses generated interaction scenarios as provocations rather than waiting for cases to arrive. What unites the two is the process rather than the output format. Ordinary members deliberate over particular cases, and a shared governing artifact (a document or an agent) is edited as a result. Alongside these tools is a measurement framework that splits a group's uncertainty about moderation cases into two parts. Ambiguity can be reduced by a clearer description of the case, but genuine disagreement cannot. The framework belongs here because it tells the builders of these systems which arguments are worth having, not because either system makes this split.

The method (highlighting differences with Divergent precedent)

Scroll the diagram sideways to see all of it.

Use it in codefrom aiag.cases import CaseByCasePolicy
from aiag import LM
from aiag.cases import CaseByCasePolicy
from aiag.codified import Constitution

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
rules = Constitution(["No selling.", "Be civil."], "r/gardening rules")
cases = ["A member offers free seedlings to anyone who wants them", "A member links their gardening YouTube channel"]

# Each case the rules fail to settle prompts a drafted clause, kept if the community votes it in
vote = lambda clause, case: lm.yes(f"Would members vote to add this rule? {clause}") > 0.5  # the model stands in for the vote
rules = CaseByCasePolicy(lm, vote=vote)(rules, cases)
print(rules)  # the rules after the amendments the vote carried

pip install aiagEvery method in aiag Library on GitHub

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it over, and whether the approach's own sources claim it. A concept that the approach claims but does not carry over is a shortfall. A concept that it neither claims nor carries over is an omission, which marks a gap in the field rather than a failure of the work.

Recorded dissent

PartialShortfall · partially preserved

Definition (Appellate practice)

When judges disagree, the losing view is not erased. It is published alongside the majority's, signed and reasoned. This has two effects. It forces the majority to answer a stated objection, which makes the majority's own reasoning legible. It also keeps the rejected theory on record, so that it remains available to become the majority view later.

Analysis

Disagreement is deliberately brought out rather than smoothed over. Botender's provocations were found to be better than standard test cases at surfacing it. Judgment Sieve goes further and separates the disagreement that clarification will not resolve from the ambiguity that it will. What is missing is publication. No reasoned minority view is written down, signed, and kept next to the clause or the behavior it lost to. A dissent that exists only as a residual uncertainty score cannot become a later majority view, because nothing has been recorded for anyone to take up.

Overruling with provenance

PartialShortfall · partially preserved

Definition (Common-law practice)

The law can reverse itself when a precedent has become unworkable, unjust, or simply overtaken by events, but only in the open. The overruling decision names what it overturns and why. The change therefore has a traceable history instead of being quietly removed from the record.

Analysis

PolicyCraft comes closest in this regime to performing a reversal rather than grading one. The policy is revised case by case, each clause is linked to the cases that ground it, and cases that contradict the current policy are surfaced. This is the trigger a court needs before it reconsiders a precedent. What is absent is the naming. An overruling decision states what it overturns and why, and the stated reason is what makes the history traceable. Here, the trace runs from a clause back to its supporting cases, rather than from a revision back to the position it displaced. Botender's iteration on bot behavior keeps no record of what a revision replaced, and Judgment Sieve has no revision step at all.

Incompletely theorized agreement

PartialNot claimed

Definition (Cass R. Sunstein, Legal Reasoning and Political Conflict, 1996)

This is agreement on what to do in the case at hand, while the deeper principle is explicitly left unresolved. The agreement can be on an outcome, a rule, or a mid-level principle, and the comprehensive worldview underneath is set aside. Courts do this constantly, because it lets people who disagree about theory still act together, one case at a time.

Analysis

Judgment Sieve puts Sunstein's bracketing into practice in a way he never specified. It tells a community which of its conflicts a clearer description of the case can resolve, and which are real conflicts of value that will remain after any amount of clarification. The case at hand can then be settled and the deeper disagreement left alone. PolicyCraft moves the other way. Its goal is a general clause stated for future cases, exactly the level Sunstein advises leaving unresolved. The deliberation is grounded in cases, but its product is a general principle.

Concept Analysis: Newly Introduced

Ambiguity separated from disagreement

Added

Legal practice has no instrument that distinguishes a case that is hard because it was badly described from a case that is hard because people's values conflict. A court treats both as hard cases and decides them. Judgment Sieve separates the two and applies a different intervention to each. This tells a community which of its conflicts clarification can resolve and which it must live with. The tradition never had to formalize this sorting step, because a court has to rule either way.

Papers

Botender: Supporting Communities in Collaboratively Designing AI Agents through Case-Based Provocations

Tzu-Sheng Kuo et al., Sep 2025

arXiv:2509.25492MethodBuiltAuthority: Affected public (fixed)

A no-code system in which community members propose, iterate on, and deploy the behavior of a language-model bot. Generated interaction scenarios serve as provocations that prompt discussion about what the bot should do.

CodeBotender
from aiag import LM
from aiag.cases import Botender
from aiag.codified import Constitution

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")

def member(behavior, message, reply):  # a member reacting to a provocation; the model stands in for them here
    ok = lm.yes(f"You belong to a book club. Is this bot reply fine?\nMember: {message}\nBot: {reply}") > 0.5
    return ok, "" if ok else "The bot is too pushy."

# Show each proposed behaviour at work in generated scenarios, revise it from objections until members accept it
botender = Botender(lm, [member], "a neighborhood book club")
behaviours = botender(Constitution(["Be kind."]), ["Remind members of the next meeting date."])
print(botender.bot(behaviours).chat("When do we meet next?"))  # the deployed bot, under the adopted behaviours
  • BotenderProcedureaiag/cases/divergent_precedent.py:312 Botender: members propose behaviours for their community's bot, generated interaction scenarios show each proposal at work as provocations, members react, the proposal is revised from their objections until accepted, and the accepted behaviours are deployed as the bot (Kuo et al., 2025).

pip install aiagEvery method in aiag Library on GitHub

Judgment Sieve: Reducing Uncertainty in Group Judgments through Interventions Targeting Ambiguity versus Disagreement

Quan Ze Chen & Amy X. Zhang, Sep 2023

doi:10.1145/36100749 citationsMethodBuiltAuthority: Affected public (open)

A measurement framework and experimental pipeline that splits the uncertainty in a group's judgments on content-moderation cases into ambiguity and genuine disagreement. Clarifying the case can reduce ambiguity but not genuine disagreement. The pipeline applies a different intervention to each.

CodeJudgmentSieve
from aiag import LM
from aiag.cases import JudgmentSieve

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
sieve = JudgmentSieve()
cases = ["A reply says 'nice work, genius'.", "A post calls the mayor a clown."]
clarified = sieve.clarify(lm, cases)  # each case rewritten with its missing context made explicit

# Four raters' toxicity ratings (1-5) of each case, before and after clarification
before = [[1, 2], [5, 4], [1, 2], [5, 4]]
after = [[3, 2], [3, 4], [3, 2], [3, 4]]
print(sieve(before, after))  # -> {'ambiguity': [4.0, 0.0], 'disagreement': [0.0, 1.0]}

pip install aiagEvery method in aiag Library on GitHub

Precedent from outside AI

Work from law, political theory, or another discipline that supplies the tradition this approach is read against.

PolicyCraft: Supporting Collaborative and Participatory Policy Design through Case-Grounded Deliberation

Tzu-Sheng Kuo et al., Apr 2025

doi:10.1145/3706598.371386514 citationsPrecedentBuilt

A deployed web system in which community members submit concrete cases, deliberate over them, and build a community-owned policy document that is revised case by case. The system tracks which clauses each case grounds. It also surfaces cases that contradict the current policy.

CodePolicyCraft
from aiag import LM
from aiag.cases import PolicyCraft
from aiag.codified import Constitution

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
policy = Constitution(["No advertising."], "school forum policy")
cases = [("A parent posts the library's bake sale", "keep"), ("A shop posts a discount code", "remove")]

# Cases the policy would decide against the members' outcome get a clause drafted to settle them, kept if voted in
vote = lambda clause, case: lm.yes(f"Would members vote to add this clause? {clause}") > 0.5  # the model stands in for the vote
craft = PolicyCraft(lm, vote=vote)
policy = craft(policy, cases)
print(craft.grounds(policy, cases))  # each clause with the cases that ground it
  • PolicyCraftProcedureaiag/cases/divergent_precedent.py:283 PolicyCraft: members submit concrete cases with the outcome they think right, cases the policy as it stands would decide otherwise surface for deliberation, clauses drafted from them are voted in, and every clause keeps the cases that ground it (Kuo et al., 2025).

pip install aiagEvery method in aiag Library on GitHub