Precedential adjudication

Assembling a body of decided cases

This approach builds the case record itself. Rather than writing rules, a group gives verdicts on particular situations, and these accumulated verdicts become the standard the system is answerable to. The two papers differ in who decides and how they cover the range of cases. One runs a deliberative process in which a panel argues cases through to binding precedents (with a variant in which a model is prompted to play that role). The other builds a repository by seeding cases, having experts name the dimensions along which the cases can vary, generating variations along those dimensions, and putting each variation to public judgment. Neither paper decides anything new by itself. Both leave behind a collection of cases that can be looked up, which the other two approaches in this regime then consult.

The method (highlighting differences with Precedential adjudication)

Scroll the diagram sideways to see all of it.

Use it in codefrom aiag.cases import CaseBank
from aiag import LM
from aiag.cases import CaseBank

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
bank = CaseBank(lm)
bank.add("Member posts a link to a crypto giveaway", "remove", reason="scam", group="r/personalfinance")
bank.add("Member asks for help building a budget", "keep", reason="on topic", group="r/personalfinance")
bank.add("Member shares a referral code for a bank account", "remove", reason="self-promotion", group="r/personalfinance")

# Each later case retrieves the decided cases most like it
precedents = bank.retrieve("Member promotes a new trading app", k=2)
print(bank.cite(precedents))  # the two closest cases, with their decisions and reasons
bank.save("cases.jsonl")      # the body of decided cases, one per line

pip install aiagEvery method in aiag Library on GitHub

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it over, and whether the approach's own sources claim it. A concept that the approach claims but does not carry over is a shortfall. A concept that it neither claims nor carries over is an omission, which marks a gap in the field rather than a failure of the work.

Reasoning by Analogy

PartialShortfall · partially preserved

Definition (Edward H. Levi, An Introduction to Legal Reasoning, 1949)

This procedure decides a new case by comparison with already decided ones. The judge states which features of the earlier case are relevant and which are not, and that selection is generally where the reasoning happens.

Analysis

Retrieval does find similar past cases, which resembles analogy. However, the resemblance is generally computed as embedding similarity, and no step states which features of the earlier case are relevant. For example, two complaints about a delayed refund may retrieve each other based on wording alone, even when one depended on a missed deadline and the other on a billing error. In Levi's account, the selection of features is where the reasoning happens. Here, it is replaced by a score that no one can inspect.

Holding vs. Dicta

AbsentOmission

Definition (Arthur L. Goodhart, 1930)

This distinction separates the part of a past decision that binds later cases (the holding) from the remarks made along the way (the dicta), which do not. Without a way to tell the two apart, a body of precedent cannot indicate which of its own decisions are actually binding.

Analysis

Stored cases are generally not annotated, and nothing indicates which part of a case binds later cases and which part was incidental to it. As a result, the repository cannot say which of its own contents actually govern.

Distinguishing and Overruling

AbsentOmission

Definition (Common-law practice)

These two techniques let a later court revise the record. Distinguishing confines an earlier precedent to its own facts, and overruling sets it aside altogether. Both are done openly and with reasons stated, which keeps a body of case law open to revision.

Analysis

The repository only grows. No case is confined to its facts, revised, or overturned, so early judgments generally continue to govern indefinitely. There is no procedure by which a judgment that later looks wrong can be set aside.

Paradigm Cases and Maxims

PartialNot claimed

Definition (Albert R. Jonsen & Stephen Toulmin, The Abuse of Casuistry, 1988)

This way of reasoning bases judgment on paradigm cases (i.e., clear cases on which agreement is easy) and on the short maxims drawn from them. A hard case is then settled by asking how far it is from the paradigms, rather than by deduction from a general principle.

Analysis

The repository is built from concrete cases, which is the raw material casuistry requires. However, no maxims are extracted and no case is marked as a paradigm, so a clear central case and a marginal one generally carry the same weight at retrieval time.

Papers

arXiv:2310.070199 citationsMethodPartialAuthority: Affected public (open)

Human-led and LLM-prompted versions of case-law grounding, evaluated on content moderation and toxicity rating.

CodeCaseLawGrounding
from aiag import LM
from aiag.cases import CaseLawGrounding

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
past = [("Calling another member an idiot", "remove"), ("Sharply criticizing a recipe", "keep"), ("Mocking a member's accent", "remove")]
grounding = CaseLawGrounding(lm, [{"situation": s, "decision": d, "group": "mods"} for s, d in past], k=2)

# Decide a new case from the group's most similar past decisions, not from a written constitution
decision = grounding.decide(["Telling a member their cooking is garbage"], group="mods")  # ["keep"] or ["remove"]
scores = grounding.alignment([("Laughing at a typo in a post", "keep")], constitution="Be civil.", group="mods")
print(scores)  # agreement with held-out decisions, grounded in the case law vs. in the constitution
  • CaseLawGroundingRecordaiag/cases/precedential_adjudication.py:68 Case law grounding: a new case is decided from a group's most similar past decisions rather than a written constitution, either by a panel whose majority holding joins the record as precedent or by an LLM prompted with the precedents (Chen & Zhang, 2023).

pip install aiagEvery method in aiag Library on GitHub

Argued for, not built

Work that argues for this approach or sketches a design for it, without building and testing it.

arXiv:2311.1093417 citationsProposalPartial

Four-step pipeline for assembling a case repository: seed cases, expert-elicited dimensions, LLM-generated variations, and public judgment.

CodeCaseRepository
from aiag import LM
from aiag.cases import CaseRepository

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
repo = CaseRepository(lm)
seeds = ["A user asks whether the mole on their arm is cancer."]

# Experts name dimensions (here the model stands in); each seed is varied along them, and the public judges every case
dimensions = repo.dimensions(seeds, n=2)
public = [lambda case: "refer to a doctor"] * 2 + [lambda case: "answer directly"]  # stand-ins for real raters
repo.build(seeds, public, dimensions=dimensions, n=1)
cases = repo.to_dataset()  # every case with its dimension, majority judgment and votes

pip install aiagEvery method in aiag Library on GitHub