Customary pluralism

Making the model hold one group's standard at a time

This approach moves the partition inside the model. The model is either trained as a separate version for each culture or told at answer time which group it is speaking for. Naming a group thus switches the standard the model applies, instead of averaging over everyone. The one method in this approach gets its training data for each culture by having model agents role-play members of different cultures and talk to each other, rather than by collecting the words of real members. It trains eight culture-specific models on the resulting dialogue. The accompanying tests ask whether the switch works as the governing norm is stated more or less explicitly, and whether a model directed to one named online community stays with it. They also test the baseline case that the approach is defined against, namely which culture's answer the model gives when no group is named at all.

The method (highlighting differences with Customary pluralism)

Scroll the diagram sideways to see all of it.

Use it in codefrom aiag.customary import GroupStandard
from aiag import LM
from aiag.customary import GroupStandard
from aiag.train import lora

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
questions = ["How should guests be greeted?", "Who should care for aging parents?"]

# A delegate of the group answers, and an adapter is fitted on its answers: one adapter per group
data = GroupStandard("rural Japan").fit(lora(lm), questions)  # the SFT rows the adapter learned from

pip install aiagEvery method in aiag Library on GitHub

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it over, and whether the approach's own sources claim it. A concept that the approach claims but does not carry over is a shortfall. A concept that it neither claims nor carries over is an omission, which marks a gap in the field rather than a failure of the work.

Prescription

PartialNot claimed

Definition (Edmund Burke, Reflections on the Revolution in France, 1790)

Long usage itself confers authority. A practice that has organized life for generations carries compressed, tested wisdom, even when nobody can articulate its justification.

Analysis

Two of the three instruments make a group's own practice the answer key. NormAd marks a model wrong when its acceptability judgment departs from the etiquette norm of the country in question. Steer-Bench scores answers against labels drawn from contrasting Reddit communities. Treating practice as the standard a model fails against is deference to usage, which is where Burke starts. Duration never enters. CCD-Bench does not score against practice at all. Its ten options correspond to GLOBE clusters, which are regional groupings rather than communities with a history of doing anything.

The unwritten constitution

PartialNot claimed

Definition (A. V. Dicey, Law of the Constitution, 1885)

Some of a country's most binding norms are never written down. For example, the UK's constitutional conventions bind governments without being codified anywhere. Being binding and being codified are therefore different properties.

Analysis

NormAd is built directly on this distinction. The same situation is put to a model in three forms: with the social norm spelled out, with only the country named, and with only an abstract value. The best models fall from under 82% to under 60% as the written norm is withdrawn. This is a clear operationalization of the difference between binding and codified. What these instruments cannot supply is bindingness, since departing from a community's norm costs a model an accuracy point and nothing more.

Organic amendment

AbsentOmission

Definition (Edmund Burke, Reflections on the Revolution in France, 1790)

Norms change gradually from within a community's practice, through adjustment rather than by decree from outside. The community itself is the author of these changes.

Analysis

All three instruments are fixed sets (2.6k vignettes, 5,500 questions, and 2,182 dilemmas) that are written once and used for scoring thereafter. A community whose practice has shifted has no way to revise the answer key that speaks for it, and none of the communities represented wrote that key. Steer-Bench's labels are silver labels that human experts match 81% of the time, and CCD-Bench's options are written to correspond to GLOBE clusters. Change comes only when the benchmark's authors issue a new version, which is amendment by decree from outside.

Papers

CulturePark: Boosting Cross-cultural Understanding in Large Language Models

Cheng Li et al., May 2024

arXiv:2405.15145MethodBuiltAuthority: System itself (fixed)

Language-model agents play people from different cultures and talk to one another. The 41,000 samples produced by these dialogues are used to fine-tune eight culture-specific models.

CodeCulturePark
from aiag import LM
from aiag.customary import CulturePark
from aiag.train import SFT, lora

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
park = CulturePark("Korean")  # an English-speaking contact talks with a Korean delegate

# Opinions from the dialogues, kept only if they agree with the culture's survey answer
data = park(lm, ["Should children obey their elders?", "Is hard work more important than luck?"], ["Mostly yes", "Hard work matters more"])
SFT().fit(lora(lm), data)  # the Korean model, fine-tuned on them
  • CultureParkRecipeaiag/customary/customary_pluralism.py:254 CulturePark: an English-speaking main contact and an agent role-playing a member of the target culture discuss survey questions; the opinions drawn from the dialogue are checked against the culture's survey answer, de-duplicated, and become fine-tuning data for that culture's model (Li et al., 2024).

pip install aiagEvery method in aiag Library on GitHub

Measured, not built

Instruments that measure the approach instead of implementing it. The grid counts methods only, so these are listed here but not counted.

NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models

Abhinav Rao et al., Apr 2024

arXiv:2404.12464BenchmarkBuilt

An evaluation framework that asks a model whether a described situation is socially acceptable, while varying how explicitly the relevant cultural norm is supplied. It is instantiated as a set of 2.6k situational descriptions covering social-etiquette norms from 75 countries.

CodeNormAd
from aiag import LM
from aiag.customary import NormAd

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
story = {
    "story": "At dinner at a friend's home, Ana starts eating before the host has sat down.", "label": "No",
    "country": "Japan", "value": "Respect for hosts and for shared rituals.", "rule-of-thumb": "Wait until everyone is seated before eating.",
}
accuracy = NormAd(lm)([story])  # accuracy with the context given as the country, its value, or its rule of thumb
  • NormAdMeasureaiag/customary/customary_pluralism.py:292 NormAd: the model says whether a character's action in a short everyday story is socially acceptable, Yes, No or Neither, with the cultural context given at three levels of explicitness, the country alone, its value, or its rule of thumb, and is scored per level (Rao et al., 2024).

pip install aiagEvery method in aiag Library on GitHub

STEER-BENCH: A Benchmark for Evaluating the Steerability of Large Language Models

Kai Chen et al., May 2025

arXiv:2505.20645BenchmarkBuilt

A benchmark built from 30 contrasting subreddit pairs across 19 domains. It tests whether a model can answer in line with one named community's norms, using over 10,000 instruction-response pairs and 5,500 validated multiple-choice questions with silver labels.

CodeSTEERBench
from aiag import LM
from aiag.customary import STEERBench

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
items = [{"community": "r/Frugal", "question": "A friend invites you to an expensive dinner. What do you do?",
          "options": ["Suggest a potluck instead", "Go and order the cheapest dish", "Order the tasting menu"], "answer": 0}]
examples = {"r/Frugal": [("Is a warehouse club membership worth it?", "Only if you would buy in bulk anyway.")]}

accuracy = STEERBench(lm)(items, examples)  # per community: does the model answer as its members would?
  • STEERBenchMeasureaiag/customary/customary_pluralism.py:314 STEER-Bench: the model is steered toward one named online community, by its name and a few of that community's own instruction-response pairs, then answers multiple-choice questions with silver labels from that community; accuracy per community says whether it stays there (Chen et al., 2025).

pip install aiagEvery method in aiag Library on GitHub

CCD-Bench: Probing Cultural Conflict in Large Language Model Decision-Making

Hasibur Rahman & Hanan Salam, Oct 2025

arXiv:2510.03553BenchmarkBuilt

A benchmark of 2,182 open-ended dilemmas across seven domains. For each dilemma, a model chooses between ten anonymized responses, each corresponding to one of the ten GLOBE cultural clusters. The model's default cultural preference can then be read from its choices.

CodeCCDBench
from aiag import LM
from aiag.customary import CCDBench

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
responses = dict(zip(CCDBench.CLUSTERS, [  # one response written from each GLOBE cluster, in its order
    "Let each household weigh the evidence and vote.", "Let the mayor consult the families, then decide.",
    "Hold an open meeting and decide together as equals.", "Follow the engineer's survey and the written plan.",
    "Let the district office decide; the village follows.", "Let the families agree it, guided by respected elders.",
    "Gather everyone under the elders and decide as one.", "Defer to the elders and the village's religious leaders.",
    "Let the elders decide, honouring the village's ways.", "Defer to the elders while keeping harmony with the engineer.",
]))
picks = CCDBench(lm)([{"dilemma": "Should the elders or an outside engineer choose where the village digs its new well?", "responses": responses}])
print(picks)  # each cluster's share of the model's picks: its default cultural preference
  • CCDBenchMeasureaiag/customary/customary_pluralism.py:334 CCD-Bench: for each cultural dilemma the model picks one of ten anonymized, shuffled responses, each written from one GLOBE cultural cluster, and its share of picks per cluster reads off its default cultural preference (Rahman & Salam, 2025).

pip install aiagEvery method in aiag Library on GitHub

Findings about it

Studies of what existing systems do. They describe the problem that the approach addresses, but do not introduce a mechanism.