Customary pluralism

Building a norm record for each community

Pipelines read what a community's own members post and the stories they tell, and compile the standards implicit in that text into a separate written record for that group (a value profile for each subreddit, or a knowledge base of cultural practices drawn from users' own stories). Nothing is reconciled across groups. Each record stands on its own, and the disagreements between records remain. The partition is therefore held in the corpus and remains whether or not any model ever reads it. The records are artifacts in their own right, and they are used afterward to steer, tune, or test a model.

The method (highlighting differences with Customary pluralism)

Scroll the diagram sideways to see all of it.

Use it in codefrom aiag.customary import NormRecords
from aiag import LM
from aiag.customary import NormRecords

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
posts = [
    "Please show what you tried before posting a homework question.",
    "Mods removed my thread because I didn't add a flair.",
    "Thanks all, marking this as solved so nobody wastes time on it!",
]
records = NormRecords(lm)(posts, "r/learnpython")  # a datasets.Dataset of (situation, behaviour, judgement) rows

pip install aiagEvery method in aiag Library on GitHub

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it over, and whether the approach's own sources claim it. A concept that the approach claims but does not carry over is a shortfall. A concept that it neither claims nor carries over is an omission, which marks a gap in the field rather than a failure of the work.

Prescription

PartialNot claimed

Definition (Edmund Burke, Reflections on the Revolution in France, 1790)

Long usage itself confers authority. A practice that has organized life for generations carries compressed, tested wisdom, even when nobody can articulate its justification.

Analysis

Practice is treated as the criterion of correctness. CultureBank's descriptors are built from users' own self-narratives, and ValueScope reads norms off how community members actually write. This deference to what a community does is half of Burke's claim. Duration is the other half, and it is entirely absent. Nothing separates a practice of generations from a TikTok post from last month. CulturePark's material has no usage behind it at all, since it is generated by agents inside a single simulation.

The unwritten constitution

PartialShortfall · partially preserved

Definition (A. V. Dicey, Law of the Constitution, 1885)

Some of a country's most binding norms are never written down. For example, the UK's constitutional conventions bind governments without being codified anywhere. Being binding and being codified are therefore different properties.

Analysis

ValueScope's subject is exactly the norms that nobody wrote down, which it recovers from linguistic and stylistic expression across 13 Reddit communities. It reports that even closely related communities differ sharply. The recovery is where it departs from Dicey. A norm that existed only in practice ends up as a score, as one descriptor among some 23,000, or as one of 3,307 taxonomized values. Dicey's point is that such norms bind without codification. Here, they are codified in the act of being found, and nothing in the pipelines is actually bound by them.

Organic amendment

PartialNot claimed

Definition (Edmund Burke, Reflections on the Revolution in France, 1790)

Norms change gradually from within a community's practice, through adjustment rather than by decree from outside. The community itself is the author of these changes.

Analysis

ValueScope traces how community norms evolve and records the emergence of new sub-communities, so change within practice is at least visible. The other change it names is the influence of a significant external event (the US presidential elections). This change comes from outside, rather than from a community adjusting its own practice. The other three artifacts are snapshots, and in none of the four can a community amend the record that now speaks for it.

Papers

No method has been built for this approach yet. It rests on the work listed below.

Source artifacts

Primary documents and shared infrastructure that this approach works from or produces.

CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies

Weiyan Shi et al., Apr 2024

arXiv:2404.15238ResourceBuilt

A pipeline turns users' self-narratives from online platforms into a knowledge base of cultural descriptors (12K from TikTok and 11K from Reddit). The knowledge base is then used both to evaluate models and to fine-tune one.

CodeCultureBank
from aiag import LM
from aiag.customary import CultureBank

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
narratives = [
    "In Japan we always say itadakimasu before eating, and I love it.",
    "My Japanese host family said itadakimasu before every meal.",
    "In Germany you wish everyone guten Appetit before the first bite.",
]
bank = CultureBank(lm)
records = bank(narratives, "tiktok")  # one record per cluster of similar practices, with its support and agreement

data = bank.data(records)  # grounded advice questions answered with the record in view, as SFT rows
  • CultureBankProcedureaiag/customary/customary_pluralism.py:195 CultureBank: an LM reads users' self-narratives into structured cultural descriptors, similar descriptors are clustered and summarized into one record each with its support and the share of narrators who endorse it, and the records then yield grounded evaluation questions and tuning data (Shi et al., 2024).

pip install aiagEvery method in aiag Library on GitHub

Background

Work this approach builds on that falls outside the inclusion criteria (e.g., a single-objective baseline). It is listed for reference and not counted anywhere.

ValueScope: Unveiling Implicit Norms and Values via Return Potential Model of Social Interactions

Chan Young Park et al., Jul 2024

arXiv:2407.02472PrecedentBuilt

A framework that uses language models to quantify the implicit norms and values of individual online communities from how their members write. It is applied to 13 Reddit communities grouped under gender, politics, science, and finance.

CodeValueScope
from aiag import LM
from aiag.customary import ValueScope

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
# The community's preference model: which of two comments in a thread its members upvoted more
scope = ValueScope(lm).train("r/askscience", [("Here is the paper with the measurements.", "lol no"), ("It depends on air pressure; see the phase diagram.", "just google it")])

profile = scope(["Water boils at 100C at sea level."], "r/askscience", ["formality", "humor"])
print(profile[0])  # return potential curve, point of maximum return, tolerable range
  • ValueScopeProcedureaiag/customary/customary_pluralism.py:161 ValueScope: a community preference model trained on which of two comments its members upvoted more scores comments rewritten at graded intensities of a norm dimension, giving the community's return potential curve and from it the point of maximum return and the range of tolerable behaviour (Park et al., 2024).

pip install aiagEvery method in aiag Library on GitHub