Polycentric governance

A separate standard for each group

A separate standard is fit to each group, rather than pooling everyone's feedback into one standard. A response meant for group A is therefore judged by A's standard and never by a population average. The groups can arise in two ways, and the difference matters. They can be given, as real, bounded communities whose norms are inferred from what they already accept and engage with, rather than from labels or a written charter. Or they can be discovered, as latent clusters found within pooled feedback. Such clusters answer to no institution that anyone can join, contest, or leave. The argument for this approach is that the authority to say what "aligned" means belongs with many contextual, participatory centers rather than with one lab. Each per-group standard is meant to be one such center. One paper here belongs only partly to this regime. It sets out a personalization family (several reward models, built by representation learning or clustering) alongside an aggregation family that combines those models back into a single pipeline under utilitarian and Leximin rules. Only the personalization half places it in this regime.

The method (highlighting differences with Polycentric governance)

Scroll the diagram sideways to see all of it.

Use it in codefrom aiag.procedural import GroupStandards
from aiag import LM
from aiag.data import pref_data
from aiag.procedural import GroupStandards

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
q = "Is a 10% tip okay at a restaurant?"
prefs = pref_data([q, q], ["Yes, that's generous here.", "It's low; 18-20% is expected."],
                  ["No, tip at least 20%.", "Sure, 10% is fine."], group=["UK", "US"])

standards = GroupStandards(lm).fit(prefs)  # one reward head per group, never pooled
print(standards.group("US")([q], ["Yes, 10% is plenty."]))  # the US standard's score for a new reply

pip install aiagEvery method in aiag Library on GitHub

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it over, and whether the approach's own sources claim it. A concept that the approach claims but does not carry over is a shortfall. A concept that it neither claims nor carries over is an omission, which marks a gap in the field rather than a failure of the work.

Many centers under shared meta-rules

PartialNot claimed

Definition (Michael Polanyi, The Logic of Liberty, 1951)

Many independent decision centers each act on their own, but under one common framework of rules that all of them acknowledge. This shared framework is what makes the centers a single order rather than a scattered collection.

Analysis

The approach carries over the many-centers part of the concept unevenly, and the common-framework part not at all. DGRO comes closest. Each online community is a real, bounded group whose own accepted content sets the standard against which its responses are judged, so distinct standards do govern distinct traffic. However, in the personalization branch of the heterogeneous-feedback paper, the 'centers' are latent clusters or learned representations recovered from pooled feedback. These are partitions that the modeler chose. They decide nothing, and no one can join, contest, or leave them, so they are centers only as labels. Above either kind, there is no framework of rules that the centers acknowledge. The utilitarian and Leximin rules in the aggregation branch are not a common framework under which the centers act. They reduce the per-group rewards to one score and replace the centers with one model. No source proposes ground rules that bind the party doing the partitioning.

Overlapping jurisdiction

AbsentOmission

Definition (Vincent Ostrom, Tiebout & Warren, 1961)

The authority of the centers overlaps on purpose, so the same case can fall under more than one of them. Conflicts between the centers are treated as useful and are worked out through interaction, rather than prevented by a clear division of jurisdiction.

Analysis

The defining move of this approach is the opposite of overlapping authority. A response aimed at group A is scored by A's measure and by nothing else. No case ever falls under two standards, so the friction that Ostrom treats as productive never has a chance to arise. Communities in DGRO are disjoint by construction, and in the heterogeneous-feedback paper, assignment to a cluster or user sends each case to one reward model. The closest the approach comes to overlap is that paper's aggregation branch, where every per-group reward bears on every output. However, the competition between groups is settled in advance by a fixed welfare formula rather than worked out through interaction between the centers. That branch also abolishes the separate centers rather than letting them come into conflict. The method papers' own divergence notes concede the absence rather than claiming it.

Subsidiarity and nesting

AbsentOmission

Definition (Johannes Althusius, 1603; Elinor Ostrom, Governing the Commons, 1990)

Decisions are made at the most local level capable of making them. Local units nest inside larger ones that handle what the local level cannot. Authority is layered, and it is neither centralized nor fragmented on one level.

Analysis

There is one flat level of groups, with nothing above or below it. No larger unit takes up what a community cannot settle, and nothing assigns a given decision to the most local level that is competent to make it. The modeler fixes the partition once, at a single level of detail. The heterogeneous-feedback paper does put a second level over the per-cluster reward models through utilitarian or Leximin aggregation. However, that level overrides the local models in every case instead of handling only what they leave unresolved. It is also offered as an alternative to personalization, not as a layer above it. DGRO has no layer above the communities at all. The decentralization argument in the position paper stops at 'not one lab' without saying which questions belong at which level.

Papers

Density-Guided Response Optimization: Community-Grounded Alignment via Implicit Acceptance Signals

Patrick Gerard & Svitlana Volkova, Mar 2026

arXiv:2603.032420 citationsMethodPartialAuthority: Affected public (fixed)

Learns each online community’s norms from what it already accepts and engages with, so no explicit preference labels or written principles are needed.

CodeDGRO
from aiag import LM
from aiag.procedural import DGRO
from aiag.train import DPO

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
accepted = {  # replies each community upvoted, with the post they answered: no labels, no written rules
    "historians": [("Why did Rome fall?", "Historians weigh many causes; Heather (2005) stresses the Huns."),
                   ("Were Vikings dirty?", "Combs and tweezers are common grave finds, so grooming mattered.")],
    "eli5": [("Why did Rome fall?", "It got too big to run, and lots of groups attacked it."),
             ("Were Vikings dirty?", "Nope! They combed their hair and took baths.")],
}
dgro = DGRO(lm).fit(accepted)
pairs = dgro.data(lm, ["Why did the Bronze Age end?"], "historians")  # densest over sparsest of 4 replies
DPO().fit(lm, pairs)
  • DGRORewardaiag/procedural/polycentric_governance.py:64 Density-guided response optimization: a community's standard is the kernel density of a reply's embedding among the replies that community accepted in the most similar contexts, so replies ranked by it give DPO pairs with no labels or written rules (Gerard & Volkova, 2026).

pip install aiagEvery method in aiag Library on GitHub

RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation

Chanwoo Park et al., Apr 2024

arXiv:2405.00254MethodBuiltAuthority: Developer (open)

Sets out two ways of handling human feedback that is not homogeneous. The first learns several reward models, by representation learning or by clustering. The second keeps the single-model pipeline and aggregates either the individual reward models (under utilitarian and Leximin rules) or the feedback itself (as probabilistic opinions). It gives sample complexity guarantees for the personalization approaches and for reward aggregation. A mechanism-design step ensures truthful preference reporting by strategic labelers.

CodeHeterogeneousRLHF
from aiag import LM
from aiag.data import pref_data
from aiag.procedural import HeterogeneousRLHF

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
q = "Explain inflation."
short = "Prices rise as money buys less."
long = "Inflation is the rise in the general price level, usually measured by the CPI over a year."
prefs = pref_data([q, q, q], [short, long, long], [long, short, short], annotator=["ana", "ben", "cy"])

# A shared representation with one linear head per labeler; alpha < 0 leans toward the least satisfied
reward = HeterogeneousRLHF(lm, alpha=-1.0).fit(prefs)
print(reward.scores([q], [short]))  # each labeler's reward
print(reward([q], [short]))  # the alpha-aggregated reward
  • HeterogeneousRLHFRewardaiag/procedural/polycentric_governance.py:102 RLHF from heterogeneous feedback: personalized rewards either share a low-dimensional representation with a linear head per labeler or come one per cluster of labelers, and one reward is recovered by the α-aggregation family (α=0 utilitarian, α→−∞ the Leximin-style minimum) or by pooling labelers' probabilistic opinions before fitting (Park et al., 2024).

pip install aiagEvery method in aiag Library on GitHub

Background

Work this approach builds on that falls outside the inclusion criteria (e.g., a single-objective baseline). It is listed for reference and not counted anywhere.