Polycentric governance

Training locally, sharing only model updates

Each group trains on its own members' feedback where that feedback is already stored. It then sends out only the result of training (model updates, raw parameters, or compact selectors that stand in for preferences). No group's data leaves the group, and no pooled corpus is ever assembled. This approach differs from the approach that fits a separate standard for each group in what is partitioned. Here, it is the pipeline (the data and the computation), and not necessarily the number of standards at the end. This distinction has consequences in both directions. One paper trains a personalized policy for each client, and so also meets that approach's test. It is placed here because its defining feature is the federated architecture. Another paper keeps data local but converges, by averaging, on a single shared preference predictor. Its plurality lies entirely in the process (who holds the data, who computes, and who can withhold or fork) and not in the outcome. This process-level plurality is a real lever that a contributor to a pooled corpus does not have, which is why these papers belong in a procedural regime. However, the single-model case has the weakest claim to partition among these papers.

The method (highlighting differences with Polycentric governance)

Scroll the diagram sideways to see all of it.

Use it in codefrom aiag.procedural import Federated
from aiag import LM
from aiag.data import sft_data
from aiag.procedural import Federated
from aiag.train import lora

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
clinics = {  # each clinic's notes stay on its own machine
    "north": sft_data(["How long does a flu shot take to work?"], ["About two weeks."]),
    "south": sft_data(["Can I exercise after a flu shot?"], ["Yes, light exercise is fine."]),
}
local = Federated().fit(lora(lm), clinics)  # each trains locally; only the adapter weights are averaged

pip install aiagEvery method in aiag Library on GitHub

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it over, and whether the approach's own sources claim it. A concept that the approach claims but does not carry over is a shortfall. A concept that it neither claims nor carries over is an omission, which marks a gap in the field rather than a failure of the work.

Many centers under shared meta-rules

PartialShortfall · partially preserved

Definition (Michael Polanyi, The Logic of Liberty, 1951)

Many independent decision centers each act on their own, but under one common framework of rules that all of them acknowledge. This shared framework is what makes the centers a single order rather than a scattered collection.

Analysis

Both parts of Polanyi's definition come closer here than anywhere else in the regime. Each group is a real decision center that trains its own preference model on its own data. The federated protocol is a common framework that every center acknowledges and operates under. However, the framework is procedural rather than normative. It governs how updates are combined, but not what any center may decide. What the centers share is therefore a common technical channel for updates rather than meta-rules.

Overlapping jurisdiction

AbsentOmission

Definition (Vincent Ostrom, Tiebout & Warren, 1961)

The authority of the centers overlaps on purpose, so the same case can fall under more than one of them. Conflicts between the centers are treated as useful and are worked out through interaction, rather than prevented by a clear division of jurisdiction.

Analysis

Clients are partitioned, and where preferences conflict, similar clients are grouped together so that heterogeneity is reduced. This is the opposite of what the tradition does. Overlap is treated as a problem to be engineered away rather than as useful friction.

Subsidiarity and nesting

PartialNot claimed

Definition (Johannes Althusius, 1603; Elinor Ostrom, Governing the Commons, 1990)

Decisions are made at the most local level capable of making them. Local units nest inside larger ones that handle what the local level cannot. Authority is layered, and it is neither centralized nor fragmented on one level.

Analysis

There is a real two-layer structure (local training under a global aggregation step), which is more nesting than any other approach in this regime has. However, subsidiarity is a rule about which decisions belong at which level, and here the algorithm fixes that allocation. Everything local is data, everything global is the merge, and nothing can be escalated because a group could not settle it.

Concept Analysis: Newly Introduced

Jurisdiction as a privacy consequence

Added

The centers here exist because raw preference data must not leave its group, rather than because any group claimed authority over its own affairs. Polycentricity is thus a by-product of a confidentiality constraint. This delivers the architecture the tradition describes but none of the authority that is supposed to justify it.

Papers

PluralLLM: Pluralistic Alignment in LLMs via Federated Learning

Mahmoud Srewa et al., Mar 2025

arXiv:2503.0992517 citationsMethodBuiltAuthority: Affected public (open)

Lets several user groups train a shared transformer preference predictor by federated averaging, without any group's feedback leaving the group. Compared with centralized training, it converges 46% faster with a 4% higher alignment score and near-identical group fairness.

CodePluralLLM
from aiag import LM
from aiag.procedural import PluralLLM

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
questions = ["Fund more bike lanes?", "Ban gas leaf blowers?", "Build a new stadium?",
             "Raise the bus fare?", "Extend library hours?"]
yes = {"urban": [0.7, 0.6, 0.4, 0.2, 0.8], "rural": [0.3, 0.2, 0.5, 0.4, 0.6]}  # share of each group answering Yes
groups = {g: [(q, ["Yes", "No"], [p, 1 - p]) for q, p in zip(questions, shares)] for g, shares in yes.items()}

predictor = PluralLLM(lm, context=3)
predictor.fit(groups)  # federated averaging: no group's answers leave it
print(predictor.alignment(groups["rural"][:3], groups["rural"][3:]))  # 1 - JS divergence on unseen questions
  • PluralLLMTraineraiag/procedural/polycentric_governance.py:222 PluralLLM: user groups train one GPO-style preference predictor, a small transformer that reads a group's own (option, preference) examples and predicts its preferences on new questions, by federated averaging weighted by group size, so no group's answers leave it (Srewa et al., 2025).

pip install aiagEvery method in aiag Library on GitHub

Background

Work this approach builds on that falls outside the inclusion criteria (e.g., a single-objective baseline). It is listed for reference and not counted anywhere.

Towards Federated RLHF with Aggregated Client Preference for LLMs

Feijie Wu et al., Jul 2024

arXiv:2407.03038MethodBuilt

Encodes each client's preferences as binary selectors and aggregates the selectors rather than the data. To handle heterogeneity, it groups clients with similar preferences. To resist reward hacking, it uses several selectors at once.

CodeFedBiscuit
from aiag import LM
from aiag.data import pref_data
from aiag.procedural import FedBiscuit
from aiag.train import DPO

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
q = "How should I answer a rude coworker?"
clients = {  # each user's own preferences, kept on their device
    "ann": pref_data([q], ["Stay calm and talk to them privately."], ["Snap back in front of everyone."]),
    "bo": pref_data([q], ["Raise it with your manager."], ["Ignore it forever."]),
    "cy": pref_data([q], ["Talk to them privately first."], ["Post about it online."]),
}
selectors = FedBiscuit(lm).fit(clients)  # 3 selectors, each federated over its cluster of clients
pairs = selectors.data(lm, [q])  # the model's own replies, labelled by the selectors' majority vote
DPO().fit(lm, pairs)
  • FedBiscuitTraineraiag/procedural/polycentric_governance.py:287 FedBiscuit: clients train binary selectors (which of two replies is better) on their own preferences in both orders, clients are grouped into balanced clusters by which selector fits their data best, each cluster's selector is federated-averaged, and the selectors' majority vote labels pairs of the model's replies for DPO; one selector is FedBis (Wu et al., 2024).

pip install aiagEvery method in aiag Library on GitHub

FedRLHF: A Convergence-Guaranteed Federated Framework for Privacy-Preserving and Personalized RLHF

Flint Xiaofeng Fan et al., Dec 2024

arXiv:2412.15538MethodBuilt

Each client incorporates its own human feedback into a local reward function and updates its own policy through a personalized RLHF loop. No raw data or human feedback leaves the client. On the MovieLens and IMDb datasets, it performs on a par with centralized RLHF while improving personalization across client environments. It also carries convergence guarantees and sample complexity bounds that scale efficiently with the number of clients.

CodeFedRLHF
from aiag import LM
from aiag.procedural import FedRLHF
from aiag.train import lora

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
ask = ["Recommend a novel for the weekend."]
brief = lambda prompts, replies: [-len(r) / 500 for r in replies]  # R0, the task reward: keep it short
likes = lambda word: (lambda prompts, replies: [float(word in r.lower()) for r in replies])  # H, a reader's feedback
readers = {"ana": (ask, brief, likes("mystery")), "ben": (ask, brief, likes("history"))}

# Each reader runs RLHF locally on R0 + λ·H; only parameter updates leave, and lm becomes their average
personal = FedRLHF(lam=1.0).fit(lora(lm), readers)  # each reader's last local adapter, their personalized policy
  • FedRLHFTraineraiag/procedural/polycentric_governance.py:356 FedRLHF: each client shapes a local reward from its own environment reward plus λ times its own human-feedback model, runs local RLHF steps on its copy of the policy, and only parameter updates leave it, averaged without weights; each client's last local policy is its personalized one (Fan et al., 2024).

pip install aiagEvery method in aiag Library on GitHub

Collaborative Content Moderation in the Fediverse

Haris Bin Zia et al., Jan 2025

arXiv:2501.05871MethodBuilt

Lets Fediverse servers exchange the parameters of their partially trained local moderation models with similar servers. Together these form a federated model shared among the collaborating servers. Average per-server macro-F1 is 0.71 on harmful content detection, 0.73 on bot content detection, and 0.58 on content warning assignment.

CodeFedMod
from aiag import LM
from aiag.procedural import FedMod

lm = LM("Qwen/Qwen2.5-1.5B-Instruct")
servers = {  # each server's posts, its moderators' labels (1 = remove), and its trending hashtags
    "art.example": (["Buy followers cheap!!!", "New watercolor today"], [1.0, 0.0], ["#art", "#painting"]),
    "photo.example": (["Free crypto, click here", "Sunset over the bay"], [1.0, 0.0], ["#photography", "#art"]),
    "tech.example": (["DM me for cheap followers", "Rust 1.80 is out"], [1.0, 0.0], ["#rust", "#linux"]),
}
fedmod = FedMod(lm, peers=1)
fedmod.fit(servers)  # each server averages only with the peer whose hashtags overlap most; no central server
print(fedmod.predict("tech.example", ["Get 10k followers now"]))  # 1 = remove, by tech.example's own model
  • FedModTraineraiag/procedural/polycentric_governance.py:374 FedMod: each Fediverse server trains its own moderation classifier on its own posts and, after each local step, averages parameters with the k servers whose trending hashtags overlap most with its own, peer to peer, with no central server (Zia et al., 2025).

pip install aiagEvery method in aiag Library on GitHub