Social convention

The researcher fixes in advance what the agents choose between (e.g., a name for an object or an amount to donate). The researcher then varies properties of the population itself. These include how many agents there are, who is paired with whom, what each agent can see of what others did before, how many agents hold a fixed position and refuse to change it, and how many generations pass. Nothing in the environment states a rule, and nothing inside the agent stores one. The convention is simply the option the population settles on, and it holds only because everyone expects everyone else to keep to it. The papers report which option wins, how fast it wins, and how small a committed minority has to be to change the choice of the whole group.

The researcher builds a persistent world in which actions have material consequences, such as a shared fish stock that can be fished out, or a Minecraft world with roles, trade, and building. The agents are given ordinary language and no rulebook, and there is no set of conventions to choose from. Whatever arrangements they negotiate among themselves become the convention. The design assumes that a standard will form and last if there is something real to lose and the freedom to bargain over it. So far, however, the papers find that agreements are reached and then broken more often than they are kept.

This approach designs the system of sanctions written into the environment. It sets which acts count as violations, what it costs to punish someone, and what happens to an agent that lets a violation go unpunished. One group of papers adds rules that have no benefit of their own but whose violation is unmistakable. The argument is that cheap, obvious violations give a population constant practice at noticing and punishing, and that this habit then carries over to the rules that actually matter. The other group sets the payoffs directly (a gain from breaking the rule, a cost to punishing, and a further penalty for failing to punish). It asks whether agents that argue in ordinary language will build and maintain the chain of enforcement that keeps the convention in place.

Each agent is given a dedicated module whose only job is to handle the norm, rather than letting norm-following emerge from ordinary training or ordinary payoffs. The module infers which standard is actually in force by observing peers (e.g., who gets punished, which of several competing institutions the group defers to, and what neighbors say in conversation). It then feeds this back into how the agent plans and whom it sanctions. Two of the three papers go further and keep the norm stated in words, so the agent can check it and pass it to another agent. The third keeps the norm in a learned classifier of approved and disapproved behavior. This classifier behaves the same way from the outside, but it cannot be inspected or passed on.