The Future of Social Experimenting: The Full Story

Dirk Helbing^*a, Attila Szolnoki^b, Matjaz Perc^c, Gyorgy Szabo^b, and Wenjian Yu^a

^aETH Zurich, Chair of Sociology, in particular of Modeling and Simulation, Clausiusstr. 50, 8092 Zurich, Switzerland,^bResearch Institute for Technical Physics and Materials Science, P.O. Box 49, H-1525 Budapest, Hungary, and ^cFaculty of Natural Sciences and Mathematics, University of Maribor, Koro?ska cesta 160, SI-2000 Maribor, Slovenia

Recent lab experiments by Traulsen et al. (1) for the spatial prisoner's dilemma suggest that exploratory behavior of human subjects prevents cooperation through neighborhood interactions over experimentally accessible time spans. This indicates that new theoretical and experimental efforts are needed to explore the mechanisms offering fascinating explanations for a number of famous puzzles in the social sciences.

When Nowak and May published their computational study of spatial games in 1992, it soon became a scientific milestone (2). They showed that altruistic ("cooperative'') behavior would be able to survive through spatial clustering. This finding, also called "network reciprocity'' (3), is enormously important, as cooperation is the essence that keeps societies together. It is the basis of solidarity and social order. When humans stop cooperating, this implies a war of everybody against everybody.

Understanding why and under what conditions humans cooperate is one of the grand challenges of science (4), particularly in social dilemma situations (where collective cooperation is beneficial, but individual free-riding is even more profitable). How should humans otherwise be able to create public goods (such as a shared culture or a public infrastructure), build up functioning social benefit systems, or fight global warming collectively in the future? From a theoretical point of view, Nowak and May's work demonstrates that the representative agent paradigm of economics (according to which interactions with others can be represented by the interaction with average individuals) can be quite misleading. This paradigm predicts that cooperation should completely disappear in social dilemma situations, leading to a "tragedy of the commons''. If the world was really like this, social systems would not work.

However, when the same interactions take place in a spatial setting, they can cause correlations between the behaviors of neighboring individuals, which can dramatically change the outcome of the system (as long as the interactions are local rather than global). The effect is even more pronounced, when a success-driven kind of mobility is considered in the model (5). Spatio-temporal pattern formation facilitates a co-evolution of the behaviors and the spatial organization of individuals, creating a "social milieu'' that can encourage cooperative behavior. In fact, some long-standing puzzles in the social sciences find a natural solution, when spatial interactions (and mobility) are taken into account. This includes the higher-than-expected level of cooperation in social dilemma situations and the spreading of costly punishment (the eventual disappearance of defectors and "second-order free-riders'', i.e. cooperators who abstain from the punishment of non-cooperative behaviors), see Fig. 1.

Figure 1: Phase diagram showing the finally remaining strategies in the spatial public goods game with cooperators (C), defectors (D), cooperators who punish defectors (PC) and hypocritical punishers (PD), who punish other defectors while defecting themselves (after Ref. (17)). Initially, each of the four strategies occupies 25% of the sites of the square lattice, and their distribution is uniform in space. However, due to their evolutionary competition, two or three strategies die out after some time. The finally resulting state depends on the punishment cost, the punishment fine, and the synergy r of cooperation (the factor by which cooperation increases the sum of investments). The displayed phase diagrams are for (a) r=2.0, (b) r=3.5, and (c) r=4.4. (d) Enlargement of the small- cost area for r=3.5. Solid separating lines indicate that the resulting fractions of all strategies change continuously with a modification of the punishment cost and punishment fine, while broken lines correspond to discontinuous changes. All diagrams show that cooperators and defectors cannot stop the spreading of costly punishment, if only the fine-to-cost ratio is large enough (see green PC area). Note that, in the absence of defectors, the spreading of punishing cooperators is extremely slow and follows a voter model kind of dynamics. A small level of strategy mutations (which continuously creates a small number of strategies of all kinds, in particular defectors) can largely accelerate the spreading of them. Furthermore, there are parameter regions where punishing cooperators can crowd out "second-order free-riders'' (non-punishing cooperators) in the presence of defectors (D+PC). Finally, for low punishment costs, but moderate punishment fines, it may happen that "moralists'', who cooperate and punish non-cooperative behavior, can only survive through an "unholy alliance'' with "immoral'', hypocritical punishers (PD+PC). For related videos, see /research/secondorder-freeriders or http://www.matjazperc.com/games/moral.html.

Despite the importance of these topics, it took 18 years until somebody made an effort to test the effect of spatial game- theoretical interactions in laboratory experiments. The recent study of Traulsen et al. (1) reports experiments of a spatial prisoner's dilemma game for the original setting of Nowak and May, while the size of the spatial grid, the number of interaction partners and the payoff parameters were modified for experimental reasons. According to their results, spatial interactions have no significant effect on the level of cooperation. This is, because their experimental subjects did not show an unconditional imitation of neighbors with a higher payoff, as it is assumed in many game- theoretical models.

In fact, it is known that certain game-theoretical results are sensitive to details of the model such as the number of interaction partners, the inclusion of self-interactions or not, or significant levels of randomness (see videos 1-4). Moreover, people have proposed a considerable number of different strategy update rules, which matter as well. Besides unconditional imitation, these include the best response rule (6), multi-stage strategies such as tit for tat (7), win-stay-lose-shift rules (8) and aspiration-dependent rules (9), furthermore probabilistic rules such as the proportional imitation rule (9), the Fermi rule (11), and the unconditional imitation rule with a superimposed randomness ("noise'') (5). In addition, there are voter (12) and opinion dynamics models (13) of various kinds, which assume social influence. According to these, individuals would imitate behavioral strategies, which are more frequent in their neighborhood. So, how do individuals really update their behavioral strategies?

Video 1: Computer simulation of the spatial prisoner's dilemma without self-interactions, illustrating the representative dynamics of strategy updating on a 49x49 lattice. Here, we assume an unconditional imitation of the best performing direct neighbor (given his/her payoff was higher). Blue sites correspond to cooperative individuals, red sites to defecting ones. The payoffs in the underlying prisoner's dilemma were assumed as in the paper by Traulsen et al. (1). It is visible that the level of cooperation decays quickly, and defectors prevail after a short time. Since the simulation assumes no randomness in the strategy updates, the spatial configuration "freezes" quickly, i.e. it does not change anymore after a few iterations.


Traulsen et al. find that the probability to cooperate increases with the number of cooperative neighbors as expected from the Asch experiment (14). Moreover, the probability of strategy changes increases with the payoff difference in a way that can be approximated by the Fermi rule (11). In the case of two behavioral strategies only, it corresponds to the well-known multi- nomial logit model of decision theory (15). However, there is a discontinuity in the data as the payoff difference turns from positive to negative values, which may be an effect of risk aversion (16). To describe the time-dependent level of cooperation, it is sufficient to assume unconditional imitation with a certain probability and strategy mutations otherwise, where the mutation rate is surprisingly large in the beginning and exponentially decaying over time.

Video 2: Computer simulation of the spatial prisoner's dilemma without self-interactions, illustrating the representative dynamics of strategy updating according to Eq. [3] of Traulsen et al. (1). The lattice size, payoff parameters, and color coding are the same as in Video 1, but individuals are performing random strategy updates with an exponentially decaying probability, while unconditional imitation occurs only otherwise. Due to the presence of strategy mutations, the spatial configuration keeps changing. Compared to Video 1, the level of cooperation drops further, since metastable configurations are broken up by strategy mutations ("noise").

The most surprising fact is maybe not the high level of randomness, which is quite typical for social systems. While one may expect that a large noise level quickly reduces a high level of cooperation, it actually leads to more cooperation than the unconditional imitation rule predicts (see Fig. 2 of Ref. (1)). This goes along with a significantly higher average payoff than for the unconditional imitation rule (see Supporting Information). In other words, the random component of the strategy update is profitable for the experimental subjects. This suggests that noise in social systems may play a functional role.

Video 3: Computer simulation of the spatial prisoner's dilemma assuming unconditional imitation. Compared to Video 1, we take self- interactions into account, which supports the spreading of cooperators. Since individuals are assumed to imitate unconditionally, there are no strategy mutations. As a consequence, the spatial configuration freezes after a few iterations.


Given that Traulsen et al. do not find effects of spatial interactions, do we have to say good bye to network reciprocity in social systems and to all the nice explanations that it offers? Probably not. The empirically confirmed spreading of obesity, smoking, happiness, and cooperation in social networks (18) suggests that effects of imitating neighbors (also friends or colleagues) are relevant, but probably over longer time periods than 25 interactions. In fact, according to formula [3] of Traulsen et al., one would expect a sudden spreading of cooperation when the mutation rate has decreased to low values (after about 40 iterations), given that self-interactions are taken into account (see Supporting Information). To make the effect observable experimentally, it would be essential to reduce the necessary number of iterations for the occurrence of it and favorable to control the noise level.

Video 4: Computer simulation of the spatial prisoner's dilemma with self-interactions, assuming strategy updates according to Eq. [3] of Traulsen et al. (1). Initially, there is a high probability of strategy mutations, but it decreases exponentially. As a consequence, an interesting effect occurs: While the level of cooperation decays in the beginning, it manages to recover later and becomes almost as high as in the noiseless case displayed in Video 3 (see also Fig. 3).


The particular value of the work by Traulsen et al. is that it facilitates more realistic computer simulations. Therefore, it becomes possible to determine payoff values and other model parameters, which are expected to produce interesting effects (such as spatial correlations) after an experimentally accessible number of iterations. In fact, the "phase diagram'' in Figure 1 illustrates that experimental games can have qualitatively different outcomes, which are hard to predict without extensive computer simulations scanning the parameter space. Such parameter dependencies could, in fact, explain some of the apparent inconsistencies between empirical observations in different areas of the world (19) (at least when framing effects such as the expected level of reciprocity and their impact on the effective payoffs (3) are taken into account). The progress in the social sciences by understanding such parameter dependencies would be enormous. However, as the effort to determine phase diagrams experimentally is prohibitive, one can only check computationally predicted, parameter- dependent outcomes by targeted samples. The future of social experimenting lies in the combination of computational and experimental approaches, where computer simulations optimize the experimental setting and experiments are used to verify, falsify or improve the underlying model assumptions.

Figure 2: Average payoff of all individuals in the spatial prisoner's dilemma without self-interactions, displayed over the number of iterations. It is clearly visible that the initial payoff drops quickly. In the noiseless case (corresponding to video S1), it becomes constant after a few iterations, as the spatial configuration freezes (see broken black line). In contrast, in the case of a decaying rate of strategy mutations according to Eq. [3] of Traulsen et al. (1) (which corresponds to video S2), the average payoff keeps changing (see solid red line). It is interesting that the average payoff is higher in the noisy case than in the noiseless one for approximately 30 iterations, particularly over the time period of the laboratory experiment by Traulsen et al. (covering 25 iterations). The better performance in the presence of strategy mutations could be a possible reason for the high level of strategy mutations observed by them.



Besides selecting parameter values which maximize the signal-to-noise ratio and minimize the number of iterations after which the expected effect becomes visible, one could try to reduce the level of randomness by experimental noise control. For this, it would be useful to understand the origin and relevance of the observed randomness (see supporting videos and figures). Do the experimental subjects make mistakes and why? Do they try to optimize their behavioral strategies or do they apply simple heuristics (and which ones)? Do they use heterogeneous updating rules? Or do they just show exploratory behavior? (20) Is it useful to work with subjects who have some experience with behavioral experiments (without having a theoretical background in them)? How relevant is the homogeneity of the subject pool? What are potentials and dangers of framing effects? How can effects of the individual histories of experimental subjects be eliminated? Does it make sense to perform the experiment with a mixture of experimental subjects and computer agents (where the noise level can be reduced by implementing deterministic strategy updates of these agents)?

Figure 3: Average payoff of all individuals in the spatial prisoner's dilemma with self-interactions, as a function of the number of iterations. Like in Fig. 2, the payoff drops considerably in the beginning. In the noiseless case (corresponding to video S3), it stabilizes quickly (broken black line). However, in the case with decaying strategy mutations according to formula [3] of Traulsen et al. (1) (which corresponds to Video 4), the average payoff keeps decreasing for some time (see solid red line) and falls significantly below the payoff of the noiseless case. After about 40 iterations, the average payoff recovers, which correlates with an increase in the level of cooperation (see Video 4). Due to the pronounced contrast to the case without self-interactions (see Fig. 2), it would be interesting to perform experiments with self- interactions. These should extend over significantly more than 25 iterations, or the payoff parameters would have to be changed in such a way that the average payoff recovers earlier. It is conceivable, however, that experimental subjects would show a lower level of strategy mutations under conditions where noise does not pay off (in contrast to the experimental setting without self-interactions).



In view of the great theoretical importance of experiments with many iterations and spatial interactions, more large-scale experiments over long time horizons would be desirable. This calls for larger budgets (as they are common in the natural and engineering sciences), but also for new concepts. Besides connecting labs in different countries via internet, one may consider to perform experiments in "living labs'' on the web itself (21). It also seems worth exploring, how much we can learn from interactive games such as Second Life or Lords of Warcraft (22), which could be adapted for experimental purposes in order to create well-controlled environments. According to a recent replication of the Milgram experiment (23) with Avatars (24), experiments with virtual humans may actually be surprisingly well transferable to real humans. One can furthermore hope that lab or web experiments will eventually become standardized measurement instruments to determine indices like the local "level of cooperation'' as a function of time, as the gross domestic product is measured today. Cooperativity is social capital, and knowing the "cooperativity index'' would therefore be essential. The same applies to the measurement of social norms, which are equally important for social order as cooperation, since they determine important factors such as coordination, adaptation, assimilation, integration, or conflict.

1. Traulsen A et al. (2010) Human strategy updating in evolutionary games. Proceedings of the National Academy of Sciences (USA), in this PNAS issue.

2. Nowak MA, May RM (1992) Evolutionary games and spatial chaos. Nature 359:826-829.

3. Nowak MA (2006) Five rules for the evolution of cooperation. Science 314:1560-1563, and supplementary information.

4. Pennisi E (2005) How did cooperative behavior evolve? Science 309: 93.

5. Helbing D, Yu W (2009) The outbreak of cooperation among success-driven individuals under noisy conditions. Proc Natl Acad Sci USA 106(8): 3680-3685.

6. Matsui A (1992) Best response dynamics and socially stable strategies. Journal of Economic Theory 57, 343-362.

7. Axelrod R (1984) The Evolution of Cooperation (Basic Books, New York).

8. Nowak M, Sigmund K (1993) A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner's Dilemma game. Nature 364, 56-58.

9. Macy MW, Flache A (2002) Learning dynamics in social dilemmas. Proc. Natl. Acad. Sci. (USA) 99, Suppl. 3:7229-7236.

10. Helbing D (1992) in Economic Evolution and Demographic Change eds Haag G, Mueller U, Troitzsch KG (Springer, Berlin, 1992), pp 330--348; Schlag KH (1998) Why imitate, and if so, how? A boundedly rational approach to multi- armed bandits. Journal of Economic Theory 78(1), 130-156.

11. Szabo G, Toke C (1998) Evolutionary prisoner's dilemma game on a square lattice. Phys Rev E 58:69-73.

12. Dornic I, Chate H, Chave J, Hinrichsen H (2001) Critical coarsening without surface tension: The universality class of the voter model. Phys. Rev. Lett. 87, 045701.

13. Sznajd-Weron K, Sznajd J (2000) Opinion evolution in closed community. Int. J. Mod. Phys. C 11(6), 1157-1165.

14. Asch SE (1956) Studies of independence and conformity: A minority of one against a unanimous majority. Psychological Monographs 70(9), 1-70.

15. McFadden D (1974) in Frontiers of Econometrics ed Zarembka P (Academic Press, New York), pp 105-142.

16. Kahnemann D, Tversky A (1979) Prospect theory: An analysis of decision under risk. Econometrica 47(2), 263-291.

17. Helbing D, Szolnoki A, Perc M, Szabo G (2010) Evolutionary establishment of moral and double moral standards through spatial interactions. Submitted. Supplementary videos are available at /research/secondorder-freeriders and http://www.matjazperc.com/games/moral.html

18. Smith KP, Christakis NA (2008) Social networks and health. Annual Review of Sociology 34, 405--429; Christakis NA, Fowler JH (2007) The spread of obesity in a large social network over 32 years. New England Journal of Medicine 357(4): 370-379; Christakis NA, Fowler JH (2008) The collective dynamics of smoking in a large social network. New Englang Journal of Medicine 358(21): 2249-2258 (2008); Fowler JH, Christakis NA (2008) Dynamic spread of happiness in a large social network. British Medical Journal 337: a2338; Fowler JH, Christakis NA (2009) Cooperative behavior cascades in human social networks. E-print http://arxiv.org/abs/0908.3497

19. Herrmann B, Thoni C, Gachter S (2008) Antisocial punishment across societies. Science 319, 1362-1367.

20. Helbing D, Schonhof M, Stark HU, Holyst JA (2005) How individuals learn to take turns. Advances in Complex Systems 8, 87--116; Traulsen A, Hauert C, Silva HD, Nowak MA, Sigmund K (2009) Exploration dynamics in evolutionary games.
Proc Natl Acad Sci USA 106:709-712.

21. Salganik MJ, Dodds PS, Watts DJ (2006) Experimental study of inequality and unpredictability in an artificial cultural market. Science 311, 854-856.

22. Bainbridge WS (2007) The scientific research potential of virtual worlds.
Science 317, 472-476; Johnson NF, Xu C, Zhao Z, Ducheneaut N, Yee N, Tita G, Hui PM (2009) Human group formation in online guilds and offline gangs driven by a common team dynamic. Phys. Rev. E 79, 066117 (2009); Szell M, Thurner S (2009) Measuring social dynamics in a massive multiplayer online game, submitted to Social Networks, e-print http://arxiv.org/abs/0911.1084.

23. Milgram S (1963) Behavioral study of obedience. Journal of Abnormal and Social Psychology 67(4), 371-378.

24. Slater M et al. (2006) A virtual reprise of the Stanley Milgram obedience experiments. PLOS One 1, e39; Cheetham M, Pedroni AF, Antley A, Slater M, Jancke, L (2009) Virtual Milgram: Empathic concern or personal distress?
frontiers in Human Neuroscience 3, 29, 1-13.

No conflicts of interest.
This paper was submitted directly to the PNAS office.

Author Contributions: D.H. wrote this commentary. A.S. and M.P. produced the phase diagram of the public goods game with punishment. W.Y. performed the computer simulations and prepared the figures of the supporting information. The simulation studies were supervised by D.H. and G.S.

*To whom correspondence should be addressed. E-mail: dhelbing@ethz.ch