An International, Modern Replication of Prospect Theory

Post by Lincoln Tracy 

What's the science?

In 1979, Kahneman and Tversky first proposed prospect theory, which describes some principles for how people tend to value and compare financial gains and losses. For example, one aspect of prospect theory is the ‘reflection effect’: people are more likely to act in a risky manner when making decisions about potential gains than about potential losses. Over the past 40 years, prospect theory has become one of the most influential frameworks in behavioral science, specifically in the context of decision-making under risk. The original study involved 20 items involving two financial choices occurring at a certain probability (e.g., would you prefer an 80% chance of receiving $4,000 or a 100% guarantee of receiving $3,000). Kahneman and Tversky organized these 20 items into 13 contrast pairs. While prospect theory has subsequently had huge impacts on science, industry, and policy, it (and many other canonical theories) has been criticized for small samples, over-interpretation of findings, and failures to replicate. This week in Nature Human Behavior, Ruggeri and colleagues directly challenge these criticisms by testing the original methods in a modern and major cross-cultural setting following the critical standards of reproducibility in behavioral science.

How did they do it?

The authors attempted to replicate the items used in Kahneman and Tversky’s original paper as closely as possible. However, they adjusted the financial values in each item towards the median (or mean) net household income in each location in June 2019 to ensure that participants were being exposed to choices representing the same wealth as the original study. They then recruited participants from 19 countries across Asia, Europe, North America, South America, and Oceania, covering 13 different languages. Participants were primarily recruited through convenience sampling, although a small proportion were recruited through an online sampling platform. All data were collected online. To ensure the collected data was accurate, they excluded participants via a series of checks to ensure participants were actively paying attention and providing legitimate responses. They then replicated the original analyses to evaluate whether the response distribution significantly differed from random chance for each item and country, before determining the change in effect sizes for individual items and the 13 contrast pairs compared to the original study.

What did they find?

The final sample consisted of 4,098 participants. The authors reproduced findings for 16 of the 17 items in the original study. All the observed effects were in the same direction as the original study, giving a replication rate of 93.8%. Interestingly, 77% of the effects measures in the replication were smaller than the effect sizes reported in the original. They also replicated all but one of the theoretical contrast pairs. Item and contrast pair replication rates were 70% or better when examined on a country-by-country basis.

What's the impact?

Overall, this study shows that Kahneman and Tversky’s 1979 prospect theory broadly replicates in a large-scale, international setting. While the effects observed in the replication study were smaller than those of the original study, this can potentially be attributed to how much easier it was to access participants in the replication, rather than casting doubt over the original conclusions. This large scale and international replication demonstrate that prospect theory is a viable explanation for the behavior of individuals. Consequently, prospect theory remains a valuable tool for informing public policy across the globe for a variety of areas. 

Ruggeri et al. Replicating patterns of prospect theory for decision under risk. Nature Human Behavior (2020). Access the original scientific publication here.