In 1988, a groundbreaking study found that the more we smile, the happier we are. Ten years later, another study showed that a person’s willpower can be worn out over time. Despite the fact that they are cornerstone studies and greatly represented in the media, the two studies might not be true.

Image in CC0 License.

Studies – even highly revered studies – are not necessarily flawless. It’s not necessarily that the scientists have done anything wrong, it’s just that it can be hard to control all the external variables. Psychology is especially vulnerable to this because it’s hard to create similar conditions for similar people and every tiny detail can lead to significant differences. The scientists asking questions in a different way, having the study in one type of room and not another, everything can screw things up. The setup has to be extremely careful thought out and this generates difficulties in reproducibility.

Reproducibility is the ability of an entire experiment or study to be duplicated, either by the same researcher or by someone else working independently. Reproducible research is key to science, and researchers working in psychology have been aware for some time of reproducibility issues within the field. A 2006 study found that of 141 authors of a publication from the American Psychology Association (APA) empirical articles, 103 (73%) did not respond with their data over a 6-month period. Even two famous studies which you most likely know as myths, seem to fail the reproducibility test.

Study 1: Smile and you’ll be happier

The 1988 study concluded that our facial expressions have an impact on our mood – the more you smile, the happier you’ll be. In the original paper, German researchers asked participants to read The Far Side comics by artist Gary Larson. They asked them to hold a pen either between their teeth (thus forcing a smile) or between their lips (thus forcing a pout). They found that those who were smiling found the comics funnier than those who made a pout and therefore concluded that changing our facial expressions changes our mood. This is called the facial feedback hypothesis.

However, when a team from the University of Amsterdam in the Netherlands replicated the study, they didn’t get the same results, even when they used the same comic.

“Overall, the results were inconsistent with the original result,” the team conclude in Perspectives in Psychological Science – a separate paper to the ego depletion replication, but also due to be published in a few weeks.

Study 2: Human willpower can be worn out

The 1998 study, led by Roy Baumestier from Case Western University established what is called ego depletion – the wearing out of human willpower. Of course, this raised tremendous interest and several follow-up studies were conducted. Most notably, Martin Hagger from Curtin University in Australia had researchers in 24 labs recreate the original study and found no significant results. That’s right, the famous ego depletion theory, who has huge real-life implications and is often referenced in popular culture, was not successfully replicated. The results were published in the journal Perspectives in Psychological Science.

What this means

This doesn’t necessarily mean that the studies are wrong. The first study, for instance, was replicated in 17 Dutch labs. Nine labs reported similar results to the original study, but the others didn’t – and when all results were mixed together, results were blurred out with no significant similarity to the original study. This is where psychology’s inherent replicability problems emerge.

It’s really, really hard to replicate psychology studies. Perhaps humor has changed and people just don’t find that comics funny anymore, or can’t relate to them. Furthermore, the participants in this study were psychology students, who might have been aware of the original study or perhaps are not representative for a more general population.

“It shows how much effort and attention has gone towards improving the accuracy of the knowledge produced,” John Ioannidis, a Stanford University researcher who led a 2005 reproducibility study, told Olivia Goldhill at Quartz.

“Psychology is a discipline that has always been very strong methodologically and was at the forefront at describing various biases and better methods. Now they are again taking the lead in improving their replication record.”

In a strange way, this is not necessarily a bad thing. From this reproducibility crisis, science will emerge stronger and more accurate, though we’re not yet sure how; and as Brian Nosek, who leads the Reproducibility Project that repeated 100 experiments says, science isn’t about truth and falsity – it’s about reducing uncertainty.