Replication Crisis in Psychology: Which Famous Studies Failed?

Imagine citing a famous study in your term paper, then finding out that 23 labs tried to reproduce it and came up with almost nothing. That is the everyday reality of the replication crisis in psychology. Since the early 2010s, researchers have been re-running the field’s best-known experiments, and the results have been humbling. Some celebrated findings collapsed. Others survived with smaller effects. A few held up just fine. Knowing which is which changes how you read every study on your syllabus.

What Replication Actually Means

Replication in psychology means running a study again to see whether the same result shows up a second time. A direct replication copies the original procedures and materials as closely as possible. A conceptual replication tests the same idea with a different method, which shows whether an effect survives outside one particular setup.

Why does this matter so much? Any single study is just one sample of people tested at one moment. Even when everything is done correctly, the standard .05 threshold lets some false positives through by chance. Repeating a study is the only way to tell a real effect from a lucky one.

Reproducibility is something else. It asks whether another researcher can take the original data and analysis code and get the same numbers. That is why the popular label “reproducibility crisis” is a little loose. A study can pass that math check perfectly and still fail when new participants are tested.

Generalizability is a third question. A finding that replicates reliably with American undergraduates might not hold for older adults or other cultures. It is a close cousin of the external validity concerns that separate a consistent test from an accurate one.

How the Replication Crisis Started

Most historians of the field point to 2011. That year, social psychologist Daryl Bem published nine experiments in a top journal claiming people could sense the future. The paper used completely standard methods, which was exactly what alarmed critics. If ordinary methods could support precognition, the methods themselves needed a closer look.

The same year, Dutch researcher Diederik Stapel was found to have fabricated data, which eventually led to dozens of retractions. And Simmons and colleagues showed how routine analysis choices could make almost any hypothesis look statistically supported (Simmons et al., 2011).

The warning signs were not entirely new. Back in 2005, John Ioannidis had argued that small studies and flexible analyses meant many published findings across science were probably false. Psychology just became the field that tested the idea on itself most publicly.

It is worth keeping these apart. Stapel committed fraud. Most failed replications involve honest researchers following the accepted norms of their time, and that is what makes the crisis so uncomfortable.

The 2015 Number Everyone Quotes

The turning point came with the Reproducibility Project: Psychology. A team of 270 researchers, working as the Open Science Collaboration, redid 100 studies published in three leading psychology journals, using the original materials and high-powered samples.

Ninety-seven percent of the original papers had reported a significant result. Only 36 percent of the replications reached statistical significance, and the replicated effects averaged about half their original size (Open Science Collaboration, 2015). Social psychology fared worse than cognitive psychology, with roughly a quarter of its studies replicating compared with about half.

That 97 versus 36 gap is the figure everyone remembers. It does not mean two-thirds of psychology is fake. It means the field had been running on optimistic assumptions about sample sizes and effect sizes, and 2015 was when those assumptions finally got tested.

The exact number is still debated. Critics later argued that some replications differed from the originals in meaningful ways, such as testing a different population. Supporters countered that the overall pattern held up. Either way, the project made it impossible to treat a single published result as settled.

Replication Crisis Examples That Reshaped the Field

Statistics fade, but stories stick, and the replication crisis in psychology is easiest to understand through specific cases. These four replication crisis examples come up in almost every research methods course, and each one failed in a slightly different way.

Power Posing

A 2010 study by Dana Carney, Amy Cuddy, and Andy Yap reported that holding an expansive pose for two minutes raised testosterone, lowered cortisol, and increased risk-taking. Cuddy’s TED talk on the idea became one of the most watched ever.

A 2015 replication with 200 participants found that people did feel more powerful after posing. The hormone and risk-taking effects, however, did not appear (Ranehill et al., 2015). In 2016, Carney, the original lead author, publicly stated that she no longer believed the effects were real.

Ego Depletion

The idea that self-control runs on a limited resource shaped decades of research on willpower. In 2016, a Registered Replication Report tested it across 23 labs and more than 2,000 participants using one shared, preregistered protocol. Participants first crossed out letters under tiring rules, then completed a demanding computer task.

The combined effect was close to zero (Hagger & Chatzisarantis, 2016). The most striking detail is that 22 of the 23 labs had predicted beforehand that they would find an effect. These were not skeptics trying to debunk the idea.

The story did not end there. Some ego depletion researchers argued that the letter task was simply too weak to tire anyone out. That argument shows a real limit of replication work: a failure tells you a specific procedure did not work, which is not quite the same as proving an idea is false.

Precognition

Bem’s 2011 paper included an experiment suggesting that practicing words after a memory test improved earlier recall. Independent researchers repeated that procedure three times and found no effect at all (Ritchie et al., 2012). Today, the paper is cited far more as a warning about methods than as evidence about the future.

The Elderly Walking Study

A famous 1996 priming study reported that people exposed to words about old age walked more slowly when leaving the lab. In 2012, a team led by Stéphane Doyen repeated it with infrared timing sensors and a larger sample. The effect vanished (Doyen et al., 2012).

Then came the twist. In a second experiment, the team told some experimenters to expect slower walking and others to expect faster walking. A slowing effect appeared, but only for the experimenters who expected it. It is a textbook case of how researchers’ expectations, much like confirmation bias in everyday thinking, can quietly shape what gets recorded.

The episode worried Nobel laureate Daniel Kahneman, who had praised priming research in his own book. In a widely shared 2012 email, he warned priming researchers that their field was heading for serious trouble unless they replicated their own work.

What Actually Held Up

Here is the part that rarely makes headlines. In the first Many Labs project, researchers tested 13 classic and newer effects across 36 samples and more than 6,000 participants. Ten replicated consistently, including anchoring, the sunk cost effect, and gain versus loss framing (Klein et al., 2014).

One effect, imagined contact reducing prejudice, showed only weak support. Two subtle priming effects, involving flags and money, did not replicate at all.

The most useful lesson came from the comparisons. Whether an effect replicated depended far more on the effect itself than on the country, the lab, or whether participants took part online. Sturdy findings stayed sturdy almost everywhere they were tested.

The 2015 project pointed in the same direction. Studies with larger original effects and stronger statistical evidence were more likely to replicate than those that only scraped past the significance threshold (Open Science Collaboration, 2015). In other words, a result that barely made it the first time is the one most likely to disappear the second time.

Why This Kept Happening

Most of what drove the replication crisis in psychology did not require anyone to lie. A few ordinary research habits, repeated across thousands of studies, were enough to fill journals with results that looked solid but were not.

Small Samples

Smaller studies are cheaper and faster, so they were common, especially in social psychology. But small samples produce noisy estimates. A study with 40 participants can stumble into a significant result by luck far more easily than one with 400. How researchers choose and recruit their samples matters just as much as how many people they test.

Flexible Analysis

Researchers often made analysis decisions after seeing the data: which outliers to drop, which measures to report, when to stop collecting participants. Each choice seemed reasonable on its own.

Combined, they add up. Simmons and colleagues showed that stacking just a few of these choices could push the false positive rate from the assumed 5 percent to around 61 percent (Simmons et al., 2011). This practice is now widely known as p-hacking.

Publication Bias

Journals have long preferred surprising, positive findings. Studies that found nothing often went unpublished, a pattern known as the file drawer problem. As a result, the published literature overrepresented lucky hits and hid the quieter failures that would have balanced them out.

Older landmark studies have faced similar scrutiny. Questions about how Zimbardo ran the Stanford Prison Experiment and how much experimenters shaped Milgram’s obedience experiment raise the same worry about dramatic one-off findings.

Career Incentives

Hiring, grants, and promotion have long rewarded researchers for new, eye-catching discoveries. Running a careful replication of someone else’s study earned far less credit. That left few people with a reason to check whether famous findings were solid, which is how some of them went unchallenged for decades.

Is the Replication Crisis Over? What the 2026 SCORE Project Found

In April 2026, the largest replication effort yet published its results in Nature. The SCORE project attempted to replicate 274 claims from 164 papers published between 2009 and 2018 across 54 social and behavioral science journals.

About 55 percent of the claims replicated, or roughly 49 percent when counted by paper. The median effect size dropped from r = .25 in the originals to r = .10 in the replications (Tyner et al., 2026).

Two details matter for students. First, replication rates ranged from about 42 to 63 percent across disciplines, so the replication crisis in psychology was never only a psychology problem. Second, the team tried 13 different definitions of replication success, which produced estimates from about 29 to 75 percent. How you define success changes the headline.

The researchers also pointed out that some shrinkage was expected. Because they only retested results that had originally come out positive, some of those results were likely flukes to begin with, and some drop toward the true effect is natural. The size of the drop, though, is what worries people.

How Psychology Is Fixing Itself

The reforms have been real. With preregistration, researchers post their hypotheses and analysis plan before collecting data, which removes most of the after-the-fact flexibility behind p-hacking. Registered Reports go further: journals accept a study based on its methods before anyone knows the results.

Data sharing has become routine for many labs through platforms like the Open Science Framework. Some journals now award badges to papers that share their data or preregister their plans, and more outlets are willing to publish replications and null results. Large multi-lab projects, like the ones that tested ego depletion and the Many Labs effects, are now a normal way to settle disputed findings.

Reform has had its own stumbles. A 2023 paper by leading open-science researchers reported very high replicability for new findings produced with rigorous methods. It was retracted in 2024, partly because the authors had misdescribed what had been preregistered. Even the reformers, it turns out, need checking.

How to Read a Psychology Study After the Replication Crisis

You do not need to distrust every study. You just need a few habits that experienced researchers now use automatically:

• Check the sample size. A few dozen participants is a warning sign when the claimed effect is subtle.
• Look for preregistration or a Registered Report label, which means the analysis plan was fixed in advance.
• Ask whether an independent lab has replicated the result, ideally with a larger sample.
• Be suspicious when a tiny intervention, like a two-minute pose or a few words, claims a big change in behavior.
• Check the p-value. Results that barely cross the .05 line are the least likely to hold up.
• Treat a single dramatic finding as a hypothesis, not a fact.

The replication crisis in psychology is less a story about a broken science than about a science learning to check its own work. A field willing to retest its most famous findings in public is doing exactly what science is supposed to do.

References

• Doyen, S., Klein, O., Pichon, C.-L., & Cleeremans, A. (2012). Behavioral priming: It’s all in the mind, but whose mind? PLOS ONE, 7(1), e29081.
• Hagger, M. S., & Chatzisarantis, N. L. D. (2016). A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11(4), 546-573.
• Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Jr., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., . . . Nosek, B. A. (2014). Investigating variation in replicability: A “many labs” replication project. Social Psychology, 45(3), 142-152.
• Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.
• Ranehill, E., Dreber, A., Johannesson, M., Leiberg, S., Sul, S., & Weber, R. A. (2015). Assessing the robustness of power posing: No effect on hormones and risk tolerance in a large sample of men and women. Psychological Science, 26(5), 653-656.
• Ritchie, S. J., Wiseman, R., & French, C. C. (2012). Failing the future: Three unsuccessful attempts to replicate Bem’s ‘retroactive facilitation of recall’ effect. PLOS ONE, 7(3), e33423.
• Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359-1366.
• Tyner, A. H., Abatayo, A. L., Daley, M., Field, S., Fox, N., Haber, N. A., Hahn, K. M., Kline Struhl, M., Mawhinney, B., Miske, O., Silverstein, P., Soderberg, C. K., Stankov, T., Abbasi, A., Aberson, C. L., Aczel, B., Adamkovič, M., Albayrak, N., Allen, P. J., . . . Errington, T. M. (2026). Investigating the replicability of the social and behavioural sciences. Nature, 652(8108), 143-150.

How to cite this article:

The Psychology Notes Headquarters. (2026). Replication Crisis in Psychology: Which Famous Studies Failed?. Retrieved from https://www.psychologynoteshq.com/replication-crisis-in-psychology/