There are many major issues that contribute:

  • Pressure to publish in academia, regardless of quality. Perverse motives.
  • Peer review is a poor filter for fraud and may not have the effect on credibility it’s meant to.
  • No motivation to reproduce some else’s research. Hard to be convinced on that usage of grant money.
  • Journals not as likely to publish negative findings (null results), as they’re perceived to have less value. They also don’t typically care about reproduction of previous findings, which contributes to a lack of motivation for researchers to replicate them.
  • Prestige and ego cause people to make poor choices.
  • Self-admitted questionable research practices:
  • Journals could be considered scientific gatekeeping…normalizing open access could help?
  • Honestly obtained incorrect results could lead to publication of outliers as they are surprising.
  • Poor experimental design…suggestive wording in surveys, unreliable sources for sampling, overfitting, confounding issues, etc. Unfortunately, many studies require deep domain knowledge to understand why it’s badly designed.
  • p-hacking or hypothesis changing after the fact.
  • View that research is a means of influencing policy and public opinion rather than seeking truth.
  • Can’t always publish the data used for outside verification (healthcare).

Some fields like particle physics and astronomy actually do focus on reproducibility.

All boils down to a question of incentives?

How do we spot issues proactively?

Further Reading