All it takes is 14 grad students studying the same thing targeting a 95% confidence interval for, on average, one to stumble upon a 5% case. Factor in publication bias and you get a bunch of junk data.
I think I heard this idea from Freakonomics, but a fix is to propose research to a journal before conducting it and being committed to publication regardless of outcome.
>but a fix is to propose research to a journal before conducting it and being committed to publication regardless of outcome.
Does not fix the underlying issue. Having a "this does not work" paper on your resume will do little for your career. So the incentives to make data fit a positive hypothesis are still there.
A great idea. Also known as a pre registered study.
Not familiar with this idea, but this idea is commonly applied for grant applications: only apply for a grant when you finished the thing you promise to work on. Then use the grant money to prototype the next five ideas (of which maybe one works), because science is about exploration.
Most pharma / medicine studies are pre-registered now. Sometimes the endpoints change based on what the scientists are seeing, but if they're worth their salt, they still report the original scoped findings as well.