Most scientists suspect that erroneous results occasionally make their way into publication. Now, Ioannidis confirms these suspicions, and worse, with statistical simulations. In a remarkably readable paper, he shows that "most published research findings are false"! Not surprisingly, bias increases the chances that the finding is false and large effects are more likely to be true than are small effects. Less expected is the conclusion that "a research finding is less likely to be true... when more teams are involved in a scientific field". In other words, "hot" fields have more mistakes published. The analysis is straightforward and may suggest reasonable ways to reduce the errors in publications.
Ioannidis, JPA. August 2005 PLoS Medicine 2(8):696-701
Sunday, January 21, 2007
Publishing your mistakes?
Subscribe to:
Post Comments (Atom)
2 comments:
I Lie
The self referential nature of a publication questioning publications brings to mind several pithy quotes:
* "All men are liars" Psalms (ch. CXVI, v. 11).
* "There are three kinds of lies: lies, damned lies, and statistics." Benjamin Disraeli.
And, maybe the best we can expect...
* "I do not mind lying, but I hate inaccuracy" Samuel Butler.
RE: "I Lie"
Actually one could also add to this that the very act of making a measurement disrupts the system, shades of Schrodinger, and a corollary would be the mode of measurement would do the same, thus, hmmm...what's the solution, or for that matter, is there one? Even the author concedes "Nevertheless, most new discoveries will continue to stem from hypothesis-generating research with low or very low pre-study odds." I suspect that as long as one's position in a scientific institution is directly related to one's 'productivity' - never mind the quality etc, there will always be this concern that "increasing concern that most current published research findings are false."
Post a Comment