Redefine Statistical Significance with Edouard Machery

Are decisions made by scientists one century ago still with us and weigh down science? One decision involves the rules scientists play by when it comes to statistical significance, and more specifically, the rule that results fall below a .05 threshold to count as significant. The point of this threshold is to help minimize false positives. But .05 is consistent with at least 33% of results being false...or worse!

Links and Resources

• Edouard Machery
• The paper
• What a nerdy debate about p-values shows about science — and how to fix it
• The Alpha War by Edouard Machery
• Justify your Alpha by Lakens et al.
• Should We Redefine Statistical Significance? A Brains Blog Roundtable
• Abandon Statistical Signifcance by McShane et al.

Paper Quotes

"Ronald Fisher understood that the choice of 0.05 was arbitrary when he introduced it. Since then, theory and empirical evidence have demonstrated that a lower threshold is needed. A much larger pool of scientists are now asking a much larger number of questions, possibly with much lower prior odds of success. For research communities that continue to rely on null hypothesis significance testing, reducing the P value threshold for claims of new discoveries to 0.005 is an actionable step that will immediately improve reproducibility."