In one of my current courses, some students have been struggling to understand p-values and why we do or do not use them. I stumbled across this NYT article this morning and was struck that so many scientists rushed to try to prove/disprove the original research. In my native field of education, we see few attempts at replication. I like the phrasing in this article about the ‘bump that wasn’t there.’ It seems that the original researchers admitted that there was a 1 in 93 chance that the particle they thought they had found was not really there. One in 93 equates to a probability of 0.0108, in essence, their p-value. For some reason, they went ahead with the claim of discovery when the prevailing alpha, or significance level, for their science is five sigmas – 0.00000029. Perhaps the journal in question defaults, like far too many, to an alpha of 0.05?
The Particle That Wasn’t