Tuesday, February 10, 2015

Bonferroni multiple comparison test

When performing multiple tests of significance upon a single dataset, there is an ever increasing chance that at least one of these tests will be statistically significant. For example, a p-value of 0.05 means there is a 1-in-20 likelihood of the statistical result occurring by chance alone.  If we perform 20 such tests, then there is a 1-in-1 likelihood of at least one result having a p-value of 0.05 or less.

The Bonferroni correction is simple. When multiple tests of significance are performed, then the cutoff p-value for statistical significance is set to equal 0.05 divided by the number of tests performed.

For example, if 10 tests of significance are performed, the cutoff p-value for statistical significance would be 0.05 / 10 = 0.005 when using the Bonferroni correction. If 5 tests are performed, then the cutoff p-value would be 0.05 / 5 = 0.01.