# Bonferroni correction with Pearson's correlation and linear regression

I am running stats on 5 IVs (5 personality traits, extroversion, agreeableness, conscientiousness, neuroticism, openness) against 3 DVs Attitude to PCT, Attitude to CBT, Attitude to PCT vs CBT. I also added in age and gender to see what other effects there are.

I am testing to see whether personality traits can predict attitudes of the DVs.

I initially used Pearson's correlation for all variables (45 tests).

The main finding was that extroversion was correlated to attitude of PCT at p=0.05. But as I was running 45 tests I did a Bonferroni correction of alpha = 0.05/45 = 0.001, therefore making this finding insignificant.

I then ran a simple linear regression on all variables, again extroversion was significant with attitude to PCT. If I do the Bonferroni correction this it comes out insignificant again.

Questions:

1. Do I need to Bonferroni correct at Pearson's correlation?
2. If I do, and therefore making extroversion with attitude to PCT insignificant, is there still a point in doing linear regression?
3. If I do a linear regression, do I need to do the Bonferroni correction for this also?
4. Do I only report corrected valued or both uncorrected and corrected values?
-
Question 1 is related to Look and you shall find (a correlation) and Question 3 to Is adjusting p-values in a multiple regression for multiple comparisons a good idea?. More generally results from this query may be of interest. –  chl Jul 27 '12 at 13:46

I think Chl has pointed you to a lot of good material and references without directly answering the question. The answer I give may be a little controversial because i know some statisticians don't believe in multiplicity adjustment and many Bayesian's don't believe in p-value. In fact I once heard Don Berry say that using the Bayesian approach particularly in adaptive designs controlling type I error is not a concern. He took that back later after seeing how important it is practically to the FDA to make sure that bad drugs don't get to market.

My answer is yes and no. If you do 45 test you certainly need to adjust for multiplicity but no to Bonferroni because it could be far too conservative. The inflation of type I error when you data mine for correlation is clearly an issue that got attention with the cited post "look and you shall find correlation". All three links provide great information. What I think is missing is the resampling approach to p-value adjustment as developed so nicely by Westfall and Young. You can find examples in my bootstrap book or complete details in their resampling book. My recommednation would be to consider bootstrap or permutation methods for p-value adjustment and perhaps consider false discovery rate over the stringent family-wise error rate.

Recent book by Bretz et al on multiple comparisons: http://www.amazon.com/Multiple-Comparisons-Using-Frank-Bretz/dp/1584885742/ref=sr_1_2?s=books&ie=UTF8&qid=1343398796&sr=1-2&keywords=peter+westfall

My book with material in section 8.5 and tons of bootstrap references: http://www.amazon.com/Bootstrap-Methods-Practitioners-Researchers-Probability/dp/0471756210/ref=sr_1_2?s=books&ie=UTF8&qid=1343398953&sr=1-2&keywords=michael+chernick

-
+1 The reproduction of Graham Martin's Munchausen's Statistical Grid at the end of Westfall & Young says it all in a very engaging way. You can read this in the Amazon "look inside" feature. (It's almost as amusing to see Amazon offer a \$7 trade-in price for this \$150 book.) –  whuber Jul 27 '12 at 20:39
@whuber I think I saw a cartoon once sort of showing the Baron pulling himself out of a lake by his bootstraps. Efron may have been wise to call it the bootstrap since many are skeptical that it can be done in statistics just like many are skeptical about the legend of the Baron! –  Michael Chernick Jul 27 '12 at 23:01