# lmPerm p-values and multiple testing

I've started using lmPerm in order to perform regressions in R. The equation I want to fit has the form:

out3 <- lmp(outcome ~  bin1 + bin2 + cont1 + cont2, perm="Exact")


Where "outcome" is a non-normally distributed continuous variable, and bin* and cont are binary and continuous regressors (similarly, they are non-normally distributed). Each variable has a length of approx. 110 cases.

Here are my questions:

1. This code works fine, but every time it runs in R, different p-values appear for each regressor. Which p-value should be reported in my results? I've tried repeating the test several times (in a loop) and getting an estimate from that, but I'm not sure it works...

2. If some of the predictors are changed and (then) several models / hypothesis are tested, should Bonferroni corrections be used in the same way they are applied for ordinary regressions? Is lmp somehow robust to multiple testing procedures?

-
Is this question only about how R implements this function? If so, it is probably better asked & answered on Stack Overflow rather than here, if not, please clarify. –  gung Sep 14 '12 at 20:30
Thanks for your fast reply. But no, my question is not about the implementation of the function. I put it here so that everybody could see it. As stated in the main post, my two questions are about 1) choosing a p-value after a regression using permutations, and 2) robustness of this procedure, and wheter or not (and how) Bonferroni corrections should be used. Regards, –  Elabore Sep 14 '12 at 23:28
This question needs to be separated into the two distinct questions contained within it: (1) Why can p-values vary between runs of a permutation test? (The answer, btw, is in the lnperm documentation at the top of p. 9.) (2) How does one protect against multiple comparisons when evaluating alternative models and multiple hypotheses? –  whuber Sep 16 '12 at 16:02