# Reporting pseudo p-values for bootstrap-estimated linear regression coefficients

I've just received peer reviews for an applied paper that reported a table of multiple linear regression coefficients. While I reported 95% confidence intervals for these coefficients, a reviewer has requested I report exact p-values as well. The difficulty is that the confidence intervals were estimated via an ordinary non-parametric bootstrap, given the peculiarities of the data set.

Is there a way I can estimate 'pseudo p-values' from these confidence intervals? If so, under what assumptions/conditions are these p-values reasonable?

## 1 Answer

Sorry - was on a mobile platform and couldn't reply properly.

For bootstrapping, I'm assuming you resampled with replacement, computing the beta value each time to generate a distribution from which you computed the 2.5%ile and 97.5%ile. This gave you 95% confidence intervals.

For permutation, you'd resample from without replacement (shuffling up the case/control status or whatever the main factor is), get a distribution of null p values, and then compute you empirical p value based on wherever your observed falls in this null distribution. For details on the math, check out the book recommended on the link I posted above. Sorry for the piecemeal reply.

• Thanks, this is helpful. Is it reasonable to report both p-values calculated from a permutation test and confidence intervals calculated by a non-parametric bootstrap? – fmark Apr 28 '15 at 6:06
• Another obvious question (I'm new to permutation tests): Would I "get a distribution of null p values" or get a distribution of beta coefficients under the null hypothesis of no relationship? – fmark Apr 28 '15 at 6:10
• Please don't post one-line answers (your other one); they're generally better suited to comments. Please don't post two versions of the same answer from two different accounts. Please delete the shorter one. Please arrange to merge your accounts. – Glen_b Apr 28 '15 at 6:19
• You can report both the permuted p value and the 95% CI. In fact, I'd say it's always good to report both when possible. By the way, if your regression analysis involves time series or repeated measures, the permutation process can get a little bit more complicated than simple shuffling. Please see the following posts: stats.stackexchange.com/questions/20217/… and stats.stackexchange.com/questions/3520/… – N P Apr 28 '15 at 7:01
• @Glen_b, sorry about that. I am working on resolving the issue and I will go through the site tutorial. – N P Apr 28 '15 at 7:03