# Permutation test for the significance of several dependent regression coefficients from separate models

I have $i>100$ Cox regression models which differ only in one explanatory variable, $x_i$. They have the same outcome data and the same other explanatory (adjustment) variables.

I want to control the global Type 1 error probability/adjust for multiple testing of the significance of the coefficients of the $x_i$.

Some of the $x_i$ show high pairwise correlations. I inferred that therefore their coefficients and p-values are dependent. Neither Bonferroni nor Holm correction take into account these dependencies. The most attractive/exact option in my eyes is a permutation test that does consider them.

I thought about scrambling the rows of the outcome survival matrix, thereby conserving the correlations between the $x_i$ while destroying their relationship with the outcome. Then I compute the coefficients of all the $x_i$. In doing this for a few thousand different resamples I get the multivariate null distribution of the coefficients.

And this is where I'm stuck. I wanted to take the proportion of the null/resampled distribution for each coefficient estimated in the original data that is higher than it (or lower in the case of negative coefficients) as its p-value. But that would, again, destroy the dependency structure, as it treats the multivariate distribution as $i$ separate univariate ones.

How would one go about doing a permutation test for this problem? Is it even possible?

• Thank you. If I understand correctly, this would give me one p value expressing the probability of my coefficients given that there is no true relationship of any of the $x_i$ with the outcome? While I find this very interesting, it does not give me the answer I'm looking for; I need to be able to do inference for each coefficient while adjusting for multiple testing. Also I think as some of the coefficients have very low p values even after bonferroni correction, I can expect the global p obtained by the Mahalanobis distance procedure you described to turn out significant too. No? – miura Nov 9 '12 at 10:48
• I'm interested in the significance level of the coefficients of the $x_i$ after correction for multiple testing. I feel that the higher the correlation between an $x_i$ and another $x_j$, the more a test of the significance of the coefficient of $x_j$ is just a repetition of one of $x_i$, and they should not both contribute 1 to the number of tests a bonferroni or holm procedure corrects for - that would be overly conservative. After giving it some thought, maybe a multivariate procedure naturally considering the correlation structure is superior to my initial approach here. – miura Nov 14 '12 at 7:41
• This is very interesting. Let's say the general permutation test is significant, and there are 100 coefficients, would using $\alpha = 0.05$ be ok if we want to find which one of coefficients causing the outcome? Is this what you meant by a protected test? – NULL Oct 22 '17 at 1:06