3
$\begingroup$

I'm investigating bootstrap properties lately.

I took a multivariable analysis, generated the 10000 resamplings bootstrap empirical distribution for the effect size of each predictor (some continuos, some categorical) and investigated the structure of the correlations between these distributions.

enter image description here

The picture on the left shows the correlation matrix between the bootstrap distribution of each couple of predictors. The one on the right shows the same as a graph, with the edge thickness representing the correlation coefficient and the vertex sizes showing the mean / standard deviation of the bootstrap distribution; the edges which represent an absolute correlation lower than 0.6 are filtered out.

What is the meaning of these correlations? What do I learn from them?

Thanks

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

You have learned the well-known fact that the estimates of the model coefficients are generally speaking not independent. For many regression methods, there is a known formula for this covariance matrix. For example, for a linear regression $$V = \sigma^2 (X'X)^{-1}$$ So in this case this covariance matrix depends mostly on the design matrix: the stronger the correlations between the predictors, the larger the correlations between the estimated coefficients, and fully orthogonal predictors ($X'X=0$) give independent estimates. For glm's it is a bit more complicated: $$V = a(\psi)(X'WX)^{-1}$$ where $a(\psi)$ is a function of the dispersion parameter $\psi$, and $W$ is a weight matrix that depends on the link and variance functions of the model, but the idea is the same - the structure of the predictors explains the covariance patterns.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Ok thanks! These correlations what do tell me about the model? I mean what do I learn about the effect of this predictors on the outcome? Could I imagine that there is an interaction between the strongly correlated variables? Or do this correlation more have to do with confounding? That is I can remove from the model uncorrelated variables like "diab si" (diabetes yes) which seem neither to have an effect on the outcome nor needed to avoid confounding? $\endgroup$
    – Bakaburg
    Commented Oct 30, 2015 at 16:47
  • 1
    $\begingroup$ The information is definitely about the confounding, not interactions. But it picks up a lot of "fake" confounding because it is affected, for example, by centering covariates. $\endgroup$
    – Aniko
    Commented Oct 30, 2015 at 18:21
  • 1
    $\begingroup$ What you mean with the lfake confounding? I didn't center the covariates. $\endgroup$
    – Bakaburg
    Commented Oct 30, 2015 at 21:25
  • $\begingroup$ Example: If a covariate has a large mean, than it's estimate will have a large correlation with the intercept, however if you center it, that correlation disappears. The same can happen with two covariates with large means. Since these effects don't reveal anything of substantial interest, I called it "fake" confounding. $\endgroup$
    – Aniko
    Commented Nov 3, 2015 at 15:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.