# Correlation between bootstrap distribution of coefficients in multivariabile glm

I'm investigating bootstrap properties lately.

I took a multivariable analysis, generated the 10000 resamplings bootstrap empirical distribution for the effect size of each predictor (some continuos, some categorical) and investigated the structure of the correlations between these distributions.

The picture on the left shows the correlation matrix between the bootstrap distribution of each couple of predictors. The one on the right shows the same as a graph, with the edge thickness representing the correlation coefficient and the vertex sizes showing the mean / standard deviation of the bootstrap distribution; the edges which represent an absolute correlation lower than 0.6 are filtered out.

What is the meaning of these correlations? What do I learn from them?

Thanks

You have learned the well-known fact that the estimates of the model coefficients are generally speaking not independent. For many regression methods, there is a known formula for this covariance matrix. For example, for a linear regression $$V = \sigma^2 (X'X)^{-1}$$ So in this case this covariance matrix depends mostly on the design matrix: the stronger the correlations between the predictors, the larger the correlations between the estimated coefficients, and fully orthogonal predictors ($X'X=0$) give independent estimates. For glm's it is a bit more complicated: $$V = a(\psi)(X'WX)^{-1}$$ where $a(\psi)$ is a function of the dispersion parameter $\psi$, and $W$ is a weight matrix that depends on the link and variance functions of the model, but the idea is the same - the structure of the predictors explains the covariance patterns.