I have fitted a model using
summary(model) I get a correlation matrix, but I don't understand how the correlations are calculated and what the interpretation is.
The documentation says "The estimated correlations of the estimated coefficients", but the estimated coefficients are real numbers (one-dimensional) so the correlation does not make sense.
I have fitted a model using
You are actually mentioning the answer to your question in your question's body. The coefficients you see are actually estimated. This means coefficients themselves are actually random variables that follow a distribution. What you see is one value of the random variable. The calculated correlation is the correlation between the random variables and not the correlation between the estimates which as you mention would not make sense.
This is why we do the t-test (hypothesis testing) for each of the coefficients and check how significant each one is.
To prove my point consider a super simple model:
a <- rnorm(100) b <- rnorm(100) df <- data.frame(a,b) > summary(glm(a~b, data=df), corr=TRUE) Call: glm(formula = a ~ b, data = df) Deviance Residuals: Min 1Q Median 3Q Max -2.48721 -0.64103 0.00034 0.66420 2.50019 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -0.095567 0.103944 -0.919 0.360 b 0.007084 0.107731 0.066 0.948 (Dispersion parameter for gaussian family taken to be 1.075244) Null deviance: 105.38 on 99 degrees of freedom Residual deviance: 105.37 on 98 degrees of freedom AIC: 295.02 Number of Fisher Scoring iterations: 2 Correlation of Coefficients: (Intercept) b 0.07
As you can see in the summary output, for the coefficients you have 4 columns. The estimate, the standard error, the t value and the p-value. The t-statistic (beta / standard error) follows a t-distribution and has an associated p-value.
So since both b0 (the intercept) and b1 are random variables a correlation between them can be calculated.
$\begingroup$ So since the correlation 0.07 is close to 0 it means that the things explained by the intercept is not explained by b, hence both parameters need to be included? In my case I have a Z-value, what difference does that make? Does it mean that my estimates can be considered normally distributed due to CLT (I have a large data set)? $\endgroup$– CamillaSep 15, 2015 at 10:35
$\begingroup$ @Camilla I don't know what family argument you are using in your glm call or what the call looks like but yeah this means that the coefficients are normally distributed. It doesn't necessarily mean that it is due to the central limit theorem. It might be that the model was constructed in such a way that the coefficients follow the normal distribution. $\endgroup$– LyzandeRSep 15, 2015 at 10:39
$\begingroup$ Uh, sorry I wasnt specific. I use
family=binomial(link="logit")$\endgroup$– CamillaSep 15, 2015 at 10:52
$\begingroup$ @LyzandeR, you have any links to prove your point about "correlation between random variables" B0 and B1? E[cor(B0,B1)]=0 if both random variables are independent. Point is that they are not, so the calculation is more subtle. And also I have some question marks about its meaning. $\endgroup$ Dec 28, 2015 at 17:20
The correlation of estimates is a scaling parameter so that the standard deviation of the estimated probabilities are constant regardless of linear transformations of the predictor variable. They have no substantive meaning. They only set the metric of the covariance.