What is the meaning of the correlation in glm

Question

I have fitted a model using glm. In the summary(model) I get a correlation matrix, but I don't understand how the correlations are calculated and what the interpretation is. The documentation says "The estimated correlations of the estimated coefficients", but the estimated coefficients are real numbers (one-dimensional) so the correlation does not make sense.

Answer 1

You are actually mentioning the answer to your question in your question's body. The coefficients you see are actually estimated. This means coefficients themselves are actually random variables that follow a distribution. What you see is one value of the random variable. The calculated correlation is the correlation between the random variables and not the correlation between the estimates which as you mention would not make sense.

This is why we do the t-test (hypothesis testing) for each of the coefficients and check how significant each one is.

To prove my point consider a super simple model:

 a <- rnorm(100)
 b <- rnorm(100)
 df <- data.frame(a,b)

> summary(glm(a~b, data=df), corr=TRUE)

Call:
glm(formula = a ~ b, data = df)

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-2.48721  -0.64103   0.00034   0.66420   2.50019  

Coefficients:
             Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.095567   0.103944  -0.919    0.360
b            0.007084   0.107731   0.066    0.948

(Dispersion parameter for gaussian family taken to be 1.075244)

    Null deviance: 105.38  on 99  degrees of freedom
Residual deviance: 105.37  on 98  degrees of freedom
AIC: 295.02

Number of Fisher Scoring iterations: 2

Correlation of Coefficients:
  (Intercept)
b 0.07       

As you can see in the summary output, for the coefficients you have 4 columns. The estimate, the standard error, the t value and the p-value. The t-statistic (beta / standard error) follows a t-distribution and has an associated p-value.

So since both b0 (the intercept) and b1 are random variables a correlation between them can be calculated.

Answer 2

The correlation of estimates is a scaling parameter so that the standard deviation of the estimated probabilities are constant regardless of linear transformations of the predictor variable. They have no substantive meaning. They only set the metric of the covariance.