34
$\begingroup$

I have the following output:

Generalized linear mixed model fit by the Laplace approximation 
Formula: aph.remain ~ sMFS2 +sAG2 +sSHDI2 +sbare +season +crop +(1|landscape) 

 AIC   BIC    logLik deviance
 4062  4093  -2022   4044

Random effects:
Groups    Name        Variance Std.Dev.
landscape (Intercept) 0.82453  0.90804 
Number of obs: 239, groups: landscape, 45

Fixed effects:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  2.65120    0.14051  18.868   <2e-16     
sMFS2        0.26922    0.17594   1.530   0.1260    
sAG2         0.09268    0.14529   0.638   0.5235    
sSHDI2       0.28345    0.17177   1.650   0.0989  
sbare        0.41388    0.02976  13.907   <2e-16 
seasonlate  -0.50165    0.02729 -18.384   <2e-16 
cropforage   0.79000    0.06724  11.748   <2e-16 
cropsoy      0.76507    0.04920  15.551   <2e-16 

Correlation of Fixed Effects:
           (Intr) sMFS2  sAG2   sSHDI2 sbare  sesnlt crpfrg
sMFS2      -0.016                                          
sAG2        0.006 -0.342                                   
sSHDI2     -0.025  0.588 -0.169                            
sbare      -0.113 -0.002  0.010  0.004                     
seasonlate -0.034  0.005 -0.004  0.001 -0.283              
cropforage -0.161 -0.005  0.012 -0.004  0.791 -0.231       
cropsoy    -0.175 -0.022  0.013  0.013  0.404 -0.164  0.557

All of my continuous variables (denoted by a small s before the variable name) are standardized (z-scores). season is a categorical variable with 2 levels (early and late), and crop is a categorical variable with 3 levels (corn, forage, and soy).

This correlation of fixed effects matrix is really confusing me, because all of the correlations have the opposite sign that they do when I look at the simple regressions of pairs of variables. i.e., the correlation of fixed effects matrix suggests a strong positive correlation between cropforage and sbare, when in fact there is a very strong NEGATIVE correlation between these variables - forage crops tended to have much less bare ground compared to corn and soy crops. Pairs of continuous variables have the same issue, the correlation of fixed effects matrix says everything is the opposite of what it should be ... Could this just be due to the complexity of the model (not being a simple regression)? Could it have something to do with the fact that the variables are standardized?

Thanks.

$\endgroup$
33
$\begingroup$

The "correlation of fixed effects" output doesn't have the intuitive meaning that most would ascribe to it. Specifically, is not about the correlation of the variables (as OP notes). It is in fact about the expected correlation of the regression coefficients. Although this may speak to multicollinearity it does not necessarily. In this case it is telling you that if you did the experiment again and it so happened that the coefficient for cropforage got smaller, it is likely that so too will would the coeffienct of sbare.

In part his book "Analyzing Linguistic Data: A Practical Introduction to Statistics using R " dealing with lme4 Baayen suppresses that part of the output and declares it useful only in special cases. Here is a listserv message where Bates himself describes how to interpret that part of the output:

It is an approximate correlation of the estimator of the fixed effects. (I include the word "approximate" because I should but in this case the approximation is very good.) I'm not sure how to explain it better than that. Suppose that you took an MCMC sample from the parameters in the model, then you would expect the sample of the fixed-effects parameters to display a correlation structure like this matrix.

$\endgroup$
10
  • 4
    $\begingroup$ I'm sorry, this will probably be a silly question, but then why is it important to consider that correlation? I mean, in which situations should that output be considered? $\endgroup$
    – mtao
    Mar 30 '17 at 17:41
  • 1
    $\begingroup$ @Teresa It depends on what you are using it for. If you care about the interpretation, then it is telling you about how confusable two sources of effect are. If you care about prediction, it tells you a bit about what other prediction models might look like and gives you some hint as to how the model might change if you dropped a predictors. $\endgroup$ Apr 1 '17 at 15:11
  • 1
    $\begingroup$ So, imagine that I have two variables in that output with a correlation of 0.90, for instance. In terms of interpretation, I assume I should drop one of them, because they are "confusable" and seem to be telling the same information. As for prediction, if I drop one of them, other models shouldn't change that much, as they are correlated, am I right? Or am I interpreting this wrongly? $\endgroup$
    – mtao
    Apr 5 '17 at 7:48
  • 6
    $\begingroup$ You know, I think you're echoing what I said correctly; but, on reflection I'm not 100pct sure I'm right. You may be best served by opening a new question - that'll get more eyes on your question and increase the likelihood of you receiving a correct answer. $\endgroup$ Apr 5 '17 at 11:08
  • 2
    $\begingroup$ Continued: if there are relatively high correlations you may fit a GLMM, and the way to check whether it (or, more precisely, its random effects) satisfactorily modelled the dependencies is by computing the correlation matrix of the fixed effect models and comparing it to the one from the GLM. When GLMM addresses the issue of dependencies (e.g., longitudinal data), you will see a dramatic decrease in the correlations. $\endgroup$
    – Coca
    Feb 15 '19 at 11:33
1
$\begingroup$

It can be helpful to show that those correlations between fixed effects are obtained by converting the model's "vcov" to a correlation matrix. If fit is your fitted lme4 model, then

vc <- vcov(fit)

# diagonal matrix of standard deviations associated with vcov
S <- sqrt(diag(diag(vc), nrow(vc), nrow(vc)))

# convert vc to a correlation matrix
solve(S) %*% vc %*% solve(S)

and the correlations between fixed effects are the off-diagonal entries.

$\endgroup$
0
$\begingroup$

If your negative and positive correlations are the same in their value and only their sign differ, you are entering the variable mistakenly. But I don't think this is the case for you as you already seem quite advanced in stats.

The inconsistency you are experiencing can be and is likely caused by multicollinearity. It means when some independent variables share some overlapped effects, or in other words are correlated themselves. for example modeling to variables "growth rate" and "tumor size" can cause multicollinearity, as it is possible and likely that larger tumors have higher growth rates (before they are detected) per se. This can confuse the model. And if your model has few independent variables which are correlated with each other, interpreting the results can sometimes become quite difficult. It sometimes leads to totally strange coefficients, even to such extents that the sign of some of the correlations reverses.

You should first detect the sources of multicollinearity and deal with them and then rerun your analysis.

$\endgroup$
12
  • 1
    $\begingroup$ -1; misleading. OP didn't enter his variables incorrectly and multicollinearity may not be an issue. A correlation between the raw fixed effects might speak to this point, but Simpson's paradox may allow that approach to lead you in the wrong direction. $\endgroup$ May 26 '13 at 4:44
  • 1
    $\begingroup$ Why "misleading"? Which part was misleading? I talked very clearly and avoided inferring clear conclusions. What I said is indeed one of multicollinearity signs and tells us we should check VIFs too. But I don't understand how you know or are sure the "OP didn't enter his variables incorrectly and multicollinearity may not be an issue."? $\endgroup$
    – Vic
    May 26 '13 at 17:19
  • 1
    $\begingroup$ Besides you haven't even read my post completely (and downvoted it and call it misleading). If you had, you had seen that I have suggested that the OP should check VIFs (as official indicators for multiC) to make sure whether those high correlations are really pointing to MC or not? but anyways, I am open to learning as long as it is free of arrogance and personal attacks. $\endgroup$
    – Vic
    May 26 '13 at 17:19
  • 1
    $\begingroup$ @Vic: Didn't see your comments until just now. I didn't mean for you to view my response as a personal attack. I was of the opinion it was misleading and I provided what I believe to be the correct answer above. I read your post in its entirety at the time. I do not know whether I dug into the comments or not. I stand by my downvote. $\endgroup$ Jul 6 '14 at 19:37
  • 1
    $\begingroup$ ... but I do allow that I may be mistaken in that judgement. However, it seemed better to explain why I downvoted rather than to just downvote. $\endgroup$ Jul 6 '14 at 19:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.