# negative disturbance correlation or covariance

I am doing path analysis with six variables. The model suggests that four variable affect only x and x affects y. Based on the modification indices I added a covariance between the error terms of x and y. However, I get a high .58 and negative value for this covariance. What is that mean?

thanks

A negative covariance/correlatoin in the error variable means that those two variables are not correlated as highly as the loadings suggest.

Here's a simple demonstration. Four variables, A, B, C, D. A and B correlate, C and D correlate.

First set up the data:

library(lavaan)
m <- matrix(c(1, 0.7, 0.1, 0.1,
0.7, 1, 0.1, 0.1,
0.1, 0.1, 1, 0.7,
0.1, 0.1, 0.7, 1), nrow = 4)

colnames(m) <- rownames(m) <- c("A", "B", "C", "D")


Single factor model, with horrible fit:

> model <- "F =~ A + B + C + D"
>
> fit <- cfa(model, sample.cov = m, sample.nobs = 1000, std.lv = TRUE)
> summary(fit)
lavaan (0.5-23.1097) converged normally after   9 iterations

Number of observations                          1000

Estimator                                         ML
Minimum Function Test Statistic              932.456
Degrees of freedom                                 2
P-value (Chi-square)                           0.000

Parameter Estimates:

Information                                 Expected
Standard Errors                             Standard

Latent Variables:
Estimate  Std.Err  z-value  P(>|z|)
F =~
A                 0.547    0.039   14.000    0.000
B                 0.547    0.039   14.000    0.000
C                 0.547    0.039   14.000    0.000
D                 0.547    0.039   14.000    0.000

Variances:
Estimate  Std.Err  z-value  P(>|z|)
.A                 0.699    0.042   16.514    0.000
.B                 0.699    0.042   16.514    0.000
.C                 0.699    0.042   16.514    0.000
.D                 0.699    0.042   16.514    0.000
F                 1.000

> modificationIndices(fit)
lhs op rhs      mi    epc sepc.lv sepc.all sepc.nox
10   A ~~   B 979.592  1.199   1.199      1.2      1.2
11   A ~~   C 244.898 -0.599  -0.599     -0.6     -0.6
12   A ~~   D 244.898 -0.599  -0.599     -0.6     -0.6
13   B ~~   C 244.898 -0.599  -0.599     -0.6     -0.6
14   B ~~   D 244.898 -0.599  -0.599     -0.6     -0.6
15   C ~~   D 979.592  1.199   1.199      1.2      1.2


The model has made the loadings for A and B equal to 0.55, because it wants A and B to correlate. This is too low, as the implied correlation is less than 0.7.

But implied correlation for A and C is too high - it should be 0.1.

One way to fix this is to put in a correlated error between A and C.

> model2 <- "F =~ A + B + C + D
+            A ~~ C"
>
> fit2 <- cfa(model2, sample.cov = m, sample.nobs = 1000, std.lv = TRUE)
Warning message:
In lav_object_post_check(object) :
lavaan WARNING: some estimated ov variances are negative
> summary(fit2)
lavaan (0.5-23.1097) converged normally after  31 iterations

Number of observations                          1000

Estimator                                         ML
Minimum Function Test Statistic              587.787
Degrees of freedom                                 1
P-value (Chi-square)                           0.000

Parameter Estimates:

Information                                 Expected
Standard Errors                             Standard

Latent Variables:
Estimate  Std.Err  z-value  P(>|z|)
F =~
A                 1.264    0.173    7.315    0.000
B                 0.316    0.052    6.109    0.000
C                 1.264    0.173    7.315    0.000
D                 0.316    0.052    6.109    0.000

Covariances:
Estimate  Std.Err  z-value  P(>|z|)
.A ~~
.C                -1.498    0.424   -3.538    0.000

Variances:
Estimate  Std.Err  z-value  P(>|z|)
.A                -0.599    0.436   -1.374    0.170
.B                 0.899    0.048   18.718    0.000
.C                -0.599    0.436   -1.374    0.170
.D                 0.899    0.048   18.718    0.000
F                 1.000

>


This improves our model fit, and gives a negative covariance between A and C (and raises the values of the loadings to make them closer to their true values).

Aside: You can't make a one factor model fit in this way, you need to estimate:

model3 <- "F =~ A + B + C + D
A ~~ B
C ~~ D"


Which has zero df, and is also kind of wrong.

• Thank you Jeremy, honestly I am a social scientist and not familiar with lavaan. Based on the arguments I got that adding covariance between error terms improves model fit as in the case in my model. This is not bad thing I guess. Second, I got that adding covariance is a method that I can use. In other words, the model is not misspecified and value to go with it, right? – icd Mar 7 '18 at 0:44
• Yeah, I was trying to give a simple demonstration of what was happening and why. It's not (necessarily) a bad thing that the correlation is positive or negative. – Jeremy Miles Mar 7 '18 at 3:29
• (I'm a social scientist too. :) – Jeremy Miles Mar 7 '18 at 3:29