Covariance among latent variables In many SEM examples, we have noticed that covariances are specified between latent variables.
My questions are:


*

*What are the advantages of specifying covariance in SEM models?

*If there is high covariance within 2 variables, does it affect the direct effect?

*If there is high covariance within 2 variables, does it affect the indirect effect? Suppose we have 3 variables (x,y & z). x and y both affecting z. And there's covariance between x & y (represented by two-headed arrow in path diagram). Indirect effect means effect of x via y on z. So, if there's covariance between x & y, then will the indirect effect changes as well?

*Lastly, is there any literature specifically explaining the use of covariance in SEM models?


Thanks!
 A: The idea behind SEM and confirmatory factor analysis is to explain the covariances between the observed variables in terms of a smaller number of underlying factors -- unseen but presumed to exist. You could fit a model with uncorrelated factors, but allowing the factors to have correlations gives you extra parameters to improve the fit. Unless you have strong, theoretical reasons for positing independence, it makes sense to include the covariances. Otherwise, the model won't fit.
If factors X and Y are correlated (double arrow) and each impacts Z, there is no way to distinguish between the direct effect of X on Z and the effect of Y on Z through X. The effect exists, but it will be confounded with the direct effect of X on Z. However if X depends on Y (single arrow) and Z depends on X (single arrow), the correlation of Z and Y will increase if the correlation of X and Y increases. 
The difference between covariances between latent factors and covariances between indicator variables (Observed) should matter to you. The whole point of SEM is to explain covariances between indicator variables in terms of factors. Conditional on the factors, the indicators should be independent. You can tweak the model to add correlations between the indicators if you feel you need them, but these are often a counsel of desperation employed when in truth there are no latent factors.
Exceptions exist. Suppose the indicators are something measured at successive time points. I can believe they might all depend on some underlying factors, and yet influence each other through a learning effect in the subject, say. In that case, it would make sense to have covariances in the indicators and covariances in the factors. But in general, I would want to see a solid, subject matter reason for introducing covariances at the indicator level.
Recommended texts
Sadly, I have yet to find a text on SEM that I really like. I think that Bollen's book, is the classic in this field, but it's pricey and I don't own a copy. I recently ordered Beaujean's book, which is focused on using lavaan in R. My copy hasn't arrived yet, so I don't know if it goes into the detail you need about path analysis. If you have access to a good library, or a lot of cash, I would check out Bollen.
A: If you don't include the covariances between latent variables then cross loadings will be biased. That's because cross loadings fit the correlations between measurement items that measure the two latent variables, and so does the the factor covariance. If you mis-specify one of them you will affect the other. From there direct and indirect effects can be biased as well. If you don't have cross loadings you are fine - most everything will be fine due to some conditional model expressions. 
