The idea behind SEM and confirmatory factor analysis is to explain the covariances between the observed variables in terms of a smaller number of underlying factors -- unseen but presumed to exist. You could fit a model with uncorrelated factors, but allowing the factors to have correlations gives you extra parameters to improve the fit. Unless you have strong, theoretical reasons for positing independence, it makes sense to include the covariances. Otherwise, the model won't fit.
If factors X and Y are correlated (double arrow) and each impacts Z, there is no way to distinguish between the direct effect of X on Z and the effect of Y on Z through X. The effect exists, but it will be confounded with the direct effect of X on Z. However if X depends on Y (single arrow) and Z depends on X (single arrow), the correlation of Z and Y will increase if the correlation of X and Y increases.
The difference between covariances between latent factors and covariances between indicator variables (Observed) should matter to you. The whole point of SEM is to explain covariances between indicator variables in terms of factors. Conditional on the factors, the indicators should be independent. You can tweak the model to add correlations between the indicators if you feel you need them, but these are often a counsel of desperation employed when in truth there are no latent factors.
Exceptions exist. Suppose the indicators are something measured at successive time points. I can believe they might all depend on some underlying factors, and yet influence each other through a learning effect in the subject, say. In that case, it would make sense to have covariances in the indicators and covariances in the factors. But in general, I would want to see a solid, subject matter reason for introducing covariances at the indicator level.
Sadly, I have yet to find a text on SEM that I really like. I think that Bollen's book, is the classic in this field, but it's pricey and I don't own a copy. I recently ordered Beaujean's book, which is focused on using lavaan in R. My copy hasn't arrived yet, so I don't know if it goes into the detail you need about path analysis. If you have access to a good library, or a lot of cash, I would check out Bollen.