5
$\begingroup$

I've generally assumed that shared variance and common variance were the same thing. However, here it is written that "Common variance is the realm of total collinearity. On the other hand, the term "shared variance" is not quite defined". I'm not sure of the distinction being made here and would welcome clarification.

I've regarded covariance as being a raw form of shared variance/common variance. Then, correlation is a standardized form of shared variance/common variance.

I started thinking about this when reading the answer to this question, but I think it has more general applicability.

$\endgroup$
3
$\begingroup$

Because the 1st link in your question sets against my comments in that other thread I'm just repeating here what some of the comments there said. I think that your question is chiefly about terminology, which might be used differently by different people.

I believe that "shared variance" between two variables could be identified as their covariance. Texts/books often define correlation coefficient to be "shared variance divided by combined [or hybrid] variance". But "common variance" is a well established term coming from factor analysis. It is that collinearity variance due to common factor(s) which make the variables correlate and thus funds the covariance.

Note that covariance is the inside affair between the variables, but common variance is the variance of (and given by) the third party, a factor (another, latent variable).

Covariance is depicted just below. Two correlated variables $X_1$ and $X_2$ are vectors displayed in the reduced subject space, and because we assume the variables were centered their vectors length squared are their variances, while the cosine of the angle between them is their correlation: $\sigma_1^2=|X_1|^2$, $\sigma_2^2=|X_2|^2$, $r_{12}=\cos \phi$.

enter image description here

Orthogonal projection of $X_2$ on $X_1$, labeled here as $X_2'$, is the linear prediction of $X_2$ by $X_1$ (simple linear regression) and its variance $\sigma_2'^2=|X_2'|^2$. Symmetrically the prediction of $X_1$ by $X_2$ is $X_1'$, with variance $\sigma_1'^2=|X_1'|^2$. Then the covariance or "shared variance" is

$\sigma_{12}^2= \sigma_1\sigma_2 r_{12}= \sigma_1\sigma_2' = \sigma_1'\sigma_2$.

Covariance is the predicted variability amplified (multiplied) by the variability of the predictor.

Common variance is portrayed on the following picture; it is brought from here where it was explained in detail.

enter image description here

Instead of just 2 variables $X_1$, $X_2$, we have also latent common factor $F$. This factor partly is the generator of both $X_1$ and $X_2$ and thus is the cause of their correlatedness. The remaining, not generated by $F$ portions of the variables are their latent unique factors $U_1$ and $U_2$. Unique factors are independent of each other and of the common factor and they are what attenuate the absolute collinear correlation (covariation) to its observed magnitude.

The $F$'s variance or common variance is (Pythagorean):

$\sigma_F^2= a_1^2+a_2^2 = (\sigma_1^2-u_1^2) + (\sigma_2^2-u_2^2) = (\sigma_1^2+\sigma_2^2)-(u_1^2+u_2^2)$,

where $a$s are the factor loadings, the covariances between the factor and the variables. And, according to factor theorem,

$\sigma_{12}^2=a_1a_2$,

so you can see the difference between "shared" $\sigma_{12}^2$ and "common" $\sigma_F^2$ variances.

By "common variance" we sometimes may mean not the common factor variance as before but the portion of it, that can be linearly explained, predicted by the variables $X_1$ and $X_2$. I'm speaking now of the factor scores variance. Factor scores are the approximately estimated values of a factor (true factor's values aren't available). That prediction vector (and its variance is its length squared) lies in "plane X", the space of the variables - see pic at the end of this answer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.