Variance explained by a set of variables (dimensionality reduction) I am interested in estimating the amount of variance that can be accounted for by a set of variables. After reading this previous post where a similar question is answered but for only one variable: See post I was wondering if this could be extended to a set of independent variables. For example, if we have a total of 6 variables, where there are two groups of highly correlated variables (lets say 4 in one group and 2 in the other), but the variables in each group are not correlated with the other variables (the 4 variables are not correlated with the other 2, and viceversa).
Would it be correct, in that case, to say that the proportion of variance explained by two variables (one from the group of 4 and one from the group of 2) is the sum of their individual variances as suggested in the linked post?
 A: No it is not correct to say the following:

Would it be correct, in that case, to say that the proportion of variance explained by two variables (one from the group of 4 and one from the group of 2) is the sum of their individual variances as suggested in the linked post?

Because you have to take into account the covariances between the two variables that you are considering and all the other variables in each respective group. In other words, you are taking one variable from each group and you know that those variables are not correlated between each other. This is fine. But the problem is that each variable is still correlated with the others in its respective group. Therefore, the contribution to the total variance also depends on:


*

*the covariance of variable 1 with the other 3 variables in the same group (the group of 4 variables)

*and the covariance of the variable 2 with the other variable of the same group (the group of 2 variables)
To rephrase again, taking two variables from a broader set of features, you can say that the sum of their variances of is equal to the sum of their contributions to the total variance of the features if each of two variables is uncorrelated with all the others features. That is why we use PCA: to make sure that all the transformed variables have 0 correlations with all the others transformed variables. This is clearly not the case, as each of the two variables is still  correlated to at least one of the other variables that you are not considering in the sum.
