Can we proof that two principal component scores are not equal or one is better than the other in terms of variation? For example, the 1st principal component score is defined as $z_1 = a_{11}(y_1 - \mu_1) +...+ a_{1p}(y_p - \mu_p)$ with the eigenvalue $\lambda_1$. We all know that the proportion of total population variance due to $j^{th}$principal component is determined by its eigenvalue, such as $w_j = \frac{\lambda_j}{\lambda_1 + ... +\lambda_p}$. If we calculate a new score such as $s = z_1\times w_1 + ... + z_p\times w_p$. In addition, we also know that,

Var$(z_1) = \lambda_1$, Cov$(z_i, z_j) = 0$, total population variance = $\lambda_1 + ... + \lambda_p$, and $\Sigma{w_p} =1$.

My question is can we proof that Var$(s) = \lambda_1\times {w_1}^2 + ... + \lambda_p\times {w_p}^2 \geq$ or $\leq \lambda_1$?

Many thanks in advance.

  • 1
    $\begingroup$ I'm not sure why you would care about whether or not the eigenvalues are equal. There is a lot of research on identifying what are the most important eigenvalues. For instance, the literature on Random Matrix Theory looks at the eigenvalues of the covariance matrix as following the Marcenko-Pastur distribution and frequency identifies the ones beyond what random matrices' eigenvalues would look like as the significant ones. $\endgroup$ – John Jul 5 '12 at 21:13

${\rm Var}(s)\leq \lambda_1$. To see this, note that we have $${\rm Var}(s)=\frac{\sum_{i=1}^p \lambda_i^3}{(\lambda_1+\cdots+\lambda_p)^2}=\lambda_1\cdot\frac{\lambda_1^2+\sum_{i=2}^p\frac{\lambda_i^3}{\lambda_1}}{(\lambda_1+\cdots+\lambda_p)^2}.$$ But $$(\lambda_1+\cdots+\lambda_p)^2\geq \lambda_1^2+\cdots+\lambda_p^2\geq \lambda_1^2+\sum_{i=2}^p\frac{\lambda_i^3}{\lambda_1}$$ since $\lambda_1\geq\lambda_i$ for $i=2,3,\ldots,p$.

Thus $$\frac{\lambda_1^2+\sum_{i=2}^p\frac{\lambda_i^3}{\lambda_1}}{(\lambda_1+\cdots+\lambda_p)^2}\leq 1$$ and $${\rm Var}(s)=\lambda_1\cdot\frac{\lambda_1^2+\sum_{i=2}^p\frac{\lambda_i^3}{\lambda_1}}{(\lambda_1+\cdots+\lambda_p)^2}\leq \lambda_1\cdot 1=\lambda_1.$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.