Proof that if two variables (X, Y) have covariance S= $\begin{pmatrix} a & b\\ c & d \end{pmatrix}$, then proof that if c $\neq$ 0, then the first principal component is $\sqrt{\frac{c^2}{c^2+(V_1-a)^2}X}+\frac{c}{|c|}\sqrt{\frac{(V_1-a)â}{c^2+(V_1-a)^2}Y}$

Where $V_1$ is the variance explained by the first principal component.

I tried finding the eigenvalues and eigenvector of S, but couldn't, so I tried substituting $V_1$ and trying to find another solution, but I didn't get anywhere.


1 Answer 1


This is just computing the top eigenvector and eigenvalues of a 2x2 symmetric matrix and interpreting it in the context of PCA. For a generic 2x2 matrix $A := \begin{bmatrix} a & b \\ c & d\end{bmatrix}$ we can find the eigenvalues by $$ p_A(\lambda) = \det\left(A - \lambda I\right) $$ $$ = (a-\lambda)(d-\lambda) - bc \\ = \lambda^2 -\lambda (a+d) + ad - bc \\ = \lambda^2 - \operatorname{tr}(A)\lambda + \det A. $$ Setting this to zero yields $$ \lambda = \frac{\operatorname{tr}(A) \pm \sqrt{(\operatorname{tr} A)^2 - 4\det A}}{2}. $$

For PCA we'll be interested in $\lambda_1$, the largest eigenvalue.

For the eigenvectors, we need to solve the system $$ Sv = \lambda v \\ \iff (S - \lambda I)v = \mathbf 0 $$ with $v \in \mathbb R^2$ and $\lambda$ being an eigenvalue. The eigenvalues are chosen to be such that $S - \lambda I$ is singular which means without even needing to do the arithmetic this'll row reduce to $$ \begin{bmatrix} a-\lambda & b \\ 0 & 0\end{bmatrix}v = \mathbf 0 $$ since $b \neq 0$ so for an arbitrary eigenvalue I'll need to have $(a-\lambda)v_1 + bv_2 = 0$. And again because $b \neq 0$ by assumption we’ll get $$ v_2 = \frac{\lambda - a}{b} v_1. $$ I want to choose my eigenvectors to be unit vectors so $$ v_1^2 + \left(\frac{\lambda - a}{b} \right)^2v_1^2 = 1 \\ \implies v_1^2 = \left(1 + \left(\frac{\lambda - a}{b} \right)^2\right)^{-1} \\ = \frac{b^2}{b^2 + (\lambda - a)^2}. $$ Eigenvectors can be scaled so I can just take the positive square root. I'll then have $$ v_2 = \frac{\lambda - a}{b}\sqrt{\frac{b^2}{b^2 + (\lambda - a)^2}}. $$

After some rearranging and using $\lambda=\lambda_1$ we have everything we need to finish the result so I'll leave it here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.