0
$\begingroup$

Proof that if two variables (X, Y) have covariance S= $\begin{pmatrix} a & b\\ c & d \end{pmatrix}$, then proof that if c $\neq$ 0, then the first principal component is $\sqrt{\frac{c^2}{c^2+(V_1-a)^2}X}+\frac{c}{|c|}\sqrt{\frac{(V_1-a)â}{c^2+(V_1-a)^2}Y}$

Where $V_1$ is the variance explained by the first principal component.

I tried finding the eigenvalues and eigenvector of S, but couldn't, so I tried substituting $V_1$ and trying to find another solution, but I didn't get anywhere.

$\endgroup$

1 Answer 1

1
$\begingroup$

This is just computing the top eigenvector and eigenvalues of a 2x2 symmetric matrix and interpreting it in the context of PCA. For a generic 2x2 matrix $A := \begin{bmatrix} a & b \\ c & d\end{bmatrix}$ we can find the eigenvalues by $$ p_A(\lambda) = \det\left(A - \lambda I\right) $$ $$ = (a-\lambda)(d-\lambda) - bc \\ = \lambda^2 -\lambda (a+d) + ad - bc \\ = \lambda^2 - \operatorname{tr}(A)\lambda + \det A. $$ Setting this to zero yields $$ \lambda = \frac{\operatorname{tr}(A) \pm \sqrt{(\operatorname{tr} A)^2 - 4\det A}}{2}. $$

For PCA we'll be interested in $\lambda_1$, the largest eigenvalue.


For the eigenvectors, we need to solve the system $$ Sv = \lambda v \\ \iff (S - \lambda I)v = \mathbf 0 $$ with $v \in \mathbb R^2$ and $\lambda$ being an eigenvalue. The eigenvalues are chosen to be such that $S - \lambda I$ is singular which means without even needing to do the arithmetic this'll row reduce to $$ \begin{bmatrix} a-\lambda & b \\ 0 & 0\end{bmatrix}v = \mathbf 0 $$ since $b \neq 0$ so for an arbitrary eigenvalue I'll need to have $(a-\lambda)v_1 + bv_2 = 0$. And again because $b \neq 0$ by assumption we’ll get $$ v_2 = \frac{\lambda - a}{b} v_1. $$ I want to choose my eigenvectors to be unit vectors so $$ v_1^2 + \left(\frac{\lambda - a}{b} \right)^2v_1^2 = 1 \\ \implies v_1^2 = \left(1 + \left(\frac{\lambda - a}{b} \right)^2\right)^{-1} \\ = \frac{b^2}{b^2 + (\lambda - a)^2}. $$ Eigenvectors can be scaled so I can just take the positive square root. I'll then have $$ v_2 = \frac{\lambda - a}{b}\sqrt{\frac{b^2}{b^2 + (\lambda - a)^2}}. $$

After some rearranging and using $\lambda=\lambda_1$ we have everything we need to finish the result so I'll leave it here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.