Is it possible to have a basis for a covariance matrix such that the greatest variance is greater than the variance of the first eigenvector? - Cross Validated most recent 30 from stats.stackexchange.com 2019-07-17T08:43:21Z https://stats.stackexchange.com/feeds/question/347646 http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://stats.stackexchange.com/q/347646 1 Is it possible to have a basis for a covariance matrix such that the greatest variance is greater than the variance of the first eigenvector? workwork https://stats.stackexchange.com/users/209272 2018-05-22T20:26:29Z 2018-05-22T22:24:13Z <p>Suppose we have a covariance matrix $C$. Define the eigendecomposition, $C = Q^{-1} \Lambda Q$ and some other arbitrary basis $C = B^{-1} D B$.</p> <p>Define $V_\text{PCA} = \text{diag}(QCQ^{-1})$ and $V_B = diag(BCB^{-1})$, and assume they're both sorted by magnitude. Define $T_\text{PCA} = \text{sum}(V_\text{PCA})$ and $T_B = \text{sum}(V_B)$. Is it possible that $V_\text{PCA}[i] / T_\text{PCA} &lt; V_B[i] / T_B$ for $i = 0$? If so, how is this possible? Doesn't it violate the idea that PCA initially finds the basis vector with maximum variance?</p> <p>I can understand how the inequality holds for $i &gt; 0$, due to the orthogonality constraint of PCA which may not result in the "optimal" vectors being found in the variance-explained sense.</p> <p>I'm trying to figure out if there's an error in my code, so please forgive the convoluted notation, but I want to be as clear as possible about what I'm seeing.</p> https://stats.stackexchange.com/questions/347646/-/347660#347660 1 Answer by Alex R. for Is it possible to have a basis for a covariance matrix such that the greatest variance is greater than the variance of the first eigenvector? Alex R. https://stats.stackexchange.com/users/61092 2018-05-22T22:24:13Z 2018-05-22T22:24:13Z <p>This is equivalent to computing the trace: $T_{PCA}=\mbox{tr}(QCQ^{-1})$ and $T_B=\mbox{tr}(BCB^{-1})$. By the rules of cyclic invariance $\mbox{tr}(ABC)=\mbox{tr}(BCA)$ (a fact you can easily derive by writing out the coefficients and summing the diagonal). So that $T_{PCA}=T_{B}=\mbox{tr}(C)$.</p>