1
$\begingroup$

Suppose we have a covariance matrix $C$. Define the eigendecomposition, $C = Q^{-1} \Lambda Q$ and some other arbitrary basis $C = B^{-1} D B$.

Define $V_\text{PCA} = \text{diag}(QCQ^{-1})$ and $V_B = diag(BCB^{-1})$, and assume they're both sorted by magnitude. Define $T_\text{PCA} = \text{sum}(V_\text{PCA})$ and $T_B = \text{sum}(V_B)$. Is it possible that $V_\text{PCA}[i] / T_\text{PCA} < V_B[i] / T_B$ for $i = 0$? If so, how is this possible? Doesn't it violate the idea that PCA initially finds the basis vector with maximum variance?

I can understand how the inequality holds for $i > 0$, due to the orthogonality constraint of PCA which may not result in the "optimal" vectors being found in the variance-explained sense.

I'm trying to figure out if there's an error in my code, so please forgive the convoluted notation, but I want to be as clear as possible about what I'm seeing.

$\endgroup$
1
$\begingroup$

This is equivalent to computing the trace: $T_{PCA}=\mbox{tr}(QCQ^{-1})$ and $T_B=\mbox{tr}(BCB^{-1})$. By the rules of cyclic invariance $\mbox{tr}(ABC)=\mbox{tr}(BCA)$ (a fact you can easily derive by writing out the coefficients and summing the diagonal). So that $T_{PCA}=T_{B}=\mbox{tr}(C)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.