Assuming we have centralized data,the covariance matrix of the sample is X'X. This is Because: $$ Cov(X)=\frac{1}{n-1} \begin{pmatrix} X_1'X_1 &...&X_1'X_n \\ ...&...&...\\ X_n'X_1&...&X_n'X_n \end{pmatrix} $$ In PCA,we make an eigendecomposition $X'X=\phi \Lambda \phi'$ to turn the covariance matrix of the sample into diagonal matrix $\Lambda$ Bacause the Non-diagonal element of the covariance matrix of the sample is zero,we reduce the linear relationship between new variables (we set $Z$) without deleting some variebles

But I dont think so. If the $\Lambda$ has a very small eigenvalue $\lambda_i$,we have $Z'Z\phi_i=\Lambda\phi\approx0$

$\Rightarrow Z\phi\approx0 \Rightarrow$ The new variable is approximately linearly dependent

I don't know what's wrong. I hope you can help me.Thank you!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.