1
$\begingroup$

I am new to dimensionality reduction and I am trying to learn different techniques about it. I am noticing that, unlike PCA, many other algorithms do not provide the explained variance of each feature (or am I wrong?). That said, how do I choose the size of the reduced feature?

I would appreciate also some reference to read about the state-of-the-art algorithms for nonlinear dimensionality reduction.

$\endgroup$

1 Answer 1

2
$\begingroup$

Suppose your input is $X_{m \times d}$, where $m$ is the number of data points and $d$ is the dimension of each point. The total variance is a property of your input $X$; call it $v(X)$.

Now, suppose the reconstructed input after applying dimensionality reduction is $\tilde{X}$ (of the same dimensions), it has a variance $v(\tilde{X})$.

Then the fraction of variance explained is simply: $v(\tilde{X})/v(X)$.

This doesn't depend on how you implement the dimensionality reduction. For PCA, the explained variance can be computed directly from the eigenvalues of the covariance matrix. For other non-linear techniques, you can compute the reconstruction and then compute the explained variance from its intuitive definition.

$\endgroup$
3
  • $\begingroup$ When I try it with PCA I get completely different results. In my case I have 14 features and using the eigenvalues I get that the first 5 are sufficient to explain all the variance. Doing what you suggested I get that only the first 2 are needed. $\endgroup$
    – Simon
    Commented Sep 1, 2015 at 10:14
  • $\begingroup$ Could you post your matrix (if it's small), its eigenvalues and the list of explained_variance_ratio output by your PCA algorithm? $\endgroup$
    – Vimal
    Commented Sep 1, 2015 at 16:51
  • $\begingroup$ My bad. I forgot that Matlab automatically centers and standardize the input data. Doing as you said returns the same result as using the eigenvalues. Thanks! $\endgroup$
    – Simon
    Commented Sep 1, 2015 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.