There are some methods like singular value decomposition (SVD), principal component analysis (PCA), factorial analysis and many more that are used to reduce a high-dimensional dataset into fewer dimensions while retaining important information. What would happen if for instance the problem was not about finding those very informative features, but synthetizing the data in a way to assure that correlation between the resulting dimensions is minimized regardless of whether the resulting number of dimensions is low enough and capable of retaining most of the variability.

With traditional dimensionality reduction techniques such as SVD, PCA or FA you don't really assure that the resulting dimensions will be uncorrelated, I wonder if there is any solution to this problem.

  • 1
    $\begingroup$ PCA axes are orthogonal to one another, and therefore uncorrelated. Or at least that is my understanding. Can you or someone else please provide a source or example that shows otherwise. $\endgroup$ – Patrick Dec 9 '12 at 18:29
  • $\begingroup$ Yes. PCA guarantees linear independence of the resulting components, you perform it to get rid of multicollinearity and this always works because in PCA axis are perpendicular to each other. Now, I don't really need perfect independece, what I actually require is just to minimize correlation up to a certain degree. Using PCA is of course a valid alternative. I just want to clarify that I want to reduce dimesionality as a first step to then continue with a classification procedure. Sorry for confusing you. $\endgroup$ – Sebastian Hastings Dec 9 '12 at 23:14
  • $\begingroup$ What's the problem with using PCA as a first step then? $\endgroup$ – Scortchi - Reinstate Monica Dec 10 '12 at 8:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.