I've encountered a question to de-correlate many data points sitting along two parallel lines in a 2D space, say $x=1$ and $x=-1$. And no labels are given to those data points, so supervised methods cannot be applied here. Apparently, PCA may not work here without manually carefully analyzing the data, because projecting onto the first eigenvector (having the largest eigenvalue) would merely mix all the points together. Of course, we may simply visualize those 2D data points and directly find out the direction, which is perpendicular to the first eigenvector, can solve the problem.

However, if we are given a set of data points with high dimensions which are lying along two parallel hyperplanes but we're not aware of in the beginning, it could be very troublesome to find out such information. And if again using PCA, the first eigenvector basically cannot help de-correlate the data at all. So it would be the best not to use the first eigenvector here.

My question is: Is there any effective strategy to get ride of the useless eigenvector(s), if we have to use PCA? Or, are there any other solutions, instead of PCA, to do the trick?


  • $\begingroup$ Do I understand correctly that you want a method that can discover that the unlabeled data are concentrated on two parallel (?) lines/hyperplanes, i.e. find a 1D projection onto a line (?) where the data look like two well-separated clusters? If so, then why do you call this "de-correlate"? $\endgroup$
    – amoeba
    Sep 25 '14 at 9:45
  • $\begingroup$ Hi amoeba, you are right. IMHO, what you said can be generally considered as data de-correlation. And I was interviewed by this question some time back, and the interviewer emphasized this is a data de-correlation problem. I also doubted during the interview. But after some time, I thought maybe data de-correlation is a more general concept. So I put "de-correlate" here. $\endgroup$
    – mintaka
    Sep 25 '14 at 9:56
  • $\begingroup$ Well, I don't really see how a concept of correlation can be applied to one-dimensional projection: what is it supposed to be correlated or not correlated with? But okay. So this was a job interview question? Interesting. $\endgroup$
    – amoeba
    Sep 25 '14 at 10:05
  • $\begingroup$ The 2D version was a job interview question. If we're not talking about PCA, may I know if you would have some method to de-correlate the data points? $\endgroup$
    – mintaka
    Sep 25 '14 at 14:17
  • $\begingroup$ I think I could come up with some ad hoc method that would work in 2D (probably some iterative approach akin to expectation-maximization) and other low dimensions, but I don't know any principled way to tackle this problem in arbitrary dimensions. Interesting question, +1. $\endgroup$
    – amoeba
    Sep 25 '14 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.