I've encountered a question to de-correlate many data points sitting along two parallel lines in a 2D space, say $x=1$ and $x=-1$. And no labels are given to those data points, so supervised methods cannot be applied here. Apparently, PCA may not work here without manually carefully analyzing the data, because projecting onto the first eigenvector (having the largest eigenvalue) would merely mix all the points together. Of course, we may simply visualize those 2D data points and directly find out the direction, which is perpendicular to the first eigenvector, can solve the problem.
However, if we are given a set of data points with high dimensions which are lying along two parallel hyperplanes but we're not aware of in the beginning, it could be very troublesome to find out such information. And if again using PCA, the first eigenvector basically cannot help de-correlate the data at all. So it would be the best not to use the first eigenvector here.
My question is: Is there any effective strategy to get ride of the useless eigenvector(s), if we have to use PCA? Or, are there any other solutions, instead of PCA, to do the trick?