I have data with points in 10 - dimensional space where the points are a superposition of a couple of different basis functions, that are unknown and I am trying to find them. However, I suspect that, for most of the points, each point is only explained by one of the basis functions.
Currently, I am doing k-means clustering first and then doing PCA. When I apply PCA to one of the clusters alone, I usually get that 95-99% of the data is explained by the first two basis functions (principal components), and around 60% of the data is explained by the first basis function. What I am looking for is some clustering algorithm that would have like 90% of the variance explained by the first basis function.
Mathematically, I want to cluster my data such that all but one of the eigenvalues of each clusters covariance matrix are close to 0 (one eigenvalue explains all the data).
I think this is a problem that can be solved by spectral clustering but none of the papers I have seen explicitly address that. It would be very helpful if someone pointed out a paper along this line.
Edit: I don't think this is the same thing as finding clusters that fit to a 10D straight line, although that would be the case if there was only 1 PC that explained 99% of the variance. What I am looking is for a method that in general maximizes a few eigenvalues at the expense of the majority. Thus for example, if I typically got 4 significant PCs in a cluster by k-means, I would get 2 significant PCs in a cluster by the method I am looking for. In other words, the objective function would be to maximize the first few eigenvalues.