I am looking for a method of clustering data that are close to the corners of an $N$-dimensional parallelepiped (but I don't know the vectors spanning it). Is there a good method for finding hyperplanes that are parallel to the surfaces of parallelepiped which could be used for labeling the data such that they fall into $2^N$ different categories?

So far, I tried to use k-means clustering for finding a first hyperplane that splits the data into two sets and reduces the problem from $N$ to $N-1$ dimensions, enabling a recursive solution. I am, however, not sure that this approach works well for larger values of $N$ (I’m mostly interested in problems where $2\leq N \leq 50$).

Not being an expert in unsupervised learning, here’s why I am interested in this problem: I want to assign the quantum state of a collection of $N$ atoms each of which is either in a fluorescing or non-fluorescing state. The atoms' fluorescence is imaged onto a CCD using an area of about 1000 pixels; each atom has a certain probability of being in either of the two states. The position of the atoms is fixed and the task is to assign the quantum states given a set of $M$ pictures.

By calculating and diagonalizing a pixel correlation matrix, I can obtain eigenpictures. The data contained in each picture can be reduced to an $N$-dimensional vector by projecting the recorded picture onto the $N$ eigenpictures with the biggest eigenvalues. As the measured fluorescence is additive, these $N$-dimensional vectors cluster around the corners of a parallelepiped as mentioned above.

I’d be grateful for suggestions how to carry out the required clustering or how to calculate the $N$ normal vectors of the faces of the parallelepiped.

  • 2
    $\begingroup$ Why not just decide that the corners are clusters a-priori, & automatically assign patterns to those categories? $\endgroup$ – gung - Reinstate Monica Feb 7 '18 at 16:13
  • 2
    $\begingroup$ A trivial comment is that (if I understand you correctly) there are $2^N$ classes already defined, perhaps after some rounding. So there is a gradation from $2^2 = 4$ classes already defined, so why do you need to classify formally at all, through to $2^{50} \sim 10^{15}$ which would be far too many classes to think about, although I guess you don't have that many data points. In between, is there some physics that would influence what clustering makes most sense? $\endgroup$ – Nick Cox Feb 7 '18 at 16:13
  • $\begingroup$ "how to calculate the N normal vectors of the faces of the parallelepiped" a parallelepiped is just the image of a linear transformation on a hypercube. Determine the transformation and apply it to the normal vectors of the hypercube, which are of course the standard basis vectors. $\endgroup$ – Paul Feb 7 '18 at 16:16
  • $\begingroup$ @NickCox: I have something like 1000 points in an N-dimensional space. I'd be happy for a method to determine the corners of the parallelepiped that best fits the data (the data scatter with a variance that is not the same for the different corners). My goal is to assign an N-bit string to each picture I have. $\endgroup$ – Christian Feb 7 '18 at 16:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.