# How to cluster data with repeated measurements?

Most clustering algorithms assume that data points in each row are independent. I have some data with repeated measurements from individuals.

I can use a standard algorithm, and then check to see if samples from the same person end up in the same cluster (for example by manual inspection of a dendrogram, or by looking at within group homogeneity and stability measures using clValid's "biological" validation).

Are there any clustering algorithms (preferably with an implementation in R) that take account of the repeated measurements while calculating clusters?

Bonus features:
My dataset is very wide (more variables than samples), so being able to deal with that situation would be very useful.
Also, there are different numbers of measurements for individuals, so it would also be nice for the algorithms to deal with that.
The variables in my dataset are continuous rather than categorical.

• Have you considered separately sampling within each subject and then sample the subjects as a whole? – John Oct 23 '14 at 14:30

If I were doing this, and I had enough samples per individual then I would approach it as follows:

• I would fit a Gaussian Mixture Model (GMM) to each person (use good model selection criteria and prove it is a good basis). I have used this to compress measures from 1000 dimensions toward 20 without loss of information.
• I would then create a space comprised of parameters of the GMM's. One axis would be the means, one the variance, and one the weights. You could sort the GMM components by weight and make a parameter space for each, so you might have an axis for first mean, one for second, etc.
• I would then cluster in that parameter space. You could do it with GMM's if you like. This will find modes within people that are similar.

The only requirement here, however, is that you have sufficient samples and that a GMM is an appropriate basis for your per-person data.

EDIT:

Consider the slide shown here: http://www.slideshare.net/EngrStudent/using-model-parameters-for-dimensionality-reduction

• Thanks. To share a presentation, you could upload to SlideShare and include a link in your answer. – Richie Cotton Oct 24 '14 at 10:47
• Do you have a feel for how many constitutes "sufficient samples"? In my case it's 3 or 4 repeats per person. – Richie Cotton Oct 24 '14 at 10:49
• For me - I'm a fan of the 30-300 samples per unit. If I only had 3-4 then I would fit a single normal distribution - and operate in the mean and variance space. Can you tell me ballpark rows (unique samples) versus columns (measures and repeat measures of the sample) your data has? What can you say about the nature of each column? – EngrStudent Oct 24 '14 at 11:48
• There are 67 rows, with samples on 17 people. There are almost 1200 columns, all positive numeric values. – Richie Cotton Oct 24 '14 at 17:08
• I'm more of a 50k columns, 15k rows guy so 67 rows is a fun challenge. So what can you tell me about the columns? Do you know anything about how they interact? My first thought is "data whitening" (projectrhea.org/rhea/index.php/…). My second thought is PCA using Singular Value Decomposition. My third is "random evil twin" or "random forests" which can be used to determine column importance. – EngrStudent Oct 24 '14 at 17:35