0
$\begingroup$

I'm using cluster analysis to examine profiles of three variables, X1, X2, and X3.

Because the mean and variance are very different between the three variables, I am considering standardizing them to have M = 0 and a SD = 1.

To provide a bit more information, there are 100 observations of individuals in total, with 10 observations per individual.

There are a few ways to standardize the variables. One is to use the grand mean for each of the three variables (X1, X2, and X3). Another that is somewhat common in "person-centered" or "individual-centered" analyses is to use the group mean, where the group consists of the observations for each individual.

In my present case, neither grand mean or the group mean centering seems appropriate, so I was wondering whether there cases similar or very different from the present in which a mean of the grand and group mean were used.

The goal of this is to take account of the group means, so the standardized values would account for individuals with higher values on variables for some observations (relative to their other observations), but would also account for how similar the scores are to the grand mean.

So, for example, if the grand mean for X1 were equal to 3, and the mean for a group were 3.5, each of the observations for X1 would be centered around 3.25.

The same would be done for the standard deviation for X1 as well as the same process for the mean and standard deviation for the other variables.

Would using a combination of grand mean and group mean centering to standardize variables be a viable approach?

$\endgroup$
  • $\begingroup$ Can you provide more detail on why do you want to center your values? What do you want to do? Why do you need the values to be centered for cluster analysis? What kind of model or algorithm are you using for your analysis? $\endgroup$ – Tim Mar 8 '16 at 15:27
  • $\begingroup$ @Tim I'm trying to use repeated measurements of three variables to examine profiles (i.e., clusters) of students' engagement in different classroom activities. Grand and group mean centering both offer benefits, but I'm trying to account for the limitations of both (grand: students with lower engagement across all three variables may be in a lower engagement profile / cluster across all observations even though they may be much more engaged in some activities: group: observations from students with very engagement across all three variables may be classified as engaged when truly they're not). $\endgroup$ – Joshua Rosenberg Mar 8 '16 at 17:18
  • $\begingroup$ Correct me if I'm wrong, but when saying about "clusters" you do not seem to mean cluster analysis per se, don't you? If not, why not using hierarchical regression to include all the group and individual effects in a single model? $\endgroup$ – Tim Mar 9 '16 at 9:57
  • $\begingroup$ @Tim I'm using cluster analysis (hierarchical and k-means) $\endgroup$ – Joshua Rosenberg Mar 9 '16 at 13:40
1
$\begingroup$

Put your objective first, not your equations!

Yes, you could subtract the mean, and scale. But there are so many things you could do. For example, you could multiply wveryrhing with 0 (probably not beneficial).

Therefore, step back and rethink what you want to do.

Here are two choices you overlooked:

  1. in each attribute, take the mean of each individual. Now compute the standard deviation of the means. Scale the attribute by 1/SDmean.
  2. in each attribute, take the standard deviation of each individual. Take the mean standard deviation, and scale the attribute by 1/meanSD.

Depending on the nature of your data, either 1 or 2 will be better. But this depends on your problem and data.

$\endgroup$
  • 1
    $\begingroup$ This is an interesting idea, so maybe you could go into a little bit more details about consequences of such re-scaling? $\endgroup$ – Tim Mar 11 '16 at 9:05
  • $\begingroup$ Agree, these are very interesting. Could you help me understand their impact and interpretation? $\endgroup$ – Joshua Rosenberg Mar 16 '16 at 11:16
  • $\begingroup$ I do not have a written down interpretation. But it should be easy to put these into a statistical model, e.g. every individual has a mean $\mu_i$ and all data is drawn from $X_i\sim N(\mu_i, \sigma_{\text{same}}$ as opposed to having a different error each. For rescaling, do you want to scale by $1/\sigma$ of each normal distribution, or by the standard deviation of the means instead? $\endgroup$ – Anony-Mousse Mar 16 '16 at 11:22
0
$\begingroup$

I do not see how could subtracting means be used in cluster analysis research. If you subtract grand mean from each of the variables, this would not impact the results anyhow since clustering algorithms do not care what are the actual values of datapoints in your variables. You can subtract, add, or multiply the values by any constant and the values should be the same since clustering algorithm is interested only in similarities between cases (e.g. a group of observations has both high values in X1 and X2 so they are grouped together, no matter what are the variables and what are the actual values).

Below you can see an R example of clustering with k-means after subtracting grand means. As you can see, the results are exactly the same (notice that the algorithm uses some randomization so they can differ by chance).

> table(kmeans(mtcars, centers = 4)$cluster,
        kmeans(scale(mtcars, scale = FALSE), centers = 4)$cluster)

     1  2  3  4
  1 16  0  0  0
  2  0  7  0  0
  3  0  0  5  0
  4  0  0  0  4

If you subtracted group means this will result in grouping together by similarities in their distances from group means. It is hard for me to imagine how this could lead to any interpretable results (besides of obviously grouping together cases that are close to their group average on all variables).

$\endgroup$
  • $\begingroup$ k-means is location invariant, so subtracting the same value from each instance does not affect the outcome. Linear scaling, however, does change the result, often substantially, if the axes have very different standard deviations. $\endgroup$ – Anony-Mousse Mar 10 '16 at 21:40
  • $\begingroup$ @Anony-Mousse I never thought about it, or tested it, but you may be right. The question and my answer are about subtracting means and this does not have much sense - correct me if I'm wrong? $\endgroup$ – Tim Mar 10 '16 at 21:50
  • $\begingroup$ He is talking about standardization to a standard deviation of 1, too. His question is very chaotic, but it mentions "standard deviation" again and again. $\endgroup$ – Anony-Mousse Mar 10 '16 at 21:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.