2
$\begingroup$

I'm running a simple K-Means clustering algorithm on a dataset with ~15 features. Some of the features I'm currently including in my model represent different percentages of the same whole. I'd prefer to keep the exact variables to myself, so pretend I'm clustering restaurant guests, and the variables are "percentage of foods eaten containing milk", "percentage of food eaten containing broccoli", and "percentage of food eaten containing mustard."

Because most foods contains either milk or mustard (hopefully), these variables might have distributions that look something like this:

Image of scatter matrix

Would it problematic to include variables with this relationship in a K-Means estimator? Why/why not?

If it is problematic, should I run a PCA analysis to decide which variables to drop? What other methods are used for this type of reduction?

Thanks

$\endgroup$
2
  • 2
    $\begingroup$ K-means is probably not a good choice for percentage data. $\endgroup$ Aug 9, 2016 at 3:42
  • $\begingroup$ @gung can you please elaborate? $\endgroup$ Aug 9, 2016 at 12:48

1 Answer 1

2
$\begingroup$

k-means does not care about the sign of your data. So it would be perfectly fine to go from $X$ to $-X$ to get rid of negative correlation without changing the result at all.

Correlations in k-means increase the importance of features. Going from $(X, Y)$ to $(X, Y, X)$ simply puts twice as much weight/importance on $X$ rather than $Y$. That is why you have to be very careful with data preprocessing and normalization/scaling. PCA is just a crude heuristic that sometimes works, but don't use it to avoid understanding your data - use it only if you understand your data, and understand PCA to be the right thing to use on your data.

Given the plots og your data (and percentage data types) I doubt that k-means will work, though. The data does not exhibit clear clusters. The k-means results will probably not be much better than assigning every object to $argmax x_i$ (i.e. to the maximum ingredient each, mostly sugar, mostly milk, ...)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.