0
$\begingroup$

I'm looking at performing k-means clustering on a dataset with 5 continuous variables. The clusters that I find however, look very similar except in one dimension e.g

cluster 1 : low avg income, low avg age ,high number of transactions

cluster 2: low avg income, low avg age, low number of transactions

cluster 3: high avg income, low avg age, low number of transactions

The "low avg age" for example, is different but in the same ballpark in each cluster, can I use a statistical test to test which ones are significantly different from each other?

If I wanted to compare the mean of another variable e.g. debt (£) (not used for clustering), does it make sense to use a t-test (or possibly something else) to test for significant difference in this variable between clusters?

I hope this makes sense.

Thanks!

$\endgroup$
  • $\begingroup$ You could try a Hotelling's T test (assuming that they are jointly normal) and test all the means at once. $\endgroup$ – Huy Pham Jul 24 at 13:45
  • $\begingroup$ you may work out your study's objectives first ! $\endgroup$ – Subhash C. Davar Jul 24 at 14:05
1
$\begingroup$

The problem is that k-means optimizes the sum of squares, which you'd also use in testing. This is called data snooping - I would expect the outcome to always be "significant", even if your I put data were uniform noise.

You could only test with attributes you did not use during clustering.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.