I have a data set of 40,000 individuals which I clustered using k-means. I used 30 variables, each ordinal from 1=minimum to 5=maximum. I reduced these 30 variables to 10 factors and ran K-means on these new variables. I kept a clustering of 12 clusters.
I now have 7000 new observations, I want to classify these using a discriminant function. I built one in SPSS using the 30 previous variables. I get a good clasification on the original dataset (40,000), but when I run it on the new observations, one segment doesn't show at all. 0 cases are of this type. This makes no sense to me.
The original clusters seem healthy and apart, I did them in R with multiple different initial center methods and ran them many many times, and stuck with the best one. The discriminant function has no problem finding this "disappearing" cluster in the original dataset.
What could be causing this?
