# Can you use discriminant analysis to classify new observations into categories generated by a previous $k$-means clustering?

After doing k-means clustering on a set of observations, I would like to construct a discriminant function so as to classify new observations into the categories I found after k-means. Is this at all a good idea? What should I be careful with?

-
Of possible interest: Cluster Analysis followed by Discriminant Analysis. –  chl Jul 11 '12 at 10:31

The idea to get a classifier for new cases after the classes had been identified in a cluster analysis is in itself natural and sane. Discriminant analysis could be such a tool, especially after K-means clustering. However, the DA's classifier will work well if assumptions of DA hold, such as (1) continuous data with approximately multivariate normal distribution for each class, (2) n in classes not very unbalanced, (3) variances/covariances not very different within classes (otherwise, Quadratic DA remains valid technique though).

Another thing that one should keep in mind is that the classifier can assign only to the existing classes; it doesn't tell you openly "this new case is so unusual that you better create a new class for it". Still, you can draw decision to create a new class if you compare Mahalanobis or euclidean distance (or PDF function which depend on the distance) of the new case to the class it has been assigned to with the one "typical" for that class in your training sample (i.e. the sample that you had clustered).

Finally, one should be aware that K-means clustering procedure can itself be used for classification of new cases; this is the "assign cases only, don't iterate" regime. A case will be assigned to the cluster to which it is closer. This simple way of classification new cases differs from DA's classification in that it does not rely on distributional assumptions and Bayes formula. Honestly, I can't tell you at this time whether I think this or that classification approach is better and why. Maybe a keen commentator will drop a hint here.

-
I was kind of aware of this but... I'm not sure how to measure the reliability of the classification. I can trying agasint the already classified data to see how well it goes but ... on new data? how can I know it will work well, in which cases? –  JEquihua Jul 3 '12 at 4:16
Just one general way to assess the reliability of classification is to split one big sample into one training and several test subsamples, to see how much less correctly the classifier behaves in test subsamples compared to the training one. –  ttnphns Jul 3 '12 at 5:46