Can you use discriminant analysis to classify new observations into categories generated by a previous $k$-means clustering?

Question

After doing k-means clustering on a set of observations, I would like to construct a discriminant function so as to classify new observations into the categories I found after k-means. Is this at all a good idea? What should I be careful with?

Answer 1

The idea to get a classifier for new cases after the classes had been identified in a cluster analysis is in itself natural and sane. Discriminant analysis could be such a tool, especially after K-means clustering. However, the DA's classifier will work well if assumptions of DA hold, such as (1) continuous data with approximately multivariate normal distribution for each class, (2) n in classes not very unbalanced, (3) variances/covariances not very different within classes (otherwise, Quadratic DA remains valid technique though).

Another thing that one should keep in mind is that the classifier can assign only to the existing classes; it doesn't tell you openly "this new case is so unusual that you better create a new class for it". Still, you can draw decision to create a new class if you compare Mahalanobis or euclidean distance (or PDF function which depend on the distance) of the new case to the class it has been assigned to with the one "typical" for that class in your training sample (i.e. the sample that you had clustered).

Finally, one should be aware that K-means clustering procedure can itself be used for classification of new cases; this is the "assign cases only, don't iterate" regime. A case will be assigned to the cluster to which it is closer. This simple way of classification new cases differs from DA's classification in that it does not rely on distributional assumptions and Bayes formula. Honestly, I can't tell you at this time whether I think this or that classification approach is better and why. Maybe a keen commentator will drop a hint here.

Answer 2

Cluster analysis is a technique for constructing groups based on data appearing to be segregated in separate regions. It may have nothing to do with belonging to different groups. Discriminant analysis is a supervised learning method where the data are identified as belonging to distinct separate groups whether or not they cluster. If the groups cluster well then classification will work well. If you simply create categories on the basis of how they cluster there is no particular meaning to the categories and if nicely separated clusters are formed you are guaranteed good discrimination. What would be your motivation for doing this?