How to do classification after clustering?

Question

I segmented products by using k-means clustering into 10 clusters with historic data (dispatch data). For new products, I can use some dimension and feature based data (ex: product size, color, budget v.s.). My purpose is matching new products with already existing clusters.

Is this possible?

---

Thank you for your answers.

Actually I used k-medoids for clustering my data.

As a second step, from your suggests, the best way is, using these clusters as target for classification make sense to me.

But; I have new troubles according to usage of classifiers:

• Recursive Partitioning and Regression Trees: Only used one feature. Message: Variables actually used in tree construction: [1] ProductCode
• Random Forest: Error Message: Can not handle categorical predictors with more than 53 categories.
• KNN: Data must be scaled before usage, but my data has lots of categorical features. So KNN is not suitable for my problem.
• SVM: I think this classifier wants only 2 features. Error Mesage: contrasts can be applied only to factors with 2 or more levels

So, what classifier should I use?

Answer 1

Clustering is done on unlabelled data returning a label for each datapoint. Classification requires labels.

Therefore you first cluster your data and save the resulting cluster labels. Then you train a classifier using these labels as a target variable. By saving the labels you effectively seperate the steps of clustering and classification.

This enables you to use any classification algorithm (Random Forest, SVM, Naive Bayes, ....).

The problematic part of this pipeline is the lack of robustness of the kmeans algorithm. Therefore you will have to evaluate the clustering result and possibly repetitivly perform k-means. Alternatively you could use other clustering algortihms and compare results.

Answer 2

Although it is feasible to apply classification incl training, cv and benchmarking using cluster labels obtained by clustering the same data, it should be clear that this is will be "successful" even if you have RANDOM high dimensional data. Benchmarking on the same data that you used for clustering leads to overoptimistic classifier performance estimates.

Therefore, data science purists do not like it.

Similarly, you get overoptimistic p values when you assess differences for data between clusters of some high dim data. So, be careful to use statistical testing for feature selection prior to classifier building - tests can just be regarded as heuristic procedures to prioritize features - p value not be interpreted the usual way.

Answer 3

Please be mindful about the other comments about if you have labelled data.

So you have clustered your products and found out the centroids. The straight forward way is to (preprocess your test data and to) compute the distance between your test data and centroids. From this you can assign the closest centroid label. Do it yourself, or let other methods do it for you:

x <- rbind(matrix(rnorm(100, sd = 0.3), ncol = 2),
           matrix(rnorm(100, mean = 1, sd = 0.3), ncol = 2))
xtest <- rbind(matrix(rnorm(10, sd = 0.3), ncol = 2),
           matrix(rnorm(10, mean = 1, sd = 0.3), ncol = 2))

clust2 <- flexclust::kcca(x, 2)
flexclust::clusters(clust2, xtest)

But a little bit of critique: be aware of how kmeans works. Did you standardize your data? (Otherweise you will have uneqaul weights in your distance computation). How do you handle categorical data? Etc.

Besides that, there are several other approaches to cluster analysis.