I understand that with a K-means or DTW algorithm one can cluster time series using a distance criterion, i also understand that with a K-NN algorithm for example one can do pattern recognition and identify a trajectory knowing some of its observation by comparing it to a known trajectory...however i dont see how clustering before classifying help in the classification of future trajectories?


1 Answer 1


Clustering and classification are very different tasks.

Some textbooks try to sell you clustering as "unsupervised classification" but that is rather misleading. When you can do classification, it usually never makes sense to do clustering.

Clustering is about discovering structure in your data. If you have labels, this kind of structure is already given. Furthermore clustering will return 'bad' results very often, and you may need to run it many times to get one interesting result (don't rely on a metric to tell you what is interesting - e.g. silhouette does not measure interestingness).

What can sometimes - carefully - be used is vector quantization. When doing e.g. k-means with a large k, the task no longer is a structure discovery (clustering) but a data reduction task. It's not an error if two "clusters" represent the same part of the data, then.

The KNN classifier is rather expensive (if you do not have an index). Then data reduction using e.g. k-means is an option to speed up your classifier. I would run k-means separately on each class, with $k=\sqrt{n_c}$. Then forget the cluster number, but treat each center as a point of the original class. Unfortunately, this is not broadly applicable. K-means works only for Euclidean distance and Bregman divergences. Often other distance functions work much much beter in KNN classification. Other (better) clustering methods usually won't give you representatives, and will produce too specific clusters, or be too expensive themselves, to be useful to "pimp" a KNN classifer.

  • $\begingroup$ Thanks, I can not up vote your answer for now due to my level on this site, but it is definitely clear, so thanks for that....I have posted another question following your explanation, please feel free to enlighten me on that one too, It will bre greatly appreciated. stats.stackexchange.com/questions/202862/… $\endgroup$
    – Abbas
    Mar 21, 2016 at 19:34
  • $\begingroup$ Just realized something clustering can help classification by building a priori probabilities for a naive bayes classifier, but it can also help in hidden markov model in building transition probabilities matrix by using the variations between two observations of the mean of the clusters as states (eg: fast growth, low growth, decrease, fast decrease, etc.) $\endgroup$
    – Abbas
    Mar 24, 2016 at 0:01
  • $\begingroup$ Of course you can sometimes use the mean, but will it be better than doing "unclustered" naive bayes? Computing the priors is just counting; and you do get the best estimate if you use all data, not clusters. $\endgroup$ Mar 24, 2016 at 6:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.