Skip to main content
Added a clarification to the author's question that will cut down on answers that do not address the problem to be solved.
Source Link

I am building a classification model (binary outcome) and would like to include an external cluster membership code as a predictor. For training the model, this is straight forward. When "scoring" new data with the model, it is less so.

  • If the clustering was k-means and used euclidean distance, assigning the new data to its closest cluster (centroid) would be easy.
  • But, how do you do this when you are using a distance matrix as input to the clustering algorithm? That is, what if you cannot compute some sort of centroid or other "aggregate representation" of a cluster?

Note: Once the classification model is built, the labeling and meaning of the clusters must remain static.

The question is general, but perhaps made more difficult in that I am looking at clustering time series data (using the dtw package with hclust) - so it seems that one possible solution to the general question ( i.e. to use a predictive model to map new data to the existing cluster assignments) is not possible.

Any experience with this?

I am building a classification model (binary outcome) and would like to include an external cluster membership code as a predictor. For training the model, this is straight forward. When "scoring" new data with the model, it is less so.

  • If the clustering was k-means and used euclidean distance, assigning the new data to its closest cluster (centroid) would be easy.
  • But, how do you do this when you are using a distance matrix as input to the clustering algorithm?

Note: Once the classification model is built, the labeling and meaning of the clusters must remain static.

The question is general, but perhaps made more difficult in that I am looking at clustering time series data (using the dtw package with hclust) - so it seems that one possible solution to the general question ( i.e. to use a predictive model to map new data to the existing cluster assignments) is not possible.

Any experience with this?

I am building a classification model (binary outcome) and would like to include an external cluster membership code as a predictor. For training the model, this is straight forward. When "scoring" new data with the model, it is less so.

  • If the clustering was k-means and used euclidean distance, assigning the new data to its closest cluster (centroid) would be easy.
  • But, how do you do this when you are using a distance matrix as input to the clustering algorithm? That is, what if you cannot compute some sort of centroid or other "aggregate representation" of a cluster?

Note: Once the classification model is built, the labeling and meaning of the clusters must remain static.

The question is general, but perhaps made more difficult in that I am looking at clustering time series data (using the dtw package with hclust) - so it seems that one possible solution to the general question ( i.e. to use a predictive model to map new data to the existing cluster assignments) is not possible.

Any experience with this?

edited title
Link
user88
user88

Assigning New Observationsnew observations into Existing Clusters : Distance Matrixexisting clusters made from a distance matrix

Source Link
B_Miner
  • 8.9k
  • 22
  • 94
  • 154

Assigning New Observations into Existing Clusters : Distance Matrix

I am building a classification model (binary outcome) and would like to include an external cluster membership code as a predictor. For training the model, this is straight forward. When "scoring" new data with the model, it is less so.

  • If the clustering was k-means and used euclidean distance, assigning the new data to its closest cluster (centroid) would be easy.
  • But, how do you do this when you are using a distance matrix as input to the clustering algorithm?

Note: Once the classification model is built, the labeling and meaning of the clusters must remain static.

The question is general, but perhaps made more difficult in that I am looking at clustering time series data (using the dtw package with hclust) - so it seems that one possible solution to the general question ( i.e. to use a predictive model to map new data to the existing cluster assignments) is not possible.

Any experience with this?