I have the following question: Can the response variable ever be used in a clustering algorithm?
I understand that in general, clustering is considered to be an "unsupervised learning algorithm" which is meant to be used when the response variable is not present. But in terms of statistical methodology, is there anything wrong with running a clustering algorithm (e.g. K-Means) on some data that has a response variable? In theory - could it even be considered more beneficial to run clustering on data that has a response variable, seeing as you now have more information about the data?
I illustrate this example below (using the R programming language). I generated the following data (where "age" is considered the response variable, and "height" and "weight" are the covariates):
set.seed(123) height = rnorm(1000, 6,0.5) weight = rnorm(1000, 200, 5) age = rnorm(1000,30,1) my_data1 = data.frame(height, weight, age) height = rnorm(1000, 5.5,0.5) weight = rnorm(1000, 150, 5) age = rnorm(1000,20,1) my_data2 = data.frame(height, weight, age) final_data = rbind(my_data1, my_data2) head(final_data) height weight age 1 5.719762 195.0210 29.48840 2 5.884911 194.8002 30.23694 3 6.779354 199.9101 29.45841 4 6.035254 199.3391 31.21923 5 6.064644 187.2533 30.17414 6 6.857532 205.2029 29.38473
Next, I run clustering algorithms with and without the response variable:
1) Without the Response Variable
library(ggplot2) cls <- kmeans(x = final_data[, 1:2], centers = 2) final_data$cluster = as.factor(cls$cluster) ggplot() + geom_point(data = final_data, mapping = aes(x = height, y = weight, colour = cluster)) + ggtitle("Clustering Without the Response Variable")
2) Clustering With the Response Variable
library(plotly) library(dplyr) cls <- kmeans(x = final_data, centers = 2) final_data$cluster = as.factor(cls$cluster) fig <- plot_ly(final_data, x = ~height, y = ~weight, z = ~age, color = ~cluster, colors = c('#BF382A', '#0C4B8E')) fig <- fig %>% add_markers() fig <- fig %>% layout(scene = list(xaxis = list(title = 'Height'), yaxis = list(title = 'Weight'), zaxis = list(title = 'Age'))) fig %>% layout(title = 'Clustering With the Response Variable ')
Question: In this example, the clustering performed on the artificially generated data does not seem to be affected by using the response variable compared to omitting the response variable. But in general, if the response variable is available - is there anything fundamentally wrong with using the response variable in a clustering algorithm? Could there be any benefits, seeing as you have additional information at your disposal?