I want to use k-means clustering algorithm to cluster my data into two clusters, tight as possible. I have the following questions:
- Are there any roles to choose the variables needed to cluster the data. e.g I have 100 patients who diagnosed of having a disease. I have the rate of disease progression and the disease duration since diagnosis.
The formula that calculate the rate of disease progression depends on a clinical scale for disease evaluation, and disease duration as follow :
rate of disease progression= (total clinical scale - current clinical scale) / disease duration
I aim to divide the patients into two groups (slow disease progression and fast disease progression) by feeding the rate of disease progression and disease duration as variables into k-means. Is this choice, for the variables, correct?.
In other words the rate of disease progression was calculated depending in part on disease duration. Can this affect clustering the data using two variables one derived from the other one? Or I need,just, any two or more variables that can make dividing the patients make sense?
- When clustering the data, using k-means, I need to choose the clusters tight as possible ( i.e. the data points are close as possible to the centroid of each cluster). Is it advisable to remove the outiers from the clusters. For example:
In the figure bellow, there are three clusters. the data points are concentrated in almost 90% around the centroids ( within each cluster). Do I need to remove the data points that are so far away from the centroid of the clusters ( #1 and #3).