0
$\begingroup$

Updates to this thread: Based on Anony-Mousse's comments on my current results, there is only one big cluster in my data set. However, I think it might still be possible to reveal the clusters if I dig deepr (e.g., get rid of attributes that might be useless or distracting). Any suggestions on how to better organize the data to help the clusters arise?

I want to classify the residential areas of a city into groups based on their social-economic characteristics, including housing unit density, population density, green space area, housing price, number of schools/health centers/day care centers, etc. So that we can understand how many different groups the residential areas can be divided into? And what are their unique characteristics? My data looks likes:

enter image description here

I have 15 variables and their correlation is not strong, most around 0.1 to 0.2. I plan to do the k-mean clustering for this grouping task. To determine the number of clusters, I first did PCA analysis, and this is the cumulative proportion of components:

enter image description here

It seemed 9 components would account for >80% of cumulative propotrion. But I think 9 clusters is too many for my project. So I used the NbClust package to determine the number of clusters. It reported 3 clusters is the best based on 26 criteria.

enter image description here enter image description here

My questions: 1) It seemed that any arbitrary data sets can be classified into groups, as long as we apply k-mean clustering. Is that possible some data sets naturally don't have any structure and cannot be divided into groups (e.g., uniform distributed in all feature space and don't have any pattern)? I am concerned about whether my data set really has any structure or not, since the cumulative proportion of components is so low.

2) Is the 3 clusters recommendation given by NbClust reasonable? How can I validate this?

By the way, using plot(comp, col=k$clust, pch=16) command, the 3 components looks like:enter image description here

$\endgroup$
2
$\begingroup$

1) It seemed that any arbitrary data sets can be classified into groups, as long as we apply k-mean clustering. Is that possible some data sets naturally don't have any structure and cannot be divided into groups (e.g., uniform distributed in all feature space and don't have any pattern)? I am concerned about whether my data set really has any structure or not, since the cumulative proportion of components is so low.

Borrowing an analogy from regression modeling, you may estimate non-zero correlations, but you need to use inference to determine if these differences are significant. Or, if you were interested in prediction, you might use split-sample validation to confirm that the regression model indeed has predictive value. Clustering is exactly like that. Estimating the number of clusters as well as predicting to which clusters individuals belong is a bit difficult to calibrate. Bayesian models make it a bit easier. If you're not comfortable with that, take it as a prediction validation problem. You should consider split sample, leave-one-out, or k-fold and compare their merits and flaws.

2) Is the 3 clusters recommendation given by NbClust reasonable? How can I validate this?

It depends entirely upon your desired application. The most practical way to verify classification from PCA is to look at distributions and at least "verify" that they make sense. For instance, I recall the Netflix prize statistical modeler first applied PCA and found 2 principal components in a scree plot accounted for a substantial amount of variation. When he applied rotation and plotted the movies, he found the axes distinctly represented fantasy vs. reality and action vs. intimacy.

$\endgroup$
  • $\begingroup$ Hi AdamO, your suggestion makes lots of sense. For question 1), are you suggesting using "Bayesian Model-Based Clustering"? You also recommended can do split sample, leave-one-out, or k-fold. I agree that would also solve the problem. Do you know any library that will use these methods? $\endgroup$ – enaJ Jun 19 '15 at 18:23
2
$\begingroup$

There might be no clusters. And unfortunately, many methods will not notice, and divide your data nevertheless.

Judging from your last visualization, I'd say all the data is in one big cluster; sorry. Judging from the WCSS plot, 12 might be your best bet. But carefully visualize and analyse the result.

But if you dig deeper into your data (you have attributes with mixed scales - such data needs to be handled very carefully. Some attributes might be useless or distracting) clusters might still arise.

$\endgroup$
  • $\begingroup$ Hi Anony-Mousse, thanks for your comments. I'd love to dig deeper. My data has mixed scales, and I used scale() function to normalize it. How can I know some attributes could be useless or distracting? Could you kindly provide examples? $\endgroup$ – enaJ Jun 22 '15 at 17:23
  • 1
    $\begingroup$ If you add additional, random distributed, columns they will eventually ruin your result. $\endgroup$ – Anony-Mousse Jun 22 '15 at 19:23
  • $\begingroup$ I see. I'll play with different combination of attributes, see if any cluster pattern arise $\endgroup$ – enaJ Jun 22 '15 at 23:18
  • $\begingroup$ Hi Anony-Mousse, do you think if I add more attributes that are highly correlated with existing features would be helpful? I think that would add more similarity to distinguish some clusters from others. $\endgroup$ – enaJ Jun 22 '15 at 23:57
  • 1
    $\begingroup$ It can help, but it can also hurt. It biases the result (it puts more weight on the already existing features it correlates with!) and it does add some random deviations, too (noise). The more correlations you have, the more likely they are to hurt. $\endgroup$ – Anony-Mousse Jun 23 '15 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.