0
$\begingroup$

I am currently working on a data set with 14 predictors, 4 of which are categorical (city, state, year, population) or at least not used in the numerical process, while the other 10 are doubles that talk about the crime in those areas. My hope is to use Kmeans clustering to detect any crime patterns in a city and year. However my question is, how do I see patterns via a Kmeans clustering algorithm when the results are from 10 predictors forming the clusters? What ways can I detect patterns (like if crime went up and population went down for example) in these clusters. Also, how do I even visualize this type of information, just a plot of the clusters of cities with similar patterns as detected by the Kmeans? I am using MATLAB and my profress this far is found in the following link:

https://github.com/conradbm/data_science/blob/master/fbi_crime_1980_2014/data_manipulations.m

Quite frankly I'm just confused on how the clustering will help me make sense of patterns and how I can visualize them. I was planning on resourcing the following link for the Kmeans function:

https://www.mathworks.com/help/stats/kmeans.html

$\endgroup$
1
  • $\begingroup$ Basically, k-means doesn't do what you seem to want to do. I think you're barking up the wrong tree. It sounds more like time series methods are what you might need. $\endgroup$ Dec 28, 2016 at 0:41

1 Answer 1

1
$\begingroup$

Firstly, k-means cluster analysis is mostly for continuously-scaled features (i.e. "means" or averages), since categorical data don't have averages. In addition, regarding probability distributions, the normal and standard normal distributions have 2 parameters: mean and variance -- hence, you commonly find averages (means) associated with analyses involving continuous data only. Categorical variables are straightforwardly nominally-scaled, for which there is no mathematical difference between groups (chevy, dodge, ford trucks; or red, green, blue without reference to 0-255 RGB bytes) -- and therefore averages are not determined. Ordinal is another type of categorical variable.

The fastest way to understand K-means is to envision a centroid (set of averages) for each cluster. If you have 10 continuous variables, and $K=5$, then there are 5 centroids, each of which include a set of 5 averages, and for each cluster, the averages of the 5 features is based on the objects whose feature values for those 5 features are closest to a given centroid.

Here's how K-means works. Assume a classroom with 25 students. At first, randomly assign the students to stand in each of the four corners of the room. Next, ask students to look around the room and migrate to a corner for which hair color, eye color, belt color, and pants/dress color is similar. Then, after this first step, repeat migration to a corner for which students match better and there is more purity among hair color, eye color, belt color, and pants/dress color within the corner. Keep doing this until no student needs to migrate to another corner for a better match. When done, there will be $K=4$ clusters, and the cluster a student is in is the "assigned" cluster, for which there is greater similarity among feature values.

Keep in mind for the above example, if there are only two extremes for hair color, eye color, belt color, and pants/dress color, then only two corners would be needed, i.e. $K=2$.

A major issue for KM cluster analysis is that the number of clusters (centroids) $K$ has to be set, so the clusters that are discernable in the data (based on centroids, where each $k$th cluster has its own set of means for the $p$-features involved) depends on $K$.

Since you're using MATLAB, look into their "cluster validity" methods, for which the optimal number (i.e. $K$) can be identified. The silhouette index is one of the better metrics, and for a single dataset, there will be a silhouette index value for each value of $K$ (number of clusters) which is preset by the user prior to the analysis.

The way to see the results when done is to plot the object-to-centroid distances (Euclidean) for the cluster to which each object is assigned, once the optimal value of $K$ has been found via cluster validity.

$\endgroup$
3
  • $\begingroup$ Thanks for the input. So this silhouette distance is a metric that tells me how close each observation is to it's cluster's (Kth) centroid? That was the only confusing part to your post. From 'cluster validity' down to Euclidean distance. So by visualizing the Euclidean (object-to-centroid it is associated with), what does this tell me? $\endgroup$
    – bmc
    Dec 24, 2016 at 23:18
  • $\begingroup$ Silhouette index is greatest when the optimal value of $K$ is used. It depends on the data. Make a plot of silhouette vs $K$ and the peak value is the optimal $K$. It also helps if you transform the continuous feature values into percentiles first (just for the cluster validity). $\endgroup$
    – user32398
    Dec 25, 2016 at 1:38
  • $\begingroup$ What is the benefit of transforming the continuous feature values into percentiles first? And when you say percentiles what exactly do you mean? $\endgroup$
    – bmc
    Dec 25, 2016 at 22:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.