Background: I want to classify the residential areas of a city into groups based on their social-economic characteristics, including housing unit density, population density, green space area, housing price, number of schools / health centers / day care centers, etc. I want to understand how many different groups the residential areas can be divided into, and what are their unique characteristics. This information could facilitate city planning.
Based on some examples (cf., this blog post: PCA and K-means Clustering of Delta Aircraft), I figured out the way to do the analysis is:
First do PCA analysis.
Determine the number of unique groups (clusters) based on PCA results (e.g., using the "elbow" method, or alternatively, the number of components that explains 80 to 90% of total variance).
After determining the number of clusters, apply k-means clustering to do the classification.
My questions: it seemed the number of PCA components is related to clusters analysis. So is that's true, if, say, we found 5 PCA components explained more than 90% of variation of all features, then we would apply k-means clustering and get 5 clusters. So would the 5 groups exactly corresponding to the 5 components in PCA analysis?
In another words, I guess my question is: What is the connection between the PCA analysis and k-means clustering?
Updates: Thanks to Emre, xeon, and Kirill's inputs. So the current answers:
Doing PCA before clustering analysis is also useful for dimensionality reduction as a feature extractor and visualize / reveal clusters.
Doing PCA after clustering can validate the clustering algorithm (reference: Kernel principal component analysis).
PCA is sometimes applied to reduce the dimensionality of the dataset prior to clustering. However, Yeung & Ruzzo (2000) showed that clustering with the PC's instead of the original variables does not necessarily improve cluster quality. In particular, the first few PC's (which contain most of the variation in the data) do not necessarily capture most of the cluster structure.
- Yeung, Ka Yee, and Walter L. Ruzzo. An empirical study on principal component analysis for clustering gene expression data. Technical report, Department of Computer Science and Engineering, University of Washington, 2000. (pdf)
It seemed PCA is necessary before a two-step clustering analysis. Based on Ibes (2015), in which cluster analysis was run using the factors identified in the PCA.
- Ibes, Dorothy C. A multi-dimensional classification and equity analysis of an urban park system: A novel methodology and case study application. Landscape and Urban Planning, Volume 137, May 2015, Pages 122–137.