Interpretation of PCA in relation to Clustering Analysis

Question

I have a dataset with hundreds of customers that have about 30 characteristics. One technique used to reduce dimensionality is PCA. I understand the underlying premise but I am unsure how to interpret the results for my clustering analysis (e.g. K-means algorithm).

To better ask my question, I will divide this question into smaller inquiries that leads me confused with how PCA and clustering analysis can be used for customer segmentation.

• Assumption 1: With 30 characteristics, I can have 30 Principal components? After transforming my dataset, using the elbow method I realize that the first 4 components represent ~90% of my dataset's variance.

• Q1: What do the values mean under each column ? What is PC1? Is it the equivalent of Column1 of my dataset (i.e the first feature/variable)?

• Assumption 2: When I apply a type of cluster algorithm (e.g. K-Means) over the 4 PCs, I can see about 3 different clusters. Great.

• Q2: What do these clusters represent? What are the characteristics that been used to properly segment them ? What is the x and y axes represent? How can I use the final cluster result concretely by applying it with new data, for example: New customer X is part of cluster 2(e.g. Valuable customer) based on this and that data.

Essentially, what I am trying to do is, to properly explain to a layperson how I went from this dataset to justifying that there are 3 clusters and how they can be implemented in a real world application, for example, in marketing.

Thank you, for your patience and understanding

Answer 1

I will try to answer your questions, hopefully I understand you correctly. Further I will try to omit mathematical formulas, since you ask for it to be easily understandable. Online you can easily find good mathematical explanations.

Regarding your first assumption

Assumption 1: With 30 characteristics, I can have 30 Principal components? After transforming my dataset, using the elbow method I realize that the first 4 components represent ~90% of my dataset's variance.

Correct, in you case you can have 30 principal components.

Question 1 Q1: What do the values mean under each column ? What is PC1? Is it the equivalent of Column1 of my dataset (i.e the first feature/variable)?

• I am not completely sure what you mean by the value under each column.
• PC1 is the first principal component.
• no, PC1 is the scaled eigen vector, corresponding to the dimension of the subspace of you data describing most variation. PC1 is the weights of the 30 variables that describes the variation of you data best. In interpretation, you can say, that PC1 is one new variable that describe your data best, if you wish to describe it with only one variable (generated from the 30 variables). Note some of the 30 variables may not influence PC1 much (this will be indicated by a close to 0 value). Thus you do not need these variables to describe most of the variation in you data. Your PC2 works similar. Only this is the new variable that explain the second most variation in you data.

Question 2 Assumption 2: When I apply a type of cluster algorithm (e.g. K-Means) over the 4 PCs, I can see about 3 different clusters. Great.

Q2: What do these clusters represent? What are the characteristics that been used to properly segment them ? What is the x and y axes represent? How can I use the final cluster result concretely by applying it with new data, for example: New customer X is part of cluster 2(e.g. Valuable customer) based on this and that data.

• The clusters presents the grouping of you observations based on the 4 PC you feed the model. That is you find the observations which has most similar values in the principal components.
• For the characteristics used to segment them, look op the K-mean method. In short K-means uses the mean in k groups (iterative) to identify the groups in which the observations are most similar in means, based on the input variables feed the model (in your case the PC).
• The x an y- axes represents depend on what you plot? i cannot see that from you question.
• On a new observation, you can calculate the latent variables based on the weights you get from the model. After this you can identify which group the new model is most similar to based on the 4 components. You will have to extract relevant variable from the K-means method to do this. It is likely you may be able to get a probability of a new person belonging to each cluster. This will help you to look op people that does not match "well enough" with any cluster.

Answer 2

Q1: What do the values mean under each column ? What is PC1? Is it the equivalent of Column1 of my dataset (i.e the first feature/variable)?

First, it helps to have a conceptualization of what a prinicipal component is. Let's say your customers have 6 charasteristics:

height, weight, friendliness (rated by observers), number of friends, income, years of schooling

It is obvious that these characteristics have commonalities in pairs: 1-2, 3-4, 5-6. Mathematically this means that a tall person is also heavier and has higher values in both characteristics and so forth for the other pairs. In other words, even though you have 6 characteristics, these can be abstrtacted in 3 supercharacteristics: body size, extroversion, Socioeconomic status. These 3 abstractions are your significant (as identified by variance explained, elbow method) principal components, PC1, PC2 and PC3 (the ranking is based on which explains the most variance).

PC1 is the abstracted concept that generates (or accounts for) the most variability in your data. PC2 for the second most variability and so forth

The value under the column represents where the individual stands (z-score) on the distribution of the abstracted concept, e.g. someone tall and heavy would have a +2 z-score on PC1 (body size).

Q2: What do these clusters represent? What are the characteristics that been used to properly segment them ? What is the x and y axes represent? How can I use the final cluster result concretely by applying it with new data, for example: New customer X is part of cluster 2(e.g. Valuable customer) based on this and that data.

The clusters represent a tendency of some of your data points (customers) to have commonalities and thus clump together in Euclidean space. For instance, in our example assume in your customers you have programmers and business people. Former would form a cloud high in income and low in extroversion (stereotype, I know), latter would form a cloud high income high extraversion.

The characteristics for segmentation are the ones you provided in your code. I am assuming you are following code from a book and therefore they should be the significant principal components. But it is impossible to know without seeing your code.

X and Y values are again impossible to know without seeing your code. Assuming you are using code from a book, they should be the first 2 principal components.

Generally the principal Components are reliable for this analysis, otherwise you run the risk of weighting some characteristics more heavily than others.

To concretely use this result, you want first of all to compare your 3 clusters for the average revenue they bring to you. You need an ANOVA comparing the 3 groups: revenue ~ cluster.

Then you need to build a Supervised Learning model (for instance, k-nearest neighbor) which takes a new data point and assigns it to one of the clusters, based on its characteristics. If it turns out that one of the clusters yields more average revenue and the new customer is assigned by the model to this cluster, then this is a potentially valuable customer.

This is all high-level, but I hope it contributes to your understanding.