2
$\begingroup$

I already referred these posts here and here.

Currently, I am working on customer segmentation using their purchase data.

So, my data has below info for each customer

enter image description here

Based on the above linked posts I see that for clustering, we have to scale the variables if they are in different units etc.

But if I scale/normalize all of them to uniform scale, wouldn't I lose the information that actually differentiates the customers from one another? But I also understand that monetary value could construed as high weighted feature because they might go upto range of 100K or millions as well.

Let's assume that I normalized and my clustering returned 3 clusters. How do I answer below questions meaningfully?

q1) what is the average revenue from customers who are under cluster 1?

q2) what is the average recency (in days) for a customer from cluster 2?

q3) what is the average age of customer with us (tenure) under cluster 3?

Response to all the above question using normalized data wouldn't make sense because they all amight be in a unform scale mean 0, sd 1 etc. If I say average age in cluster 3 is 0.356, it wouldn't make sense to users.

So, I was wondering whether it is meaningful to do the below

a) cluster using normalized/scaled variables

b) Once clusters are identified, use customer_id under each cluster to get the original variable value (from input dataframe before normalization) and make inference or interpret clusters? Is it okay to interpret clusters this way (and not using nornalized values)

c) I believe the pattern found in nornalized data is also applicable to original data (even though we disnt feed the original data representation to the model)

So, do you think it would allow me to answer my questions in a meaningful way

Is this how data scientists interpret clusters? they always have to link back to input dataframe?

$\endgroup$

1 Answer 1

2
$\begingroup$

But if I scale/normalize all of them to uniform scale, wouldn't I lose the information that actually differentiates the customers from one another?

No, changing the scales doesn't discard information about the customer. As long as the transformation is monotonic and one-to-one, the rescaled data has exactly the same information about one customer to another customer. We want the transformation to be monotonic so that the relative ordering of the customers according to the feature is the same, and we want it to be one-to-one so that it's reversible (we can move from the original units to the scaled units and back to the original units exactly).

  1. What is the average revenue from customers who are under cluster 1?
  2. What is the average recency (in days) for a customer from cluster 2?
  3. What is the average age of customer with us (tenure) under cluster 3?

This is just an elementary database operation. One table is the customer IDs and the cluster assignments. The second table has the customer IDs and revenue/other data elements. Then do an inner join and do your aggregation by cluster assignment.

$\endgroup$
4
  • $\begingroup$ Yes, understand. My question was not really on database operation but to know whether pattern found on normalized data, also applies to raw data. Your 1st part of answer helps.. $\endgroup$
    – The Great
    Commented May 26, 2022 at 22:53
  • $\begingroup$ So, once we find clusters, you suggest that I can use the original/raw values to make inference or interpret clusters. Instead of normalized values. Did I understand that right? $\endgroup$
    – The Great
    Commented May 26, 2022 at 22:55
  • $\begingroup$ You get to decide how you want to answer your own research questions. I think most people find it easier to think about dollars than standardized dollars, such as $z$-scores of dollars relative to all customers included in the study. $\endgroup$
    – Sycorax
    Commented May 26, 2022 at 22:56
  • $\begingroup$ Understand. based on your response, I have a follow up question on this. I created a new post. Would be helpful to have your views on this - stats.stackexchange.com/questions/576745/… $\endgroup$
    – The Great
    Commented May 26, 2022 at 23:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.