2
$\begingroup$

Currently, I am working on customer segmentation using their purchase data. I plan to use clustering techniques.

So, my data has below info for each customer (9 features and 1 id field)

enter image description here

Now I am following three approaches

Approach 1

Cluster using all 9 features

Approach 2

Cluster using 6 important features that were selected through some Feature selection technique

Approach 3

Cluster using only 3 features (ex: recency, frequency and monetary) based on business understanding.

My question is as follows

a) What difference does it make whether I follow Approach 1, 2 or 3? Because, I can always link back the rest of features(that were not selected) and interpret them as characteristics of a customer from that specific cluster. So, why to spend resources and do extensive feature selection?

b) Is there any one approach that is superior over others? or indicates am doing clustering the right way? I understand features and results could depend on dataset. But am unable to understand which approach is the best? Keeping the dataset aside, how would you approach a clustering problem. Would you try all 3 approaches?

c) Since it is unsupervised problem, we want the algo to tell us what are different segments found in our data. We don't wish to influence/bias the segments with our own criteria (like Approach 3). If we are going to segment based on our own list of features, then I guess there is no need of ML.

Any advice on why do we do feature selection when we can easily link back features to a cluster and interpret their characteristics?

$\endgroup$

1 Answer 1

2
$\begingroup$

a) What difference does it make whether I follow Approach 1, 2 or 3?

You can do any post analysis using the remaining features, comparing clusters with each other etc, but still, your cluster assignments will be different across approaches. Because, the features available to your clustering algorithm are different. The characteristics of an average customer in a cluster will change based on the algorithm, hyper-parameters, your features etc.

b) Is there any one approach that is superior over others?

No, not in general. But, if assigned randomly, customer id is not useful for any task. Besides, clustering is an unsupervised task. Assessing which approach is better depends on your business case.

c) ... If we are going to segment based on our own list of features, then I guess there is no need of ML.

Clustering is not about specifying which features to look at. You don't know if those features (chosen or not) form up meaningful cluster using suitable metrics. That's where different clustering methods come into play, and it is for you to explore.

$\endgroup$
4
  • $\begingroup$ Succint response. One quick question though. I see that feature selection metgods are mostly available only for supervised algos. For unsupervised algos, how do i go about selecting features to cluster my observations? $\endgroup$
    – The Great
    Commented May 27, 2022 at 13:32
  • $\begingroup$ since, we have a business objective to achieve (identify high value, moderate value, low values), then should I start with revenue variable and recency variable based on business expertise? But this defeats the purpose of AI like you mentioned abive (chosen features or not). So, then how do I start? Or if you were me, how would you approach this problem? $\endgroup$
    – The Great
    Commented May 27, 2022 at 13:35
  • $\begingroup$ And do we have to sequentially add or remove features and assess our cluster silhoutte score? Meaning, we have to do featire selection manually for clustering? Meaning, using for loop where we check for each feature's impact on silhouette score? $\endgroup$
    – The Great
    Commented May 27, 2022 at 14:20
  • $\begingroup$ You should not expect AI algorithms to always select best features for you. If you have a business expertise, you should apply it. This doesn't defeat the purpose of AI, which is also in choosing how best to utilize these features. Sequentially adding or removing features based on an assessment criterion like silhoutte score can be done. But, it's not set in stone because of the unsupervised nature of the problem. $\endgroup$
    – gunes
    Commented May 27, 2022 at 21:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.