2
$\begingroup$

I have been looking for an answer/guidance/pointer to this question of mine for a while. After going through many (100s actually) posts and articles, I finally found this question, where this response is what I believe is sending me in the right direction. However, I think I need a little more help to move forward. In the response, Anony-Mousse mentioned the following:

Pixels of an image in RGB space. Least-squares makes some sense and all attributes are comparable - k-means is a good choice.

Geographic data: least-squares is not very appropriate. there will be outliers. but distance is very meaningful. Use DBSCAN if you have a lot of noise, or HAC (hierarchical agglomerative clustering) if you have very clean data.

Species observed in different habitats. Least-squares is dubious, but e. g. Jaccard similarity is meaningful. You probably have only few observations and no "false" habitats - use HAC.

He/She says "Least-square makes sense or least-square is not very appropriate" but does not mention why. Are there any books/concepts that will help me learn the process of choosing a distance metric/clustering technique and explain what characteristics of any given dataset drives distance/technique selection process.

I am currently working on creating customer segmentation on this retail dataset and I just cannot tell which distance metric/clustering I need to choose of 10+ clustering models available for me to implement.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

There is not, and will not be, a simple if-then flowchart for choosing distance metrics and clustering algorithms. Because there also is not the one right answer, this is subjective and based on human inyerpretation.

Do not put the methods first. Put your data first, it is more important.

Study and understand your data. For a set of records, what is the appropriate (for your problem!) way to quantify similarity? Put this into math equations. Then check if you can find a similar enough measure, or prove the measure properties yourself. Etc.

On that particular data, clustering supposedly is not the tool of choice, but association rule mining...

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thank You for the direction on the distance metric. That was super helpful. Regarding modeling technique, is there a starting point (books/article/concept) to develop that knowledge/intuition where you can say by looking at the data/project that "Clustering is not the tool of choice, but association rule mining is"? $\endgroup$
    – user2078017
    Commented Aug 6, 2018 at 6:58
  • $\begingroup$ Market basket is simply the prime example for FIM. $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Aug 6, 2018 at 16:40
  • $\begingroup$ Sorry, should have been more specific. I am actually looking to understand the purchase/return pattern irrespective of the product. Is clustering choice then accurate? $\endgroup$
    – user2078017
    Commented Aug 7, 2018 at 18:42
  • $\begingroup$ How would clustering be able to analyze purchase/return? Don't randomly try stuff. Formalize the task, and choose the tool to solve your task, and not something else. So what is a "purchase/return pattern"? $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Aug 7, 2018 at 19:23
  • $\begingroup$ Oh, I thought I am solving a real world problem:-). purchase/return pattern to me is an extension of RFML model where I was trying to see if there is any pattern in the way a customer buys/returns. Is a customer buying 10 times and returning 2 times or buying 50 times and returning 10. Are they regular buyer or just buy once or twice within a month or once or twice over a year. I was asked once at my job to do something similar. Perhaps clustering will not solve this problem but I don't know what will. $\endgroup$
    – user2078017
    Commented Aug 8, 2018 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.