Skip to main content
added 615 characters in body
Source Link

Indeed, distance-based approaches do not make a lot of sense on such data.

The idea of e.g. k-means is to gind groups of low variance, and that primarily makes sense on continuous data.

The popularity of these methods e.g. in marketing is probably best explained as follows:

  1. someone read that clustering is cool
  2. they loaded some data in some program that could run k-means or hierarchical clustering
  3. after a lot of fiddling, they even got a result
  4. the result didn't totally contradict their hypothesis, so they were happy and published it
  5. now everybody wants to do this.

The results don't need to be sensible, well-founded, or better than random for this to work. If you are eager enough to puvblishpublish something, qnyany method will do. Unfortunately.

If you have categoricial or binary data, a concept that makes much more sense is that of frequent patterns. But of course it's not as sexy (it boils down to counting, just as you did in your example... 1,0 is a frequent pattern). It's also less convenient, because some users may show more than one pattern, and many users will not have a typical pattern - so you don't get this nice "users of type A prefer blue jeans and spend 100$" type of nonsense that sells well as "result". And not so much black magic where you just can pretend that the clusters must be correct, because the magic algorithm found them.

Indeed, distance-based approaches do not make a lot of sense on such data.

The idea of e.g. k-means is to gind groups of low variance, and that primarily makes sense on continuous data.

The popularity of these methods e.g. in marketing is probably best explained as follows:

  1. someone read that clustering is cool
  2. they loaded some data in some program that could run k-means or hierarchical clustering
  3. after a lot of fiddling, they even got a result
  4. the result didn't totally contradict their hypothesis, so they were happy and published it
  5. now everybody wants to do this.

The results don't need to be sensible, well-founded, or better than random for this to work. If you are eager enough to puvblish something, qny method will do. Unfortunately.

Indeed, distance-based approaches do not make a lot of sense on such data.

The idea of e.g. k-means is to gind groups of low variance, and that primarily makes sense on continuous data.

The popularity of these methods e.g. in marketing is probably best explained as follows:

  1. someone read that clustering is cool
  2. they loaded some data in some program that could run k-means or hierarchical clustering
  3. after a lot of fiddling, they even got a result
  4. the result didn't totally contradict their hypothesis, so they were happy and published it
  5. now everybody wants to do this.

The results don't need to be sensible, well-founded, or better than random for this to work. If you are eager enough to publish something, any method will do. Unfortunately.

If you have categoricial or binary data, a concept that makes much more sense is that of frequent patterns. But of course it's not as sexy (it boils down to counting, just as you did in your example... 1,0 is a frequent pattern). It's also less convenient, because some users may show more than one pattern, and many users will not have a typical pattern - so you don't get this nice "users of type A prefer blue jeans and spend 100$" type of nonsense that sells well as "result". And not so much black magic where you just can pretend that the clusters must be correct, because the magic algorithm found them.

Source Link

Indeed, distance-based approaches do not make a lot of sense on such data.

The idea of e.g. k-means is to gind groups of low variance, and that primarily makes sense on continuous data.

The popularity of these methods e.g. in marketing is probably best explained as follows:

  1. someone read that clustering is cool
  2. they loaded some data in some program that could run k-means or hierarchical clustering
  3. after a lot of fiddling, they even got a result
  4. the result didn't totally contradict their hypothesis, so they were happy and published it
  5. now everybody wants to do this.

The results don't need to be sensible, well-founded, or better than random for this to work. If you are eager enough to puvblish something, qny method will do. Unfortunately.