Skip to main content
Added follow up question on scaling
Source Link
Doug Fir
  • 1.6k
  • 1
  • 19
  • 36

I have a customer data set with several features. These features have mostly different meanings e.g.

  • (Currency / Money) Customer monthly spend in $
  • (Count) Quantity of service x customer has active with us
  • (Count) Quantity of service y that customer has with us
  • (Boolean) Is customer a premium services customer
  • (Boolean) Is customer some other kind of customer boolean
  • (Time) Customer tenure in months
  • (Time but entirely different meaning) Average time in a month spent using our call center

I had built a script using simple KMeans only because I've used it before and it's simple to understand.

Then, a post I came across told me to scale my data, that makes sense. But then I did some more research and found that KMeans is not ideal for clustering on binary fields. I have some binary fields, but not all my data are binary. The example features above pretty much cover the nature of the features that I have in my data set.

Can I still use Kmeans for clustering? Are there some more appropriate algorithms to cluster data of this nature?

Edit: On the topic of scaling, I was reading this SO post. From the accepted answer:

... and it is not appropriate to scale a discrete or categorical variable.

Should I scale some of my features and not others? Is this a standard practice? So with the example features above I would scale Customer monthly spend, the two count variables (integers) and the two time features since they are expressed as integers of whole months. Then, I would just leave the booleans, which are just dummied categoricals?

I have a customer data set with several features. These features have mostly different meanings e.g.

  • (Currency / Money) Customer monthly spend in $
  • (Count) Quantity of service x customer has active with us
  • (Count) Quantity of service y that customer has with us
  • (Boolean) Is customer a premium services customer
  • (Boolean) Is customer some other kind of customer boolean
  • (Time) Customer tenure in months
  • (Time but entirely different meaning) Average time in a month spent using our call center

I had built a script using simple KMeans only because I've used it before and it's simple to understand.

Then, a post I came across told me to scale my data, that makes sense. But then I did some more research and found that KMeans is not ideal for clustering on binary fields. I have some binary fields, but not all my data are binary. The example features above pretty much cover the nature of the features that I have in my data set.

Can I still use Kmeans for clustering? Are there some more appropriate algorithms to cluster data of this nature?

I have a customer data set with several features. These features have mostly different meanings e.g.

  • (Currency / Money) Customer monthly spend in $
  • (Count) Quantity of service x customer has active with us
  • (Count) Quantity of service y that customer has with us
  • (Boolean) Is customer a premium services customer
  • (Boolean) Is customer some other kind of customer boolean
  • (Time) Customer tenure in months
  • (Time but entirely different meaning) Average time in a month spent using our call center

I had built a script using simple KMeans only because I've used it before and it's simple to understand.

Then, a post I came across told me to scale my data, that makes sense. But then I did some more research and found that KMeans is not ideal for clustering on binary fields. I have some binary fields, but not all my data are binary. The example features above pretty much cover the nature of the features that I have in my data set.

Can I still use Kmeans for clustering? Are there some more appropriate algorithms to cluster data of this nature?

Edit: On the topic of scaling, I was reading this SO post. From the accepted answer:

... and it is not appropriate to scale a discrete or categorical variable.

Should I scale some of my features and not others? Is this a standard practice? So with the example features above I would scale Customer monthly spend, the two count variables (integers) and the two time features since they are expressed as integers of whole months. Then, I would just leave the booleans, which are just dummied categoricals?

Source Link
Doug Fir
  • 1.6k
  • 1
  • 19
  • 36

Appropriate clustering algorithm with mixed data types

I have a customer data set with several features. These features have mostly different meanings e.g.

  • (Currency / Money) Customer monthly spend in $
  • (Count) Quantity of service x customer has active with us
  • (Count) Quantity of service y that customer has with us
  • (Boolean) Is customer a premium services customer
  • (Boolean) Is customer some other kind of customer boolean
  • (Time) Customer tenure in months
  • (Time but entirely different meaning) Average time in a month spent using our call center

I had built a script using simple KMeans only because I've used it before and it's simple to understand.

Then, a post I came across told me to scale my data, that makes sense. But then I did some more research and found that KMeans is not ideal for clustering on binary fields. I have some binary fields, but not all my data are binary. The example features above pretty much cover the nature of the features that I have in my data set.

Can I still use Kmeans for clustering? Are there some more appropriate algorithms to cluster data of this nature?