1
$\begingroup$

I'm looking to cluster data on apartments. I have the following variables for each apartment:

  • Latitude
  • Longitude
  • Price
  • Number of bathrooms
  • Number of bedrooms
  • Amenities (washer, gym, etc.)

The problem I'm struggling with is the Amenities variable has several levels (like over 50). If I one-hot encode this variable and then scale all the features, I would essentially be making each level of Amenities equally important (to the clustering algorithm) as some of the other variables like price (which might obviously be highly correlated with the other features) and location, which doesn't seem like the best approach for clustering. Should I possibly only include a few amenities that I "think" are most important or most common?

Also, I know that K-Means isn't the best approach for location data because it looks to minimize variance rather than geodetic distance. Could someone possibly give any insight as to the best approach to this problem?

$\endgroup$
1
$\begingroup$

Don't just approach this by "how can I have the data to be able to run algorithm X on it".

First you need to go to the drawing board, and make sure you are asking the right question. "What would KMeans yield if I one hot encode the data" is probably not the question you want to answer...

So first begin by formalizing the problem you want to solve. what is a good cluster, on your data, for your task? It's probably not the SSQ objective... Then from that, you can try to find an approach that optimizes this quality; at least approximately.

For mixed data, hierarchical clustering is often much better. Because you can define a domain specific distance function, and then apply easy to understand optimization such as average-link, complete-link based on this distance. This yields clusters with well understood properties.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.