3
$\begingroup$

I want to do clustering by using kmeans algorithm. Despite that I searched a lot, I could not find a practical solution by weighting features. Let' s assume that there is a dataset of people with the following features:

number of children (integer) existence of husband (boolean) annual income in $ (integer, but with greater order than the previous integer) weight (double) money spent (integer, at most 3-digit number).

In order to do the clustering, I realised that normalisation is a good practice. I implemented min-max as well as z-score, but I observed that the variance plays a crucial role.

Thus, I think that it is essential to find a practical method of handling the features in a different ("unequal") way.

$\endgroup$
1
  • 1
    $\begingroup$ You have mixed data (married is Boolean). Therefore you cannot do k-means. $\endgroup$ – gung - Reinstate Monica Aug 1 '16 at 23:56
4
$\begingroup$

If you've normalized the variables, then you have made all the features matter the "same" amount. You can now make a feature matter less by scaling.

For example, lets say you have two features so each point is $[x_1, x_2]$. Now let's take 3 points, $a = [0, 0]$, $b = [0, 1]$, and $c = [1, 0]$. So $a$ is equidistant from $b$ and $c$. If we now transform all the points by multiplying the second feature by 2 you get $a' = [0, 0]$, $b' = [0, 2]$, and $c' = [1, 0]$ and $a'$ is now closer to $c'$ than $b'$. This, effectively, makes the first feature matter more.

$\endgroup$
1
$\begingroup$

There are a few ways to do this. You want the standard deviation of the scaled feature to be proportional to the feature's importance. If you have some prior knowledge about how important each feature is, you could choose these yourself. It's more difficult if you want a systematic way of determining the feature importances.

There is one approach I know of, but it requires a target variable (which is generally not available if you're doing clustering). You run the model using the data above fit to the target variable. The feature importances from the model should be the standard deviation you use to scale these features. Fit Scikit Learn models generally have a feature importances attribute.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.