0
$\begingroup$

My question deals with what is the right way to normalize my data. My data consists 6 features, all together representing a state in an environment for reinforcement learning. My goal is to cluster states with KMeans, so of course I need to normalize values first.

Below are histograms of the different features: Feature1 distribution Feature2 distribution Feature4 distribution Feature6 distribution

(X axis is feature value, Y axis is number of appearances. if you wonder where are features 3 and 5, they are kinda look like normal)

I wonder what is the right way to normalize each feature (or all of them together) to use kmeans.

I tried to apply f-score (all features toghther, not one by one) but I wonder if it is the right away. The minimum value I get is -4.6 (in feature 2, as you can see it has values around zero) with maximum of around ~1. Are there better suggestions what can I do?

$\endgroup$
0
$\begingroup$

The k-means algorithm is not guaranteed to converge to a global optimum and is very initialization dependent; furthermore, its objective of squared Euclidean distance results in it minimizing intra-cluster variance. This means for k-means, you generally want your covariance matrix to be spherical; i.e. the covariance matrix is proportional to the identity matrix.

This means techniques like z-score scaling or range normalization on each feature can be useful. To see why, try coming up with a 2D example of where k-means changes depending on the initialization -- it's a common exam question that I've gotten that lends more insight into the behavior of k-means.

Obviously, it's hard (impossible, even) to visualize things in 6 dimensions, but if you can't plot out your data to analyze it yourself, I'd recommend asking yourself -- why k-means as opposed to any other clustering algorithm?

Here's a link with more info on the assumptions of k-means clustering that I find useful.

$\endgroup$
  • $\begingroup$ Thank you for your answer - but it is not a solution for my problem. For many other clustering algorithms as well I'll need to normalize my data. And I'm still not sure how to normalize it $\endgroup$ – Roim May 20 '20 at 8:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.