0
$\begingroup$

I am using a KNN anomaly detection approach, where the distance to my nearest neighbor is an indication for an anomaly.

I am wondering how I can normalize the score between 0 and 1. I can use a test dataset without anomalies to get the normal data distribution. When I calculate the z-score it's not bounded between 0 and 1. The sigmoid function applied on the z-score returns too high values.

Is there some statistical approach based on the data distribution that returns a probability value that my distance value (or z-score) is an outlier?

$\endgroup$
2
  • $\begingroup$ why not score / max(score) ? That'll be in 0-1 (a distance is always positive) $\endgroup$
    – user603
    Oct 16 '18 at 20:39
  • $\begingroup$ because I don't know before what is my max value as there can always be a new anomaly which has a higher value. $\endgroup$
    – MikeHuber
    Oct 16 '18 at 20:52
0
$\begingroup$

If you make the assumption that the z-scores are normally distributed, it is easy to convert the score to the probability of encountering a point that far away or farther p(x >= abs(z)). For example, in R you could write: Score = 2 * (1 - pnorm(abs(z)))

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.