0
$\begingroup$

I am referring to Prof Andrew Ng Coursera ML notes (Week 9). He says that to identify outliers we first model the training data and then fit a Gaussian distribution with probability density $ p(X; \mu $, $\sigma^2 $ ). He then suggests that we can classify a new example $x$ as an outlier if "probability": $ p(x) <\epsilon $, where $\epsilon $ is a hyperparameter.

However, it doesn't make sense to me that how is he using term density and probability interchangeably without even explicitly mentioning it. I find this puzzling because if we assume $X$ to be a continuous random variable then $p(X=x)=0$ $\forall x$. Please help me in understanding what is actually going on in this algorithm at a finer level.

https://math.stackexchange.com/questions/521575/difference-between-probability-and-probability-density

Improve this question
$\endgroup$
1
  • $\begingroup$ Should be $p(X>x) < \epsilon$ or $p(X<x)<\epsilon$. $\endgroup$ – user158565 Jul 21 '19 at 13:02
0
$\begingroup$

Putting a threshold on the density of probability equals to putting a limit on the probability in the sense that you say

$$ \int_{-\infty}^{-\epsilon}p(y)dy + \int_{\epsilon}^{\infty}p(y)dy \equiv \alpha $$

and then define the rule (for a univariate Gaussian): if

$$ \int_{-\infty}^{-x}p(y)dy + \int_{x}^{\infty}p(y)dy < \alpha $$

then $x$ is an outlier. And this is the idea behind more sophisticated approaches like one-class SVMs for outlier detection. You say any sample from sample space corresponding to regions with very (according to some threshold) low probability are considered outliers.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks. This explanation makes sense. $\endgroup$ – Shadab Azeem Jul 23 '19 at 9:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.