Given a data point $x$ and a possibly multivariate normal distribution $N_1$ with known mean and variance-covariance matrix, it is trivial to compute the likelihood of the data point $x$ given the parameters. In case we have a second distribution $N_2$ and a corresponding second likelihood, we can compute the probabilities that data point $x$ is generated by $N_1$ or $N_2$ by normalizing the likelihoods.

I was wondering whether there is a way to compute a probability that a given data point $x$ is generated by a given distribution $N_1$ without having more than one distribution to compute the corresponding second likelihood. That is, can we quantify the probability that $x$ is generated by $N_1$ given only $x$ and $N_1$? I am not sure whether this is possible and if it is possible, how to normalize the likelihood of $x$ given the parameters of $N_1$.

  • $\begingroup$ Guys could you please link for me come references about normalizing two likelihoods to compare them? $\endgroup$ – Fr1 Aug 9 '19 at 15:40
  • $\begingroup$ Divide each likelihood by the sum of both likelihoods and you can interpret them as probabilities. See for instance the vast literature on classification using mixture models. $\endgroup$ – Mr. Zen Aug 9 '19 at 15:48

When you don't compare with other distribution(s), i.e. your choices are uncountable (or maybe it shouldn't be infinite at all), the question becomes the following:

  • Is this sample (i.e. $x$) drawn from $N_1(\mu,\Sigma)$ or any other distribution? In other words, what is the probability that this sample is drawn from $N_1$?

I can safely say that the answer to this question is $0$. Because the inverse of this event implies uncountably infinite number of distributions. Even if we restrict ourselves into normal distribution, it is almost as equally probably as that the sample is drawn from $N(\mu+\epsilon,\Sigma)$, where $\epsilon$ is arbitrarily small.

  • $\begingroup$ Thank you, that was exactly where this question was going. $\endgroup$ – Mr. Zen Aug 9 '19 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.