I have two normal populations S1 and S2, where S1 ~ N (μ1, σ1) and S2 ~ N (μ2, σ2) respectively. The populations are independent of each other and a data point X has to be either from S1 or from S2. Suppose that I have been given the estimates of μ1, μ2, σ1, σ2 and I’ve been given the priori probability of S1 and S2, say P(S1) = π1 and P(S2) = π2.

My question is, given a data point X1, how do I classify the data X1 into S1 or S2 with the given information. The data I have in not labelled so I cannot go for supervised learning.

I have tried to solve using simple Bayesian probability, but since, X1=x is a real values number not an interval, my numerator will logically be zero.

How should I solve this? I'm stuck. Can anyone show me some direction?

  • $\begingroup$ How did you do your Bayesian approach? I like that and think you can make some progress with such a method. $\endgroup$ – Dave Oct 10 '20 at 18:17
  • 2
    $\begingroup$ You have a gaussian-mixture, with known means $\mu_i$, standard deviations $\sigma_i$ and mixture proportions $\pi_i$. I would have thought that we already had a thread on how to calculate the probability for an observation $x$ to come from the $i$-th component, but we apparently don't. $\endgroup$ – Stephan Kolassa Oct 10 '20 at 18:22
  • $\begingroup$ Hi Dave. I used the simple Bayes Theorem formula as I have the prior probabilities but for the likelihood part, I considered a very small value ϵ so that my X is now [X-ϵ, X+ϵ] and then I computed the posterior. But I don't know if this is the right approach. $\endgroup$ – Sammy Oct 10 '20 at 18:23

Consider the two possible normal populations as models, $p(\mathcal{M_1}), p(\mathcal{M_2})$, that is, two hypothesis that compete to explain your datum $X$.

The prior information tells which is the prior odds,

$$\frac{p(\mathcal{M_1})}{p(\mathcal{M_2})} = \frac{\pi_1}{\pi_2}$$

You can apply the standard formula

$$\underbrace{ \frac{P({\cal M}_1\mid X)}{P({\cal M}_2\mid X)} }_{\text{posterior odds}} = \underbrace{ \frac{P(X \mid {\cal M}_1)}{P(X\mid{\cal M}_2)} }_{\text{Bayes Factor}} \times\underbrace{\frac{P({\cal M}_1)}{P({\cal M}_2)}}_{\text{prior odds}}$$

To compute the likelihoods $P({X \mid \cal M}_i)$ use the density of the Normal, which is another initial assumption.

An example in R:

likelihood <- function(mu, sigma) {
  function(x) dnorm(x, mu, sigma)

lik.M1 <- likelihood(0.0,0.5)
lik.M2 <- likelihood(2.0,0.5)

pi1 <- 0.2
pi2 <- 1-pi1

x <- 0.5

prior.odds <- pi1/pi2
bayes.factor <- lik.M1(x) / lik.M2(x)
posterior.odds <- bayes.factor * prior.odds


With these values, it returns odds 13.7:1

  • $\begingroup$ This gives me a direction to think, thank you. How do I compute the density of the Normal when I have X = 0.07. P(X = 0.07) = 0, right? Also, if i have say, k Normal Populations, how will I approach then? $\endgroup$ – Sammy Oct 10 '20 at 20:52
  • 3
    $\begingroup$ With continuous distributions, you are not dealing with probabilities but with densities (check this youtube.com/watch?v=ZA4JkHKZM50). With more models, multiply each by its likelihood and prior, and divide all by their sum. $\endgroup$ – jpneto Oct 10 '20 at 21:56
  • $\begingroup$ @Sammy you should accept an answer by clicking the check mark, if you think the answer is correct. $\endgroup$ – Chechy Levas Oct 14 '20 at 20:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.