# Given two normal populations,, classifying a given data point

I have two normal populations S1 and S2, where S1 ~ N (μ1, σ1) and S2 ~ N (μ2, σ2) respectively. The populations are independent of each other and a data point X has to be either from S1 or from S2. Suppose that I have been given the estimates of μ1, μ2, σ1, σ2 and I’ve been given the priori probability of S1 and S2, say P(S1) = π1 and P(S2) = π2.

My question is, given a data point X1, how do I classify the data X1 into S1 or S2 with the given information. The data I have in not labelled so I cannot go for supervised learning.

I have tried to solve using simple Bayesian probability, but since, X1=x is a real values number not an interval, my numerator will logically be zero.

How should I solve this? I'm stuck. Can anyone show me some direction?

• How did you do your Bayesian approach? I like that and think you can make some progress with such a method. – Dave Oct 10 '20 at 18:17
• You have a gaussian-mixture, with known means $\mu_i$, standard deviations $\sigma_i$ and mixture proportions $\pi_i$. I would have thought that we already had a thread on how to calculate the probability for an observation $x$ to come from the $i$-th component, but we apparently don't. – Stephan Kolassa Oct 10 '20 at 18:22
• Hi Dave. I used the simple Bayes Theorem formula as I have the prior probabilities but for the likelihood part, I considered a very small value ϵ so that my X is now [X-ϵ, X+ϵ] and then I computed the posterior. But I don't know if this is the right approach. – Sammy Oct 10 '20 at 18:23

Consider the two possible normal populations as models, $$p(\mathcal{M_1}), p(\mathcal{M_2})$$, that is, two hypothesis that compete to explain your datum $$X$$.

The prior information tells which is the prior odds,

$$\frac{p(\mathcal{M_1})}{p(\mathcal{M_2})} = \frac{\pi_1}{\pi_2}$$

You can apply the standard formula

$$\underbrace{ \frac{P({\cal M}_1\mid X)}{P({\cal M}_2\mid X)} }_{\text{posterior odds}} = \underbrace{ \frac{P(X \mid {\cal M}_1)}{P(X\mid{\cal M}_2)} }_{\text{Bayes Factor}} \times\underbrace{\frac{P({\cal M}_1)}{P({\cal M}_2)}}_{\text{prior odds}}$$

To compute the likelihoods $$P({X \mid \cal M}_i)$$ use the density of the Normal, which is another initial assumption.

An example in R:

likelihood <- function(mu, sigma) {
function(x) dnorm(x, mu, sigma)
}

lik.M1 <- likelihood(0.0,0.5)
lik.M2 <- likelihood(2.0,0.5)

pi1 <- 0.2
pi2 <- 1-pi1

x <- 0.5

prior.odds <- pi1/pi2
bayes.factor <- lik.M1(x) / lik.M2(x)
posterior.odds <- bayes.factor * prior.odds

posterior.odds


With these values, it returns odds 13.7:1

• This gives me a direction to think, thank you. How do I compute the density of the Normal when I have X = 0.07. P(X = 0.07) = 0, right? Also, if i have say, k Normal Populations, how will I approach then? – Sammy Oct 10 '20 at 20:52
• With continuous distributions, you are not dealing with probabilities but with densities (check this youtube.com/watch?v=ZA4JkHKZM50). With more models, multiply each by its likelihood and prior, and divide all by their sum. – jpneto Oct 10 '20 at 21:56
• @Sammy you should accept an answer by clicking the check mark, if you think the answer is correct. – Chechy Levas Oct 14 '20 at 20:08