Probability that a random variable comes from one of two normal distributions [duplicate]

Question

This question already has an answer here:

• Test to see what population an observation came from 2 answers

Suppose you know $Y \sim N(\mu_1, \sigma_1^2)$ or $Y \sim N(\mu_2, \sigma_2^2)$. You observe $Y=y$, some realization of the random variable $Y$. What is the probability that $Y \sim N(\mu_1, \sigma_1^2)$?

My intuition is to compare $p$-values from each distribution. Let $p_i$ be the $p$-value for $y$ under $N(\mu_i, \sigma_i^2)$. Here I am thinking of the two-sided $p$-value, $p_i = 2\Phi(-|y-\mu_i|/\sigma_i)$ where $\Phi(x)$ is the standard normal distribution function. I would answer my own question as $p_1/(p_1+p_2)$. But I cannot find any reference that would support this (or even treats this problem).

Answer 1

Let $m_1$ be the model $Y \sim N(\mu_1, \sigma_1^2)$ and correspondingly for $m_2$.

Your question is essentially to find $P(M=m_i|Y=y)/\sum_j P(M_j|Y=y)$.

\begin{eqnarray} P(M=m_i|Y=y) &=& P(M=m_i,Y=y)/P(Y=y) \\ &=& P(Y=y|M=m_i)P(M=m_i)/P(Y=y) \end{eqnarray}

(this is Bayes theorem)

We can't evaluate $P(Y=y)$.

We're left with: $P(M=m_i|Y=y) \propto P(Y=y|M=m_i)P(M=m_i)$

Which we can evaluate as long as we have a prior probability for each model; we can then scale by the sum of these quantities (which are proportional to probabilities and have the same proportionality constant, $1/P(Y=y)$) to get actual conditional probabilities (assuming the model class is correct and spans the space of possible models).

Now $P(Y=y|M=m_i)$ is just the likelihood for $m_i$. If the prior probabilities are equal, the ratio of probabilities of the models given the data is proportional to the ratio of likelihoods.

The problem with using p-values is they evaluate not the relative probabilities of the sample given the model, but the relative probability of a sample at least as extreme as the one observed (as measured by whatever test statistic you use). That is, they compute a probability that is in general different from the one you want to find (p-values answer the wrong question).

Answer 2

P-values are very specialized probabilities, and my first feeling is that they're not directly helpful. Bayes' rule, on the other hand, is. Let's refer to your two distributions as Class 1 and Class 2, respectively. Let $f_1(y)$ and $f_2(y)$ be the respective normal densities. Then

$$ \Pr(\text{Class 1} | y) = \frac{f(y , \text{Class 1})}{f(y)} = \frac{\Pr(\text{Class 1}) \cdot f_1(y)}{f(y)}.$$

By the law of total probability, we can expand $$f(y) = \Pr(\text{Class 1})\cdot f_1(y) + \Pr(\text{Class 2})\cdot f_2(y).$$ You could make the class probabilities unknowns themselves to be estimated, or just plug in .5 to reflect your uncertainty. That would give you $$ \Pr(\text{Class 1} | y) = \frac{f_1(y)}{f_1(y) + f_2(y)}$$