Finding the decision boundary between two gaussians Assume we are trying to classify between 2 classes, each has a Gaussian conditional probability, with different means but same variance, i.e. $X | y=0 \sim N(\mu_0, \Sigma); X | y=1 \sim N(\mu_1, \Sigma)$. 
Our decision rule would be $1 \iff P(y = 1 | X) > P(y=0|X)$ (and vice versa for 0). 
Using Bayes rule we can invert the conditional probabilities, and get: $\iff \frac{P(X|y=1)P(y=1)}{P(X)} > \frac{P(X|y=0)P(y=0)}{P(X)}$. 
We can next eliminate the denominator. 
Now, if $P(y=1) = P(y=0)$ we could eliminate that as well, and the decision rule would be simplified to $P(X|y=1) > P(X|y=0)$, which basically is which $\mu$ is $X$ closer to. Now intuitively this translates to $1 \iff X > c = \frac{\mu_0 +\mu_1}{2}$.
I have 2 questions:


*

*Can we prove this intuition mathematically?

*What happens if $P(y=1) \neq P(y=0)$ ?
 A: *

*Yes you can. For example, 
$$
\mathbb{P}\left(X=x|y=1\right)=\mathbb{P}\left(X=x|y=0\right)
$$
happens exactly when $x$ is between the means, which can be calculated directly from the definition. Essentially, you are finding the decision boundary for this image, , 


but it's even simpler because you are assuming that the standard deviations are equal. You'd need to show with a simple equation that the probability (area under the curve) on the right is equal to the area on the left if the boundary is in the middle. 


*I'm not willing to do the math here, but you have to slightly alter the equation above to find what happens to the boundary. I can tell it doesn't change linearly, because, for example, if you believe the left bubble to be twice as likely as the right bubble, you're looking for a boundary so that the area under what's left of the left bubble is half that of what's right of the boundary under the right bubble. 

A: For 1D, i.e. $\Sigma = \sigma^2$ : 


*

*$P(X|y=1) > P(X|y=0) \iff \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{1}{2}\frac{(x-\mu_1)^2}{\sigma^2}} > \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{1}{2}\frac{(x-\mu_0)^2}{\sigma^2}} \iff 
\\e^{-\frac{1}{2}\frac{(x-\mu_1)^2}{\sigma^2}} > e^{-\frac{1}{2}\frac{(x-\mu_0)^2}{\sigma^2}} \iff \ln(...) > \ln(...) \iff -\frac{1}{2}\frac{(x-\mu_1)^2}{\sigma^2} > -\frac{1}{2}\frac{(x-\mu_0)^2}{\sigma^2} \iff \\ (x-\mu_1)^2 < (x-\mu_0)^2 \iff x^2 -2x\mu_1 + \mu_1^2 < x^2 -2z\mu_0 + \mu_0^2 \iff \\ \mu_1^2 - \mu_0^2 < 2(\mu_1-\mu_0)x \iff (\mu_1 + \mu_0)/2 < x$
$$\tag*{$\blacksquare$}$$ 

*$P(X|y=1)p > P(X|y=0)(1-p) \iff \\ \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{1}{2}\frac{(x-\mu_1)^2}{\sigma^2}}p > \frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{1}{2}\frac{(x-\mu_0)^2}{\sigma^2}}(1-p) \iff \\e^{-\frac{1}{2}\frac{(x-\mu_1)^2}{\sigma^2}}p > e^{-\frac{1}{2}\frac{(x-\mu_0)^2}{\sigma^2}}(1-p) \iff \ln(...) > \ln(...) \iff \\ -\frac{1}{2}\frac{(x-\mu_1)^2}{\sigma^2} + \ln(p) > -\frac{1}{2}\frac{(x-\mu_0)^2}{\sigma^2} + \ln(1-p) \iff ... \iff \\x > \frac{\mu_1 + \mu_0}{2} + \frac{\sigma^2}{\mu_1-\mu_0} \ln(\frac{1-p}{p})$
$$\tag*{$\blacksquare$}$$ 
For 2D and more, it can be shown (though the derivation is too much for me to MathJax right now) that (given identical co-variance matrix $\Sigma$ for both distributions):


*

*$\iff 2(\Sigma^{-1}(\mu_1 - \mu_0))^T X > \mu_1^T \Sigma^{-1} \mu_1 - \mu_0^T \Sigma^{-1} \mu_0$

*$\iff 2(\Sigma^{-1}(\mu_1 - \mu_0))^T X > \mu_1^T \Sigma^{-1} \mu_1 - \mu_0^T \Sigma^{-1} \mu_0 + 2\ln(\frac{1-p}{p})$
You can also check here for the general case where the co-variance matrices aren't equal.
