Finding the Bayesian classifier for a bivariate Gaussian distribution

Question

Very close to: Joint Gaussian of two Gaussians

I am trying to find the Bayesian classifier for two classes given by the following bivariate Gaussian distributions:

$$p(x|c_1) = N(\mu_1, \Sigma_1)$$ $$p(x|c_2) = N(\mu_2, \Sigma_2)$$

What is the formula and the methodology I should use to do this? I am confused with out a proper understanding on a formula to do that at the moment. Prior probabilities of $c_1$ and $c_2$ are 0.5 respectively.

FYI:

μ1 = ( 2        Σ1 = (2 0            
       2 )            0 1)


μ2 = ( 2        Σ1 = (4 0            
       4 )            0 2)   

Answer 1

Define a random variable $C\in\{1,2\}$ with prior distribution $\mu_C$ given by $$ \mu_C(A) = P\{C\in A\} = \frac{1}{2} I_A(1) + \frac{1}{2} I_A(2) \, , $$ where $A$ is any subset of $\{1,2\}$.

Use the notation $X=(X_1,X_2)$ and $x=(x_1,x_2)$. Suppose that

$$X\mid C=1\sim N(\mu_1,\Sigma_1)\, ,$$ $$X\mid C=2\sim N(\mu_2,\Sigma_2)\, ,$$

where $\mu_1=(2, 2)^\top$, $\Sigma_1=\textrm{diag}(2,1)$, $\mu_2=(2,4)^\top$ and $\Sigma_2=\textrm{diag}(4,2)$.

Now, study this

http://en.wikipedia.org/wiki/Multivariate_normal_distribution

to understand that

$$ f_{X\mid C}(x\mid 1) = \frac{1}{2\pi\sqrt{2}} \exp\left(-\frac{1}{2}\left(\frac{(x_1-2)^2}{2} + \frac{(x_2-2)^2}{1} \right)\right) \, , $$

$$ f_{X\mid C}(x\mid 2) = \frac{1}{4\pi\sqrt{2}} \exp\left(-\frac{1}{2}\left(\frac{(x_1-2)^2}{4} + \frac{(x_2-4)^2}{2} \right)\right) \, . $$

Using Bayes Theorem, we have $$ P\{C=1\mid X=x\} = \frac{\int_{\{1\}} f_{X\mid C}(x\mid c) \,d\mu_C(c)}{\int_{\{1,2\}} f_{X\mid C}(x\mid c)\, d\mu_C(c)} = \frac{\frac{1}{2} f_{X\mid C}(x\mid 1)}{\frac{1}{2} f_{X\mid C}(x\mid 1) + \frac{1}{2} f_{X\mid C}(x\mid 2)} \, . $$

The idea is to decide for the first classification if $$ P\{C=1\mid X=x\} = \frac{1}{1+\frac{f_{X\mid C}(x\mid 2)}{f_{X\mid C}(x\mid 1)}} > \frac{1}{2} \, , $$ which is equivalent to $$ \frac{f_{X\mid C}(x\mid 2)}{f_{X\mid C}(x\mid 1)} < 1 \, , $$ or $$ \log f_{X\mid C}(x\mid 2) - \log f_{X\mid C}(x\mid 1) < 0 \, , $$ which gives us $$ \log \frac{1}{2} - \frac{(x_1-2)^2}{8} - \frac{(x_2-2)^2}{4} + \frac{(x_1-2)^2}{4} + \frac{(x_2-4)^2}{2} < 0 \, . \qquad (*) $$ Therefore, you decide that the point $x$ belongs to classification $1$ if it is inside the ellipse defined by $$ \frac{(x_1-2)^2}{8(2+\log 2)} + \frac{(x_2-6)^2}{4(2+\log 2)} = 1 \, , $$ otherwise, you decide for classification $2$.

Answer 2

$X_1$ and $X_2$ are independent normal random variables with means $2$ and $4$ under both hypotheses. The variances under one hypothesis are half those under the other. So, the log likelihood ratio is something like

$$\ln 2 - \frac{(x_1-2)^2}{8}-\frac{(x_2-4)^2}{4} + \frac{(x_1-2)^2}{4}+\frac{(x_2-4)^2}{2}$$

and has to be compared to $0$ since the hypotheses are equally likely. After simplification you should get a classifier saying something

Decide $c_1$ if $(x_1,x_2)$ lies inside an ellipse with center $(2,4)$; $c_2$ if outside.

where the equation of the ellipse wil be revealed to you as you do the algebra.