0
$\begingroup$

This is a continuation of my previous question.


I have two classes, $C_1$ and $C_2$.

$C_1$ is a bivariate Gaussian with mean $\mu = (0,0)$ and covariance $\Sigma = I$

$C_2$ is a bivariate Gaussian with mean $\mu = (1,3)$ and covariance $\Sigma = 2I$, where $I$ is the identity matrix.

I am trying to calculate $P(x|C_1)$ and $P(x|C_2)$ so I can eventually calculate $P(C|x)$

To calculate $P(x|C_1)$ and $P(x|C_2)$ this I'm using the formula for a bivariate normal distribution found here.

My covariance is zero, which makes this a little bit easier.

When I use this calculate $P(x|C_1)$ I have...

$z=x_1^2 - x_2^2$

$p=0$

$p(x_1,x_2) = (\frac{1}{2\pi})e^{-z/2}$

When I use this to calculate $P(x|C_2)$ I have...

$z=\frac{1}{4}( (x_1-1)^2 + (x_2-3)^2 )$

$p=0$

$p(x_1,x_2) = (\frac{1}{8\pi})e^{z/8}$

Did I do this correctly? Also, I'm a bit confused as to whether what I'm doing even gives me $P(x|C_1)$ and $P(x|C_2)$. I'm a bit over my head in the class I'm in, so if I'm totally wrong there please correct me.

Anyways, with these two values I'm supposed to calculate $P(C|x)$ using Bayes rule (I think). I have the priors of $C_1$ and $C_2$ (they are 0.4 and 0.6 respectively), but I'm lost exactly on how to calculate $P(C|x)$ with this.

Could somebody basically just check over some of my work and help me out with the process using Bayes rule to calculate $P(C|x)$?

EDIT: My end goal here is to calculate an optimal decision boundary between $C_1$ and $C_2$, if I'm going the complete wrong way here let me know, but from what I've gathered through time spent on this site, I think I'm headed the right way.

$\endgroup$
  • $\begingroup$ If this is work for a class (yes even if it's not explicitly work to hand in), please add the homework tag. $\endgroup$ – Glen_b Feb 24 '13 at 2:04
  • $\begingroup$ Sorry about that, it's been added $\endgroup$ – Hoser Feb 24 '13 at 2:13
  • $\begingroup$ When you say "$C_1$ is two Gaussian variables" do you mean a mixture? $\endgroup$ – Glen_b Feb 24 '13 at 2:20
  • $\begingroup$ I made some edits to your question; please double check I didn't screw up the mathematics on you. $\endgroup$ – Glen_b Feb 24 '13 at 2:23
  • $\begingroup$ Your edits look okay to me. And when I say that I mean that in the problem given to me it says C1 is a class of two gaussian variables. I would assume that means a mixture of the two $\endgroup$ – Hoser Feb 24 '13 at 3:17
1
$\begingroup$

OP Hoser has now posed three questions: this one, the one cited in this question, and the one where he was told the decision boundary for his problem on the same topic, and his confusion seems to increase every time.

For this problem,

  • There is no need to calculate $P(x\mid C_1)$ and $P(x\mid C_2)$.
    You are told that your observation which you denote as $x$ is a pair of real numbers that are to be treated as random variables $(X,Y)$ whose conditional joint density (this is what is meant by your $P(x\mid C_i)$ is given to be

    • jointly normal (a.k.a bivariate normal) with mean vector $[0,0]$ and covariance matrix $I$ for the case $i=1$.
    • jointly normal (a.k.a bivariate normal) with mean vector $[1,3]$ and covariance matrix $2I$ for the case $i=2$.

    In both cases, $X$ and $Y$ are (conditionally) independent random variables. So, $P(x\mid C_1)$ is the bivariate density function $f_{X,Y\mid C_1}(u,v \mid C_1)$ and so should have things like $\frac{1}{2\pi}, $exponential functions, $u^2/2$ and $v^2/2$ in it, and no product terms $uv$ anywhere. Can you write down this function of $(u,v)$? How about $P(x\mid C_2)$ which will have $\frac{1}{2\pi}$, exponential functions, $(u-1)^2/8$ and $(v-3)^2/8$ in it, and no product terms $uv$ anywhere?

    The work you have shown in your question is incorrect, by the way.

  • The $C$ in your $P(C)$ has nothing to do with $C_1$ and $C_2$. It is the probability of a correct decision, and to find this probability, you first need to determine the decision boundary which partitions the plane into two _regions which can be called $\Gamma_1$ and $\Gamma_2$, and then do two separate calculations as described below for the cases $i=1$ and $i=2$:

    • what is the probability of a correct decision when the observation $x$ (which is a pair of real numbers) belongs to class $C_i$? To do this, you work with $f_{X,Y\mid C_i}(u,v \mid C_i)$ and calculate $$P\{(X,Y) \in \Gamma_1 \mid C_i\} = \int\int_{(u,v) \in \Gamma_i} f_{X,Y\mid C_i}(u,v \mid C_i)\,\mathrm du\,\mathrm dv.$$ If you are lucky, the decision boundary will be a straight line and this integral can be evaluated easily by a change of variables involving a rotation of coordinates about the mean point $(0,0)$ or $(1,3)$ as the case may be; otherwise, numerical integration will be required. Call the probability thus obtained $P(C\mid C_i)$.

    Next, you will need to combine your answer using the law of total probability to get $$P(C) = P(C\mid C_1)P(C_1) + P(C\mid C_2)P(C_2) = P(C\mid C_1)\times 0.4 + P(C\mid C_2)\times 0.6$$

$\endgroup$
  • $\begingroup$ Thank you I really appreciate this. I apologize if I went about all of this the wrong way. $\endgroup$ – Hoser Feb 25 '13 at 15:19
  • $\begingroup$ Quick question, for $P(X|C_2)$ would it not be $\frac{1}{4\pi}$ rather than $\frac{1}{2\pi}$? This is due to the covariance matrix being $2I$ rather than $I$ I believe. $\endgroup$ – Hoser Feb 25 '13 at 16:57
  • $\begingroup$ If you don't mind, this is what my eventual boundary looks like when plotted in R. Does this look correct given a quick glance? $\endgroup$ – Hoser Feb 25 '13 at 20:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.