0
$\begingroup$

Consider a one-dimensional classification problem with $X = \mathbb R$ and $Y = \{-1, +1\}$: $$p(y=-1)=\frac{3}{5} \qquad p(x \mid y=-1)=\frac{1}{2\sqrt{2\pi}}e^{-\frac{(x+2)^2}{8}} \qquad N(-2, 2)$$ $$p(y=+1)=\frac{2}{5} \qquad p(x \mid y=+1)=\frac{1}{\sqrt{2\pi}}e^{-\frac{(x-1)^2}{2}} \qquad N(1, 1)$$

  1. Find the marginal distribution $p(x)$ and the conditional distributions $p(y = -􀀀1 \mid x)$ and $p(y = +1 \mid x)$.
  2. Guess from $p(y = 􀀀-1 \mid x)$ and $p(y = +1 \mid x)$ what the Bayes-optimal classifier is like.

Could you please direct me on how to start with solving the first order or point me to a good easy explained document with examples about marginal distributions for continuous and discrete variables, and how could $N(-2,2), N(1,1)$ be used in solving this question?


I started using the formula: $p(x) = p(x \mid y=-1) * p(y=-1) + p(x \mid y=+1) * p(y=+1)$ then replaced each term with its value from the question, then tried to simplify the formula but with no luck as it got more complex.

$\endgroup$

1 Answer 1

1
$\begingroup$

$$p(y=1|x) * p(x) = p(x|y=1) * p(y=1)$$ $$p(y=1|x) = \frac{p(x|y=1) *p(y=1)}{p(x|y=1)* p(y=1) + p(x|y=-1)* p(y=-1) }$$ $$p(y=1|X=x) = \frac{\frac{2}{5}*\frac{1}{\sqrt{2*\pi}}e^{-\frac{(x-1)^2}{2}}}{\frac{2}{5}*\frac{1}{\sqrt{2*\pi}}e^{-\frac{(x-1)^2}{2}}+\frac{3}{5}*\frac{1}{\sqrt{8*\pi}}e^{-\frac{(x+2)^2}{8}}}$$ I admit I used

http://www.wolframalpha.com/input/?i=2*exp%28-%28t-1%29%5E2%2F2%29%2F%282*exp%28-%28t-1%29%5E2%2F2%29%2B3%2F2*exp%28-%28t%2B2%29%5E2%2F8%29%29

to simplify :) $$p(y=1|x) = \frac{4}{4+3*e^{\frac{3}{8}*(x-4)*x}}$$

$$p(y=-1|x)=1-p(y=1|x)=\frac{3}{3+4*e^{-\frac{3}{8}*(x-4)*x}}$$

solving either equal to 1/2 gives you the (hint! there are 2!) transition points between most likely estimates of y

$\endgroup$
1
  • $\begingroup$ Thank you for the solution, and more importantly for the link provided. I will use it to check if there is a simplified form for the main formula p(x). $\endgroup$
    – Mousa
    Commented Nov 20, 2015 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.