# in this situation, why likelihood has this form in posterior = prior*likelihood

I know from Bayes theorem we have the formula "posterior $\propto$ prior * likelihood" or more concrete $Pr(\theta|D) \propto Pr(\theta)Pr(D|\theta)$ with $Pr(\theta)$ is prior distribution of parameter, and $D$ is observed data. But in a book I read when apply it to the following situation, the result confuses me.

Situation: We want to approximate a real function $f: X = R^k \rightarrow R$ by a linear function $h(x,w)$, $w$ is the parameter vector. Having observed data $D = (X,Y) = (x_1,y_1),...,(x_n,y_n)$, $y_i$ is label of $x_i$.

Now let $Pr(w)$ is prior distribution of parameter $w$. In the book, have result that is

$$Pr(w|X,Y)\propto Pr(w)Pr(Y|X,w))\quad (1)$$ But from my understand, by applying $Pr(\theta|D) \propto Pr(\theta)Pr(D|\theta)$ , I think

$$Pr(w|X,Y)\propto Pr(w)Pr(X,Y|w)\quad (2)$$ The difference between (1) and (2) is $Pr(Y|X,w)$ and $Pr(X,Y|w)$.

Could you tell me why we have $(1)$?

You may want to know what the book is? But it is not in english, and there is not ebook form for it, so I don't post it here.

I try to test if (1) and (2) are the same? I transform $Pr(X,Y|w)$ as follows:

$$Pr(X,Y|w)=Pr(X,Y,w)/Pr(w)=Pr(Y|X,w)Pr(X,w)/Pr(w)=Pr(Y|X,w)Pr(w)Pr(X|w)/Pr(w)=Pr(Y|X,w)Pr(X|w)$$ So $(1)$ & $(2)$ are not the same.

• Are the X's being conditioned on, as with a regression model? – Glen_b Jun 15 '17 at 6:34
• No. But what are the conditions that you expect? Is that conditions make answer clearer – ydhhat Jun 15 '17 at 7:51

This situation is that we want to know how to predict $Y$ when we know $X$, which means learning about $w$. So the RHS of the equation should not contain a $p(X|whatever)$ term, because we are interested in cases where $X$ is fixed already.

In terms of the algebra:

$p(w|X,Y)=p(X,Y|w)p(w)/p(X,Y)$

$=p(Y|w,X)p(X|w)p(w)/p(Y|X)p(X)$

$=p(Y|w,X)p(w|X)/p(Y|X)$

This does assume that we are happy with writing $p(X)$.

*I suspect the use of the $\propto$ convention here makes the derivation less clear because it sweeps terms that give you a clue how to proceeed under the carpet.