In the book an introduction to statistical learning, the following proportionality is given:

$$ p(\beta|X,Y) \propto f(Y|X,\beta)\,p(\beta|X) $$

however, I'm wondering what the proportionality constant is, and how it can be derived.

  • $\begingroup$ The total probability must be $1$. $\endgroup$
    – whuber
    Jan 19 '18 at 21:22
  • $\begingroup$ The mathematical points made above are all correct. There's also a practical issue: it is often too difficult to calculate the sum. There are sampling methods that can cope with this. $\endgroup$ Jan 19 '18 at 22:35

Just use the definition of conditional probability: $p(A|B)=\frac{p(A,B)}{p(B)}$.

Left-hand side is $p(\beta|X,Y)=\frac{p(X,Y,\beta)}{p(X,Y)}$

Right-hand side is $p(Y|X,\beta)p(\beta|X)=\frac{p(X,Y,\beta)}{p(X,\beta)}\frac{p(X,\beta)}{p(X)}=\frac{p(X,Y,\beta)}{p(X)}$

Comparing both sides, you can see that the constant of proportionality should be $\frac{p(X)}{p(X,Y)}=\frac{1}{p(Y|X)}$

  • $\begingroup$ So there is no difference between $f$ and $p$? $\endgroup$
    – Frank Vel
    Jan 19 '18 at 21:42
  • 1
    $\begingroup$ Given a specific observation $Y$, $p(Y|X,\beta)$ can be viewed as a function of $X$ and $\beta$ instead of as a probability value. So sometimes people use $f$ to emphasize that it is a likelihood function rather than probability distribution. But that is just an interpretation thing, so doesn't matter for the calculation of the constant of proportionality. $\endgroup$ Jan 19 '18 at 22:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.