Given parameter/s $\theta$, data $X$ and prior on the parameter/s $p(\theta)$, Bayes' theorem allows us to estimate the posterior distribution $p(\theta | X)$:

$p(\theta | X) = \frac{p(\theta) p(X | \theta)}{p(X)}$

$\to p(\theta | X) \ \propto \ p(\theta) p(X | \theta)$

From the Bayesian linear regression Wiki page:

$p(\beta, \sigma^2 | y, X) \ \propto \ p(y | X, \beta, \sigma^2) p(\beta | \sigma^2) p(\sigma^2)$

I was expecting something like:

$p(\beta, \sigma^2 | y, X) \ \propto p(y | X, \beta, \sigma^2) \color{red}{p(X | \beta, \sigma^2)} p(\beta | \sigma^2) p(\sigma^2)$ or

since I guess

$p(\beta, \sigma^2, y, X) = p(y | X, \beta, \sigma^2) p(X, \beta, \sigma^2)$

Without $p(X | \beta, \sigma^2)$, I guess it's still true, but why not include it?


$X$ does not depend on $\beta$, $\sigma$. These values have to do with getting $y$ from $x$. So $P(X| \beta, \sigma)=P(X)$. Since we are maximizing with respect to $\theta$ we don't care about this constant term.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.