# Posterior distribution in Bayesian linear regression - why not include $p(X | \beta, \sigma^2)$?

Given parameter/s $\theta$, data $X$ and prior on the parameter/s $p(\theta)$, Bayes' theorem allows us to estimate the posterior distribution $p(\theta | X)$:

$p(\theta | X) = \frac{p(\theta) p(X | \theta)}{p(X)}$

$\to p(\theta | X) \ \propto \ p(\theta) p(X | \theta)$

From the Bayesian linear regression Wiki page:

$p(\beta, \sigma^2 | y, X) \ \propto \ p(y | X, \beta, \sigma^2) p(\beta | \sigma^2) p(\sigma^2)$

I was expecting something like:

$p(\beta, \sigma^2 | y, X) \ \propto p(y | X, \beta, \sigma^2) \color{red}{p(X | \beta, \sigma^2)} p(\beta | \sigma^2) p(\sigma^2)$ or

since I guess

$p(\beta, \sigma^2, y, X) = p(y | X, \beta, \sigma^2) p(X, \beta, \sigma^2)$

Without $p(X | \beta, \sigma^2)$, I guess it's still true, but why not include it?

• In the usual regression model, $X$ is not treated as a random variable. – Glen_b Sep 18 '15 at 0:59
• – BCLC Sep 18 '15 at 11:45
• Bayesians don't treat "everything" as random. While you can view X as random, it only "leads to the same estimation procedures" if you condition on X. In which case, again, X will stay on the RHS of the conditioning bar – Glen_b Sep 18 '15 at 16:00
• @Glen_b Is it that frequentists treat the X's as random and parameters fixed and reverse for Bayesians? Or what? – BCLC Sep 18 '15 at 16:02
• Neither. In either case, we can condition on X throughout, or treat the X's as fixed. The effect is the same. So a frequentist can write either $E(Y|X) = X\beta$ or just $E(Y)=X\beta$ depending on which they do, though arguably the first covers both cases. – Glen_b Sep 18 '15 at 22:31

## 1 Answer

$X$ does not depend on $\beta$, $\sigma$. These values have to do with getting $y$ from $x$. So $P(X| \beta, \sigma)=P(X)$. Since we are maximizing with respect to $\theta$ we don't care about this constant term.