# Posterior density for a linear regression model

Given a classical linear regression model

$$y = X\beta + \varepsilon,$$

$$\varepsilon\sim N(0,\sigma^2I_n),$$

the posterior density is proportional to the product of the likelihood and the selected prior density,

$$f(\beta,\sigma^2\mid X,y)\propto f(y\mid X,\beta,\sigma^2)f(\beta,\sigma^2).$$

For ease of notation, I write $$\theta = (\beta, \sigma^2)$$. So,

$$f(\theta\mid X,y)\propto f(y\mid X, \theta) f(\theta). \tag 1$$ I'm interested in finding the integrating constant, i.e. what makes the "proportional to" in $$(1)$$ to an "is equal to".

As every joint pdf can be factorized into its conditionals, it follows that

$$f(\theta,X,y)=f(\theta)f(X\mid \theta) f(y\mid X,\theta) \tag 2$$

From Eq. $$(2)$$ I would follow that

$$f(\theta\mid X,y) = \frac{f(\theta) f(X\mid\theta) f(y\mid X,\theta)}{f(X,y)}.$$

So what makes the relation $$(1)$$ equal is the division of the right-hand side of $$(1)$$ by

$$\frac{f(X,y)}{f(X\mid \theta)}.\tag 3$$

However, the term $$(3)$$ does depend on $$\theta$$ and thus cannot be the integrating constant. This is what puzzles me right now. I think I have some faulty reasoning here, but I can't see it.

Is it maybe that $$X$$ and $$\theta$$ must be assumed to be stochastically independent so that $$f(X\mid \theta) = f(X)$$ and in this case, $$(3)$$ would become

$$\frac{f(X,y)}{f(X)}=f(y\mid X). \tag{3'}$$

In this case, $$(3')$$ would be the integrating constant of the right-hand-side of $$(1)$$, because $$(3')$$ does not depend on $$\theta$$. The posterior density would then be given by

$$f(\theta\mid X,y)=\frac{f(y\mid X,\theta)f(\theta)}{f(y\mid X)}.$$

I would appreciate any help.

• I extensively cleaned up your MathJax usage. But I left intact the abominable usage by which the same symbol is used for both a random variable and the argument to its density function, and the same symbol for multiple different densisty functions. Mar 3, 2019 at 17:37

Without any context, the proportional relationship is as follows: $$f(\theta\mid X,y)\propto f(y\mid X,\theta)f(\theta\mid X)$$, and the integrating constant is $$p(y\mid X)$$ if $$X$$ is given; but in (1), you write $$f(\theta\mid X,y)\propto f(y\mid X,\theta)f(\theta)$$ with implicit assumption that $$f(\theta\mid X)=f(\theta)$$, which also validates your reasoning: $$f(X\mid \theta)=f(\theta)$$, i.e. $$X$$ and $$\theta$$ are assumed to be stochastically independent. This is a natural assumption since one belongs to the model and the other is the data (without the output variable).