Posterior density for a linear regression model

Question

Given a classical linear regression model

$$y = X\beta + \varepsilon,$$

$$\varepsilon\sim N(0,\sigma^2I_n),$$

the posterior density is proportional to the product of the likelihood and the selected prior density,

$$f(\beta,\sigma^2\mid X,y)\propto f(y\mid X,\beta,\sigma^2)f(\beta,\sigma^2).$$

For ease of notation, I write $\theta = (\beta, \sigma^2)$. So,

$$f(\theta\mid X,y)\propto f(y\mid X, \theta) f(\theta). \tag 1$$ I'm interested in finding the integrating constant, i.e. what makes the "proportional to" in $(1)$ to an "is equal to".

As every joint pdf can be factorized into its conditionals, it follows that

$$f(\theta,X,y)=f(\theta)f(X\mid \theta) f(y\mid X,\theta) \tag 2$$

From Eq. $(2)$ I would follow that

$$f(\theta\mid X,y) = \frac{f(\theta) f(X\mid\theta) f(y\mid X,\theta)}{f(X,y)}.$$

So what makes the relation $(1)$ equal is the division of the right-hand side of $(1)$ by

$$\frac{f(X,y)}{f(X\mid \theta)}.\tag 3$$

However, the term $(3)$ does depend on $\theta$ and thus cannot be the integrating constant. This is what puzzles me right now. I think I have some faulty reasoning here, but I can't see it.

Is it maybe that $X$ and $\theta$ must be assumed to be stochastically independent so that $f(X\mid \theta) = f(X)$ and in this case, $(3)$ would become

$$\frac{f(X,y)}{f(X)}=f(y\mid X). \tag{3'}$$

In this case, $(3')$ would be the integrating constant of the right-hand-side of $(1)$, because $(3')$ does not depend on $\theta$. The posterior density would then be given by

$$f(\theta\mid X,y)=\frac{f(y\mid X,\theta)f(\theta)}{f(y\mid X)}. $$

I would appreciate any help.

Answer 1

Without any context, the proportional relationship is as follows: $f(\theta\mid X,y)\propto f(y\mid X,\theta)f(\theta\mid X)$, and the integrating constant is $p(y\mid X)$ if $X$ is given; but in (1), you write $f(\theta\mid X,y)\propto f(y\mid X,\theta)f(\theta)$ with implicit assumption that $f(\theta\mid X)=f(\theta)$, which also validates your reasoning: $f(X\mid \theta)=f(\theta)$, i.e. $X$ and $\theta$ are assumed to be stochastically independent. This is a natural assumption since one belongs to the model and the other is the data (without the output variable).