# Deriving a conditional joint probability model for the data in a Bayesian linear model

I have been reading Tony Lancaster's 2004 book "An Introduction to Modern Bayesian Econometrics." On pages 116-117, Lancaster derives a conditional joint distribution for the data $$p(y,X|\boldsymbol{\beta})$$ for a linear model $$y=X\boldsymbol{\beta}+\epsilon$$ using a joint distribution of $$X$$ and the residuals, $$p(\epsilon, X)$$. But the substitutions he uses en route to his result don't always seem justified. Are they justified?

He starts out with the goal: posterior inference about $$\boldsymbol{\beta}$$, given $$X$$ and $$y$$, using Bayes Theorem

$$p(\boldsymbol{\beta}|y,X)\propto p(y,X|\boldsymbol{\beta})p(\boldsymbol{\beta})$$

Which requires deriving a conditional joint distribution for the data

$$p(y,X|\boldsymbol{\beta})$$

His strategy is to first obtain a joint distribution for $$\epsilon$$ and $$X$$, under the assumption that $$\epsilon$$ and $$X$$ are independent, not just uncorrelated

$$p(\epsilon, X)=p_{\epsilon}(\epsilon)p(X)$$

His next step is to make substitutions into this joint distribution using $$y=X\boldsymbol{\beta}+\epsilon$$, and then condition on $$\boldsymbol{\beta}$$, yielding the required conditional joint distribution

$$p(y,X|\boldsymbol{\beta})=p_{\epsilon}(y-X\boldsymbol{\beta})p(X|\boldsymbol{\beta})$$

I understand the right hand side of this result, simply using $$\epsilon=y-X\boldsymbol{\beta}$$. But what about the left hand side? It can't be a substitution simply of $$y=\epsilon$$, as that doesn't make sense. Can someone help me better understand how this result is justified? Am I missing something basic?

The left-hand side of your final equation, $$p(y,X|\beta$$), can be written as $$p_y(y|X,\beta)p(X|\beta)$$, and writing the right-hand side more explicitly then gives $$p_y(y|X,\beta)p(X|\beta) = p_\epsilon(y - X\beta|X,\beta)p(X|\beta)$$ Your intuition is right in that generally $$p_y(y|X,\beta) \neq p_\epsilon(\epsilon|X,\beta)$$. Instead, you have $$p_y(y|X,\beta) = p_\epsilon(\epsilon|X,\beta) \left| \frac{\mathrm{d} \epsilon}{\mathrm{d} y} \right|$$ where $$\left| \frac{\mathrm{d} \epsilon}{\mathrm{d} y} \right|$$ is known as the Jacobian of the transformation. But in linear regression, $$\epsilon = y - X\beta$$ and therefore $$\frac{\mathrm{d} \epsilon}{\mathrm{d} y} =1$$, which means the Jacobian can be ignored.