# How to derive the form of the posterior for regression?

I have seen the general form of posterior for a regression $$y = f(x)$$ defined as $$P(\theta|y,x) = \frac{ P(y | x, \theta) P(\theta) }{ P(y|x) }$$

I would like to know how to arrive at this form, starting from Bayes rule and the laws of probability.

The approach I tried is to write Bayes theorem notated as $$p(\theta|v) = \frac{ p(v|\theta) p(\theta) }{ p(v) }$$ and then try subsituting something for $$v$$.

Approach A. With $$v \rightarrow y|x$$, this gives $$" P(\theta|y|x) = \frac{ P(y|x | \theta) P(\theta) }{ P(y|x) } "$$ and then if there is a rule that $$a|b|c \rightarrow a|b,c$$ it gives the result. However I have not seen such a rule. Does it exist?

Approach B. Start with $$v \rightarrow y$$ giving $$P(\theta|y) = \frac{ P(y | \theta) P(\theta) }{ P(y) }$$ and then assume there is a rule that you can condition every factor on some other variable $$x$$ (see rule below), giving $$P(\theta|y,x) = \frac{ P(y | \theta,x) P(\theta|x) }{ P(y|x) }$$ and lastly assume that $$P(\theta|x) = P(\theta)$$. But here I have not seen a rule that allows $$\text{if} \quad p(A) = P(B)P(C)\cdots \quad\text{then}\quad p(A|X) = P(B|X)P(C|X)\cdots$$ Does such a rule exist?

What is the right approach?

• To write that $\nu=y|x$ does not make sense: $y$ is a random variable that has both a marginal distribution (if irrelevant here) and a conditional distribution given the random variable $x$. The notation $P(y|x|\theta)$ does not make sense either. Jan 2 '19 at 7:53

\begin{align*} P(\theta \mid y,x) &= \frac{P(\theta, y,x)}{P(y,x)} \tag{defn. condtl. prob} \\ &= \frac{P(y \mid \theta, x)P(\theta \mid x)P(x)}{P(y \mid x)P(x)} \tag{defn. condtl. prob}\\ &= \frac{P(y \mid \theta, x)P(\theta \mid x)}{P(y \mid x)} \tag{cancellation}\\ &= \frac{ P(y | x, \theta) P(\theta) }{ P(y|x) } \tag{indep.} \end{align*}
Your initial idea is correct, but you should have used $$x,y$$ as $$v$$. Then $$p(v)=p(x,y)=p(y|x)p(x)$$, and:
$$p(\theta\vert v)=p(\theta\vert x,y) =\frac{p(v\vert\theta)p(\theta)}{p(v)} =\frac{p(y\vert x,\theta)p(x\vert\theta)p(\theta)}{p(y\vert x)p(x)}=\frac{p(y\vert x,\theta)p(\theta)}{p(y\vert x)}$$
Where we used the fact that $$p(x\vert\theta)=p(x)$$