Sampling from Posterior without MCMC

Question

I want to sample from the posterior density $f(x \mid y)$ which is related to the prior density $f(x)$ and likelihood density $f(y \mid x)$ via Bayes' rule: $$ f(x \mid y) = \frac{f(y \mid x) f(x)}{c} $$ where $c$ is a normalizer. Let's say both the prior and likelihood are very simple and I can sample from them directly using standard techniques, e.g. Gaussian. Specifically, say $X \sim \mathcal{N}(\mu, \sigma_1^2)$ and $Y \mid X = x \sim \mathcal{N}(x, \sigma_2^2)$. I can then sample from $f(x)$ directly, and given this sample can obtain a sample from $f(y \mid x)$ as well. Is there some method to use these samples to obtain a sample from $f(x \mid y)$? As I understand, MCMC would be used if the posterior is "difficult" to sample from, but it seems it should be easy in this case.

Answer 1

If $X \sim N(\mu, \sigma^2_1)$ and $Y|X = x \sim N(x, \sigma^2_2)$, then the posterior distribution $X|Y$ is also a normal distribution.

$$X|Y=y \sim N \left( \left(\dfrac{\mu}{\sigma^2_1} + \dfrac{y}{\sigma^2_2} \right)\left(\dfrac{1}{\sigma^2_1} + \dfrac{1}{\sigma^2_2} \right)^{-1}, \left(\dfrac{1}{\sigma^2_1} + \dfrac{1}{\sigma^2_2} \right)^{-1} \right) $$

So this is also easy to sample from. Such an instance of the posterior being from the same family of distributions as the prior is called conjugacy. You can see more combinations to priors and likelihoods that lead to feasible posteriors here: https://en.wikipedia.org/wiki/Conjugate_prior