Clarification of Monte Carlo method

Question

I am reading about the Monte Carlo method for the first time and I would like some clarification. Here is what I understand so far:

We are interested in the value of the parameter $\theta$. We take a random sample from the population that is characterized by $\theta$, and using the values of the sample in conjunction with our prior distribution for $\theta$, we determine a posterior distribution for $\theta$, given the values $y_1, ..., y_n$ from our sample.

I read that the next step of the process is to randomly sample $s$ values from the distribution $p(\theta|y_1,...y_n)$, and use these values to approximate $\theta$. My question is, why do we bother with this posterior sampling if we already know $p(\theta|y_1,...y_n)$?

Answer 1

We are interested in the value of the parameter $\theta$. We take a random sample from the population that is characterized by $\theta$, and using the values of the sample in conjunction with our prior distribution for $\theta$, we determine a posterior distribution for $\theta$, given the values $y_1, ..., y_n$ from our sample.

I read that the next step of the process is to randomly sample $s$ values from the distribution $p(\theta|y_1,...y_n)$, and use these values to approximate $\theta$. [...]

As stated like this, it doesn't make much sense, since if you know $\theta$, you obviously don't need to approximate it. Usually the scenario is different: you are able to define likelihood function $p(y_1,\dots,y_n|\theta)$ and prior $p(\theta)$, what enables you to sample from the posterior distribution. This is not the same as knowing the posterior distribution, since to achieve this, you would need to take integrals and this may not be straightforward. Instead of taking those integrals, you could simply take samples from the posterior distribution and calculate empirical statistics over the samples, to get approximate estimates of the statistics of $\theta$. That is what Monte Carlo is about.

Answer 2

If the question is

"why use simulation when the function $p(\theta|y_1,\ldots,y_n)$ is available in closed form?"

the issue has been considered repeatedly on this forum. And elsewhere. The manipulation of the posterior for Bayesian inference is a mathematical evidence, in the sense that any related quantity like $$\theta(p_\theta)=\int_\Theta h(\theta) p(\theta|y_1,\ldots,y_n)\text{d}\theta$$is uniquely and exactly defined; however it does not mean the Bayesian procedures are available in practice. Computing integrals related to this density $p(\cdot|y_1,\ldots,y_n)$ is usually analytically infeasible. Which explains for the recourse to Monte Carlo techniques.

A related school of questions deals with the missing constant in $p(\theta|y_1,\ldots,y_n)$, since the "missing" constant is not "missing" in a mathematical sense, given that it is uniquely defined by the constraint that the integral of the posterior density function integrates to one (1).