Sampling from an approximate distribution to estimate posterior mean

Question

Suppose I have a model with parameters $\theta = (\theta_1,\theta_2)$ and observed data $x=(x_1,x_2,\ldots,x_k)$. I want to estimate the posterior mean $\hat{\theta}_1 = \mathbb{E}[\theta_1|x]$.

The issue is that I can't sample from $p(\theta_1|x)$ to estimate $\hat{\theta}_1$ by the sample mean because I do not count on an expression for the likelihood $p(x|\theta_1)$. I do, however, count on an expression for $p(x|\theta_1,\theta_2) = p(x|\theta)$ as well as on a prior $\pi(\theta_1)$. I can also find the maximum likelihood estimator for $\theta_2$ (let's call it $\hat{\theta}_2$).

Let $\hat{p}(x|\theta_1)=p\left(x|\theta_1,\hat{\theta}_2\right)$. Then, I could sample from an approximate posterior distribution

$$\hat{p}(\theta_1|x) = \frac{1}{C}\frac{\hat{p}(x|\theta_1)\pi(\theta_1)}{p(x)}$$

where $C$ is some normalization constant.

If I compute the mean of $N$ samples from $\hat{p}(\theta_1|x)$, would that be a "reasonable estimator" (in any sense) for $\mathbb{E}[\theta_1|x]$? Is there any literature I could resort to that does something similar to this?

Answer 1

The issue with the question is that the expression$$\mathbb E[\theta_1|x]$$is not well-defined:

1. either $(\theta_1,\theta_2)$ is considered a random vector with joint prior $\pi(\theta_1,\theta_2)$, in which case $$\mathbb E[\theta_1|x]=\int \theta_1 \pi(\theta_1,\theta_2|x)\,\text d(\theta_1,\theta_2)$$only depends on $x$ and can be computed

2. or $\theta_1$ is considered a random variable with prior $\pi(\theta_1)$, while $\theta_2$ is an unknown fixed value, in which case $$\mathbb E_{\theta_2}[\theta_1|x]=\int \theta_1 \pi_{\theta_2}(\theta_1|x)\,\text d\theta_1$$ is indexed by $\theta_1$, hence cannot be computed when $\theta_2$ is unknown.