5
$\begingroup$

I am trying to derive the Bayes estimator. Without getting into the nuts and bolts of the question, basically I have an indicator loss function of the form

$$L(\delta , \theta) = \mathbb{1}\{A\}$$

For an event $A$. I found that $EL(\delta,\theta |X)$ is minimized for almost all $X$ when $\delta$ is a function solely of the hyperparameters of $\theta$'s prior distribution. A theorem in my textbook then implies that such a $\delta$ is the bayes estimator of $\theta$.

On the one hand this seems ridiculous, but then again I figure such a result might be correct since we have a somewhat strange loss function.

I can go into the details of the problem if needed.


To clarify. We have the posterior distribution $\theta | X$ as being a truncated $N(\mu, \sigma^2)$ distribution truncated below at $\max_i X_i$. I want to derive the Bayes estimator under the loss function

$$L_\epsilon (\theta, \delta) = 1 - 1 \{ \delta - \epsilon \leq \theta \leq \delta + \epsilon \}$$

To find the $\delta$ that minimizes this, clearly it is equivalent to find the $\delta$ that maximizes

$$E 1 \{ \delta - \epsilon \leq \theta \leq \delta + \epsilon \}$$

There is also a theorem in my textbook that says it is enough to show that $\delta$ minimizes $E[L(\theta, \delta)|X]$ for almost all $X$ for it to be a Bayes estimator. So to find the Bayes estimator I tried to find the $\delta$ that maximizes

$$E [1 \{ \delta - \epsilon \leq \theta \leq \delta + \epsilon \}|X] = P(\delta - \epsilon \leq \theta \leq \delta + \epsilon | X)$$

Since we know the posterior distribution of $\theta$ is truncated normal we can work with this probability. From here I used the CDF of the truncated normal distribution $F$ given on the Wikipedia page to find the $\delta$ that minimizes

$$F(\delta + \epsilon) - F(\delta - \epsilon)$$

and using $$\frac{d\Phi (f(\delta))}{d\delta} = f'(\delta) \phi(f(\delta))$$ I found was that this is maximized when

$$\delta = \mu$$

which does not depend on $X$ at all. So I'm pretty sure I'm making a stupid mistake, or have misunderstood how to find Bayes estimators.

$\endgroup$
1
  • $\begingroup$ This question is not precise enough, how is $A$ connected with $\delta$ and $\theta$? $\endgroup$
    – Xi'an
    Nov 23, 2018 at 11:08

1 Answer 1

2
$\begingroup$

If one considers the loss function $$L(\delta , \theta) = \mathbb{1}_{|\theta-\delta|>\epsilon}$$ the Bayes estimator $\delta^*(\cdot)$ is obtained as solving$$\arg\min_\delta\mathbb{E}[L(\delta , \theta) | x]$$for every $x$. Now \begin{align*} \mathbb{E}[L(\delta , \theta) | x] &= \int L(\delta , \theta)\pi(\theta | x)\,\text{d}x\\ &= \int \mathbb{1}_{|\theta-\delta|>\epsilon}\,\text{d}x\\ &= \mathbb{P}(|\theta-\delta|>\epsilon)\\ &= 1-\pi([\delta-\epsilon,\delta+\epsilon] | x) \end{align*} One thus wants to maximise the coverage of $[\delta-\epsilon,\delta+\epsilon]$ under the posterior distribution. This amounts to finding the location $\delta$ of the center of an interval of length $2\epsilon$ with the largest coverage. A graphic illustration is to imagine a stick of length $2\epsilon$ that is moved up under the graph of $\pi(\cdot|x)$ until it cannot move higher. It obviously depends on $\pi(\cdot)$ and on $f(x|\theta)$.

Note that differentiating $\pi([\delta-\epsilon,\delta+\epsilon] | x)$ in $\delta$ leads to$$\pi(\delta-\epsilon)=\pi(\delta+\epsilon)$$meaning that the resulting interval is an HPD region if the posterior is unimodal and monotone on each side of the posterior mode (MAP). It is only when the posterior is symmetric that $\delta^*$ is the MAP. Note also that when the posterior support is truncated, as in the truncated Normal example, the solution is an HPD but does not necessarily satisfy the equation$$\pi(\delta-\epsilon)=\pi(\delta+\epsilon)$$. For instance, if the posterior is the Normal distribution $\text{N}(\mu,\sigma^2)$ truncated below $\max x_i>\mu$, $\delta^*$ will be equal to $\max x_i+\epsilon$.

$\endgroup$
2
  • 1
    $\begingroup$ Thanks Xi'an, it is good to know the formal reasoning. I will have a think about how to deal with the truncation in order to maximize the coverage. I think my mistake before is assuming that $\mu$ is in the support of the posterior? when there's no guarantee that is the case. $\endgroup$
    – Xiaomi
    Nov 23, 2018 at 14:17
  • $\begingroup$ After working through it I found the solution is a lot closer to what I was expecting. Thanks again $\endgroup$
    – Xiaomi
    Nov 24, 2018 at 6:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.