Can the maximum a posteriori (MAP) estimator of a Bernoulli parameter be undefined?

Question

The maximum a posteriori estimator of a Bernoulli parameter $\mu$ with $Beta(a,b)$ prior is

$$\hat{\mu}_{MAP}= \frac{n_R + a - 1} {n + a + b - 2}$$

where $n_r = \sum x_i$ the count of all events in the data and $n$ the sample size. Now it is not difficult, I believe, to construct an example where MAP is outside the support of the Bernoulli distribution. Assume $n=n_R$, so that so far all trial results were positive, then for any $b<1$, $\hat{\mu}_{MAP} > 1$.

My question is whether this makes sense in Bayesian logic and whether there is a constraint on $a$ and $b$ needed that I am not aware of. Put differently, is MAP only defined for $b \ge 1$?

Answer 1

With the slightest hint of provocation, I would like to mention that the MAP estimate is always undefined! Indeed, the maximum of the posterior density function in $p$ depends on the dominating measure. Hence, if one uses the Lebesgue measure as the dominating measure, the density function is$$f(p|x)\propto p^{x+a-1} (1-p)^{n-x+b-1}\mathbb{I}_{(0,1)}(p)$$which may enjoy the value $(x+a-1)/(n+a+b-2)$ as its maximum (MAP) if it is an interior value and else has no maximum in $(0,1)$. However, if one takes the prior measure as the dominating measure, the density function is$$f(p|x)\propto p^{x} (1-p)^{n-x}\mathbb{I}_{(0,1)}(p)$$which enjoys $x/n$ as its maximum(MAP).

Answer 2

The only constraints are the ones of parameters of beta distribution, i.e. $\alpha > 0$ and $\beta > 0$ (but recall that there is also an improper Haldane prior with $\alpha = \beta = 0$). Obviously, for calculating the mode itself, you also need $n + \alpha + \beta - 2 > 0$.

Answer 3

For a start you would need valid parameters for the posterior, e.g. $a+n_R<0$ or $b+(n-n_R)<0$ does not make any sense and really a-priori $a<0$ or $b<0$ is not really a prior belief (the cases with $a=0$ or $b=0$ a-priori or a-posteriori are also problematic). As long as that is the case, you will get a posterior that lies within [0, 1] and a maximum, if it exists lies in this range.

Additionally, you may have cases, where the estimate lies on the boundary of the parameter space (i.e. 0 or 1, e.g. for a $\text{Beta}(1,3)$ posterior), or when there is no unique maximum. The most extreme example of the latter case is a flat $\text{Beta}(1, 1)$ posterior, or with maxima at 0 and 1 for a $\text{Beta}(1/2, 1/2)$. If both $a+n_R>1$ and $b+(n-n_R)>1$, you have a unqiue maximum in $(0,1)$.