# Can the maximum a posteriori (MAP) estimator of a Bernoulli parameter be undefined?

The maximum a posteriori estimator of a Bernoulli parameter $\mu$ with $Beta(a,b)$ prior is

$$\hat{\mu}_{MAP}= \frac{n_R + a - 1} {n + a + b - 2}$$

where $n_r = \sum x_i$ the count of all events in the data and $n$ the sample size. Now it is not difficult, I believe, to construct an example where MAP is outside the support of the Bernoulli distribution. Assume $n=n_R$, so that so far all trial results were positive, then for any $b<1$, $\hat{\mu}_{MAP} > 1$.

My question is whether this makes sense in Bayesian logic and whether there is a constraint on $a$ and $b$ needed that I am not aware of. Put differently, is MAP only defined for $b \ge 1$?

• The Beta distribution has support on (0,1) and thus, if all trials were successful and b<1, the MAP is 1. Apr 7, 2017 at 17:43
• @jaradniemi Are you saying MAP is set to 1? Because the formula evaluates to greater than one in that case. Apr 7, 2017 at 17:47
• $\mu \in [0,1]$ implies that the function is only defined on the range $[0,1]$ for this application. As a result, we only look for the maximum over this range, which will, for your example, be at $\mu = 1$. It's not that MAP is just arbitrarily set to $1$ when this happens, it's that the function is only defined over $[0,1]$, so the maximum really is at $1$. Apr 7, 2017 at 18:12

With the slightest hint of provocation, I would like to mention that the MAP estimate is always undefined! Indeed, the maximum of the posterior density function in $p$ depends on the dominating measure. Hence, if one uses the Lebesgue measure as the dominating measure, the density function is$$f(p|x)\propto p^{x+a-1} (1-p)^{n-x+b-1}\mathbb{I}_{(0,1)}(p)$$which may enjoy the value $(x+a-1)/(n+a+b-2)$ as its maximum (MAP) if it is an interior value and else has no maximum in $(0,1)$. However, if one takes the prior measure as the dominating measure, the density function is$$f(p|x)\propto p^{x} (1-p)^{n-x}\mathbb{I}_{(0,1)}(p)$$which enjoys $x/n$ as its maximum(MAP).

• I wish I would understand measure theory. It's on my list. Apr 10, 2017 at 15:05
• This is a very simple part of measure theory: it simply means that one can allocate any part of the (standard) density to the dominating measure, as illustrated by my example... Apr 10, 2017 at 15:24

The only constraints are the ones of parameters of beta distribution, i.e. $\alpha > 0$ and $\beta > 0$ (but recall that there is also an improper Haldane prior with $\alpha = \beta = 0$). Obviously, for calculating the mode itself, you also need $n + \alpha + \beta - 2 > 0$.

• No you don't need $n + \alpha + \beta - 2 > 0$; the mode is just at one end of the interval $[0,1]$ if that doesn't hold. Consider $n = 0, \alpha = \beta = 0.5$. There are two modes, at $0$ and $1$, even though the inequality doesn't hold. For an analogous example, the maximum of the function $y = x, x \in [0,1]$ occurs at $x = 1$. Apr 7, 2017 at 19:32
• Thanks a lot for your answer, but to fully address my question you would need to say what happens in case $\mu_{map}>1$. Apr 7, 2017 at 19:49
• @jbowman let's say it's disputable, cf. en.wikipedia.org/wiki/Beta_distribution#Mode
– Tim
Apr 7, 2017 at 21:02
• @tomka I don't understand what do you mean... mean of Bernoulli distribution cannot be greater then one.
– Tim
Apr 7, 2017 at 21:04
• The MAP cannot be $>1$. You are confusing the output of a particular formula for calculating MAP that works for certain values of the parameters but not for others with the MAP itself. That formula is NOT the UNIVERSAL AND ONLY formula for calculating MAP. It is a formula that works when the MAP is in the interior of $(0,1)$ and not otherwise. This is because it's based on the first and second derivatives of the posterior, but, when optimizing a function, derivative-based formulae don't work if the min (max) value of the function is at an endpoint of the feasible region. Apr 10, 2017 at 16:54

For a start you would need valid parameters for the posterior, e.g. $a+n_R<0$ or $b+(n-n_R)<0$ does not make any sense and really a-priori $a<0$ or $b<0$ is not really a prior belief (the cases with $a=0$ or $b=0$ a-priori or a-posteriori are also problematic). As long as that is the case, you will get a posterior that lies within [0, 1] and a maximum, if it exists lies in this range.

Additionally, you may have cases, where the estimate lies on the boundary of the parameter space (i.e. 0 or 1, e.g. for a $\text{Beta}(1,3)$ posterior), or when there is no unique maximum. The most extreme example of the latter case is a flat $\text{Beta}(1, 1)$ posterior, or with maxima at 0 and 1 for a $\text{Beta}(1/2, 1/2)$. If both $a+n_R>1$ and $b+(n-n_R)>1$, you have a unqiue maximum in $(0,1)$.

• Can you check in your first sentence "$a<0$ or $b<0$" -- Beta is not defined then, so I am not sure what you mean. Apr 10, 2017 at 14:43
• That is exactly my point, there are constraints on $a$ and $b$ in so far as they should be valid parameters for the distribution. Apr 10, 2017 at 16:13