D&D - Confidence Interval for enemy armor class In dungeons and dragons, characters and monsters have two properties called Attack Bonus($AB \, \in \, \mathbb{Z}$) and Armor Class($AC \, \in \, \mathbb{N}$). Let $AB_c$ be the character attack bonus and $AC_m$ the monster armor class. When a character attacks a monster, he can either hit or miss. To know the result, a $20$ sided die is cast, and we look at its result $D$. If $D$ is between $2$ and $19$, then the attack hits if
$$ D + AB_c - AC_m \geq 0 \quad,$$
otherwise it misses. If $D=1$, it is also considered a miss. If $D=20$, the attack is called a critical, and always hits.
You know $AB_c$, but do not known $AC_m$. As you perform attacks, you can narrow the possible values of $AC_m$. For instance, assume $AB_c = 7$ and you attacked twice, with $D_1 = 5$ and $D_2=16$. Your game master(GM) tells you that you missed the first attack, but hit the second. Therefore, you can infer that $13 \leq AC_m \leq 23$. Moreover, if you attack again with $D_3 = 15$, and the GM tells you that you missed, then you know for sure that $AC_m = 23$.
If the enemy $AC_m$ is higher than your hit range ($19+AB_c$), then best conclusion is $19 + AB_c \leq AC_m$, you can not know $AC_m$ for sure. Notice you can still hit him when $D=20$.
The Fun part
Assume your GM is a hardcore roleplayer and as such he does not allow you to see the die result! He rolls the dice and looks secretly at the result. The only thing you know about your attack is his description: "As you swing your sword, the enemy raises his shield and blocks!" (you missed the attack), and "You strike swiftly, leaving a scar on your foe, if he lives that is!" (you hit).
In a session, you attacked the monster $n$ times and recorded $h = (h_1, \ldots, h_n)$, a sample vector where $i$-th entry is $h_i = 1$, if you hit, or  $h_i = 0$, if you missed. Given $\alpha \, \in \, [0,1]$, how do you provide a $1-\alpha$ confidence interval (or credible interval) for $AC_m$?
 A: This is my approach to the problem. I used highest posterior intervals because they are straight forward to apply, as we will see.
Solution
First, assume that the records $h_i$ are i.i.d. with distribution $Bern(\theta)$, where $\theta$ is the probability that you hit the enemy. From the explanation of the combat system, we have that $\theta$ belongs to a discrete space $\Theta = \{\frac{i}{20}: i\,\in\,\{1,\ldots,19\}\}$.
Since we are interested in $AC$, we have to relate these quantities. Notice that if $AC \leq AB + 2$, then we have $\theta = \frac{19}{20}$. If $AC \geq AB+20$, then we have $\theta = \frac{1}{20}$. If $AB + 2 \leq AC \leq AB+20$, then $\theta = \frac{21 - (AC-AB)}{20}$. Therefore, we can create a bijection relating $\theta$ to $AC$ as long as restrict the latter to $\{AB+2, \ldots, AB+20\}$. That is, we identify all numbers smaller than $AB+2$ with $AB+2$, and all numbers greater than $AB+20$ to $AB+20$. With this identification, we have
$$ AC = 21 - 20\theta + AB \quad.$$
We consider a uniform prior for $\theta$ in $\Theta$. This prior is actually okay: it attributes probability $1/19$ for an enemy that you only hit on criticals ($AC \geq 20+AB$), $1/19$ for a ridiculously easy to hit enemy ($AC \leq AB + 2$), and is uniform on the values in between. Write $\theta_j = \frac{j}{20}$ and $AC_j = 21 - j + AB$, then
$$
\mathbb{P}(AC = AC_j) = \mathbb{P}(\theta = \theta_j) = \frac{1}{19} \quad.
$$
Given $H$, let $s = \sum_{j=1}^{19} h_i$. Given $\theta\,\in\,\Theta$, the likelihood is
$$
L(\theta; H) = {n\choose s}\theta^{s}(1-\theta)^{n-s} \quad.
$$
We can update the posterior by
$$
\mathbb{P}(\theta = \theta_j|H) = \frac{L(\theta_j; H)\mathbb{P}(\theta_j)}{\mathbb{P}(H)} \quad,
$$
where ${\mathbb{P}(H)} = \sum_{j=1}^{19} L(\theta_j|H)\mathbb{P}(\theta_j)$.
Notice that, since $\Theta$ is finite (only has $19$ elements) and $L$ is easy to evaluate, it is pretty straight forward to compute the posterior.  Therefore, we can compute the HPD easily as well.
Example
Consider $AB = 5$, $AC = 21$, $n = 20$, number of hits $s = 6$, and $\alpha = 0.05$. The figure below is a plot of the posterior, the vertical red line is the true value of the $AC$ and the points in purple belong to the HPD. Our HPD consists of the integers in $[16, 22]$, and the estimation of $AC$ using the posterior mode is $\widehat{AC} = 20$.

And here is a reproducible R code for the data simulation and the HPD figure.
## Setup parameters and priori
set.seed(42)
AB = 5 # attack bonus
AC = 21 # armor class
AC_space = seq(AB + 2, AB + 20)

theta = min(max(1/19, (21-(AC-AB))/20), 19/20)
theta_space = seq(1/20, 19/20, 1/20)
priori = rep(1/19, 19)

alpha = 0.05
## Utility function
build_like = function(n, s){
  like = function(theta){
    ll = choose(n, s)*(theta^(s))*((1-theta)^(n-s))
    return(ll)
  }
  return(like)
}

## Simulation and like function
n_samples = 20
h = rbinom(n_samples, 1, theta)

## Posteriori Update
s = sum(h)
like = build_like(n_samples, s)
like_vec = unlist(lapply(theta_space, like))
posteriori = priori*like_vec
posteriori = posteriori/sum(posteriori)

## HPD
hpd_dens = 0
ind = order(posteriori, decreasing = TRUE)
hpd_coloring = rep("black", 19) # a color vector for hpd coloring
for(i in ind){
  hpd_dens = hpd_dens + posteriori[i]
  hpd_coloring[i] = "purple" # will be purple if on hpd and black otherwise
  if(hpd_dens >= 1-alpha){break}
}

## Plotting
# The posteriori and the AC_space are in reverse order
subtitle_msg = paste("n =", n_samples, "hpd_density =", round(hpd_dens, 3))
plot(AC_space, rev(posteriori), col = rev(hpd_coloring),
     xlab = "AC", ylab = "Posteriori",
     sub = subtitle_msg)
abline(v = AC, col = "red")
title("Posteriori and HPD for AC class")

