# MLE for simulated case of binomial p with constant labelling rate

I simulated the following example: 2000 binomial runs (p(Heads)=0.6). Each run has a sample size ranging from 500 to 300 0. (But that is not important: we have the same result if the sample size is the same). We imagine that there is some constant counting error among the tails, the heads are over-reported. For example, if in a run, there are N=2000 coins, the actual number of heads is 1205, but the reported number of heads is 1205 + (2000-1205)*0.10, 0.10 being the constant over-counting error.

My question is : can MLE recover the p and the error rate (0.6 and 0.1) in this case? I have tried doing this, but the global MLE (by trial and error) is at 0.66 (with error rate=0). Can I set up my MLE, or use any other way to recover the 0.6 and 0.1 ?

No, it is not possible to estimate the overcounting error rate $$e$$. This experiment behaves essentially like a Binomial experiment in which the erroneous "heads" cannot be distinguished from true heads and therefore the proportion of erroneous heads cannot be estimated.

### Alternative interpretations

There is an ambiguity concerning the observations: how, exactly, would "$$1205 + (2000-1205)*0.10$$" be reported? As $$1284.5$$? If so, we can obtain a lot of information from that decimal! For instance, suppose the true error rate is $$e=0.103427779$$. Then when $$1205$$ heads appear, the reported amount would be $$1287.225084305$$ and when $$1197$$ heads appear the amount reported would be $$1280.052506537$$. There are only two feasible values of $$e$$ consistent with these results: $$0.103427779$$ and $$0.5517138895$$ (which can be found by brute force, solving $$e$$ for all the possible values of $$k$$ between $$0$$ and $$1287$$ and again solving $$e$$ for all the possible values of $$k$$ between $$0$$ and $$1280$$ and finding which solutions appear in both sets). With a third sample we are likely to be able to settle which of these possible values is the correct one--and to know it entirely without error.

If instead the observations are rounded, the analysis becomes more complex. It would appear we could exploit this rounding somehow, but the fact of the matter is that for the sample sizes posited in the question, the experiment behaves essentially in the way analyzed below where the errors are applied randomly and independently with probability $$e$$. One can indeed write down a likelihood. I did so in a comment to another answer:

Consider the probability of observing $$k$$ heads in $$n$$ trials. Due to the mislabeling, $$k=k'+e(n-k')$$ (rounded) where $$k'$$ is the number of heads that actually occurred. Therefore $$\Pr(k)=\sum_j \binom{n}{j}p^j(1-p)^{n-j}$$ where $$j$$ ranges over all integers for which $$\text{round}(j+e(n-j))=k$$. The likelihood of a sequence of independent experiments is the product of these values.

However, this leads to a spiky likelihood function (with probability spikes wherever there are two or more outcomes $$k$$ and $$k'$$ for which $$k+e(n-k)$$ and $$k'+e(n-k')$$ round to the same value). The spikes provide information comparable to that described in the preceding paragraph, but this information is "smeared" by the rounding as well as by the varying sample sizes. That smearing out causes the likelihood to reduce, for all practical purposes, to the binomial experiment described below.

Both these interpretations feel highly artificial. One leads to a number-theoretic solution and the other leads to one with pathological behavior in its likelihood function. Instead, it is plausible that the overcounting is a random process: each tail independently has a chance of $$e$$ of being mistaken for a heads. (This is an example of an asymmetric binary channel in communications theory.) For this situation there is a clear, convincing analysis, as follows.

### Model 1

A model for flipping the coin is a box full of tickets on which "heads" is written on an unknown proportion $$p$$ of the tickets and on the remaining tickets "tails" is written. After pulling a ticket randomly from this box, if it reads "tails" the experimenter goes to a second box containing tickets reading either "keep" and "change" and pulls one of them at random. If it reads "change," the word "tails" is erased from the first ticket and replaced by "heads," which is recorded as an observation.

Normally we would ask that all tickets be restored to their original states and replaced in their respective boxes before we repeat this experiment, for otherwise the proportions in each box will change slightly each time, distorting the model. However, when the boxes have enormous numbers of tickets, these slight changes are inconsequential. So bear with me and imagine conducting this experiment from the first box without replacement.

### Model 2

The argument hinges on noting that it does not matter when in the experiment words are written on the tickets: all that matters is what is finally read off of the tickets that are drawn. We may create an equivalent model, then, by first extracting all the tickets with "tails" from the first box and, for each one of them, separately drawing a ticket from the second box, performing its stipulated action, and replacing the possibly altered ticket into the first box. This causes some proportion $$e$$ of the "tails" tickets to be erased and "heads" to be written on them. The first box now has a proportion $$p + e(1-p)$$ of "heads" tickets in it due to this rewriting step. The experiment proceeds by drawing $$n$$ tickets randomly from this doctored box.

### The solution

I hope it's clear that the two models are the same: they differ only in that in the second model, the erasures occur earlier in the process. The point is that the second model consists of draws from a single box: this is a binomial experiment. As such we can estimate the box's proportion $$p + e(1-p)$$ but, no matter how many times we repeat this experiment, we cannot estimate $$p$$ or $$e$$ separately.