# Estimate distribution of variable from non-perfect predictions

Say I pull $$n$$ balls from a box while blinfolded. The balls can be either red or blue. I do not know the distribution of the balls.

After that I receive a list predicting the color of every ball I pulled out. I am also told that every prediction has a $$70\%$$ chance of being right and $$30\%$$ chance of claiming the wrong color (if blue it says red and vice versa).

Is there any way I could estimate within a range of error the true distribution of the balls? Or estimate the most likely distribution of the colors of the balls I pulled after the predictions I have been given?

• Is this a question from a course or textbook? If so, please add the self-study tag & read its wiki. Commented May 30, 2022 at 13:03
• Not really. I trained a semi-supervised model that predicts whether a variable is "yes" or "no" and after comparing the predictions it gives to the true value I noticed that it assigns the correct value a 70% of the time. So I wondered about the posed question. Training the model was part of a course yes, but the question itself is not. I was just curious.
– st30
Commented May 30, 2022 at 13:50

Try to find a likelihood function, and treat it as a likelihood estimation problem. Lets say you have an urn with $$N$$ balls, of which you draw $$n$$, say without replacement. That makes this a hypergeometric problem, with some extras ... But for now, assume $$n << N$$ and treat it as a binomial problem first. The extension I leave as an exercise. Define some random variables, and let $$\theta$$ be the fraction of red balls in the urn:
First some unobserved random variables, the true colors of the balls: $$\newcommand{\red}{\color{red}{\text{red}}} \newcommand{\blue}{\color{blue}{\text{blue}}} \DeclareMathOperator{\P}{\mathbb{P}} \P(\tilde{X}_i=\red)=\theta \\ \P(\tilde{X}_i=\blue)=1-\theta$$ Then the variables we actually observe: $$\P(X_i=\tilde{X}_i) = 0.7, \quad \P(X_i \not=\tilde{X}_i) = 0.3$$ And by conditioning we find the distribution of $$X_i$$ in terms of $$\theta$$: $$\P_\theta(X_i=\red)= \\ \P_\theta(X_i=\red \mid \tilde{X}_i=\red) \P(\tilde{X}_i=\red) + \P_\theta(X_i=\red \mid \tilde{X}_i=\blue) \P(\tilde{X}_i=\blue) = \\ 0.7 \theta + 0.3 (1-\theta)$$ and likewise (or by subtraction) $$\P_\theta(X_i=\blue)= 0.7 (1-\theta) + 0.3 \theta$$ Then the likelihood function becomes $$\mathcal{L}(\theta)=\left( 0.7 \theta + 0.3 (1-\theta)\right)^{n_{\red}}\cdot \left( 0.7 (1-\theta) + 0.3 \theta \right)^{n_{\blue}}$$ From now, usual likelihood methods. You can then do the same within the hypergeometric upset.