# Convergent Statistic for Bernoulli Random Variable

I am currently working on my senior thesis and have encountered a small statistical problem which I need to solve but I lack the rigorous training to do so with sufficient precision myself.

The problem can be stated as follows: Suppose there are N bernoulli trials, $v_n$, with probability $p_n$ of heads. Suppose further there is for each bernoulli trial a bernoulli "signal" about the outcome of the trial. More precisely, for each $v_n$ there is a bernoulli random variable $s_n$ which has conditional distribution $$[s_n |v_n =1] \sim Be(q_n)$$ $$[s_n |v_n =0] \sim Be(1-q_n)$$ i.e. it reveals the true type of $v_n$ with probability $q_n$ and further assume $q_n>0.5$. Suppose now further that all the $v_n$ are dependent in some sense. Say $|Cov(v_n,v_m)|>\rho_0$ for some $\rho_0 \in \mathbb{R}$ and for all pairs $n,m$. I.e. this inequality holds for all pairs but they do not necessarily have the same correlation coefficient.

My problem now is that, I want to show that if I observe $N \to \infty$ of the $s_n$, then I will with probability $1-\epsilon$ (or $1$?) know the true value of any $v_n$.

Intuitively, I think this should hold, but I lack the references to prove it. So I would very much appreciate if someone could point me in the right direction converning references which may be relevant (or even better, knows how to prove this).

My idea: Maybe we can show that $E[v_n|\mathcal{S}]$ is a consistent estimator? Here $\mathcal{S}$ is the sigma field generated by all the $s_n$. Not sure if this is the right approach or even correct concept though.

EDIT: I am willing to be quite lax in specifying dependence. If there is one concept which is more favorable for proving this than another, I am defenitely willing to give it a go.

• The probabilities $p_n$ and $q_n$ are themselves changing with $n$? Commented Oct 20, 2016 at 14:02
• Yes, however, we are allowed to assume that the $q_n$ are bounded below by $0.5$ (or higher if necessary) Commented Oct 20, 2016 at 14:21
• As a starting comment, you assume pair-wise equicorrelation of the $v$'s, extending to an infinite sample. This allows only for a positive correlation coefficient, $\rho_0>0$. Commented Oct 20, 2016 at 15:04
• I am not sure what equicorrelation means (googled but have not found the definition), but I am only assuming that they are correlated (in particular not independent), not that they all have the same coefficient. Howver, if this is impossible I am willing to relax that assumption to $cov(v_n,v_m)>\rho_0$ (i.e. without absolute value). Commented Oct 20, 2016 at 15:10
• While this is an interesting question, basically a discrete measurement error problem, it is not clear what your end objective is. $\nu_n$ is a random variable, so not clear the sense in which you want to estimate it. Do you mean that you want to estimate its mode or find the MAP estimate? Note that $\mathbb{E}(\nu_n | \mathcal{S})$ is not an estimator, but a feature of the random variable -- in fact a valid objective in the sense I have mentioned. Commented Oct 20, 2016 at 15:38

"My problem now is that, I want to show that if I observe $$N \to \infty$$ of the $$s_n$$, then I will with probability $$1-\epsilon$$ (or $$1$$?) know the true value of any $$v_n$$."

In the comments it has been clarified that the parameters of the model $$p_n, q_n, \rho_{m,n}$$ are a priori known. So you are not after estimating them, but rather you're after predicting/estimating perfectly (almost surely) the random variables $$v_n$$ per se.

Certainly, if the $$v_n$$'s were independent, the only relevant information for some $$v_k$$ would be $$s_k$$, and it is evident that with it alone, no perfect prediction is possible -by applying standard probability rules and expressions, one can obtain that

$$\text{Prob} (v_k =1 \mid s_k =1) = \frac {q_kp_k}{1+2q_kp_k-p_k-q_k} <1$$

So your case rests on the existence of dependence between the $$v_n$$'s.

Let's examine a case a bit more clear and certainly more favorable: assume that we actually observe the sequence of $$\{v_n\}$$'s, and not the sequence of imperfect signals $$\{s_n\}$$'s. Would that be enough to predict the next $$v_n$$ in line?

No: the existence of stochastic/statistical dependence between random variables does not make the one perfectly predictable given the others. It should be the case that the one under prediction is a deterministic function of the others to be able to obtain perfect prediction in principle. And even if we observe an "infinite" sequence of $$\{v_n\}$$, still, the next $$v_n$$ in line could not be predicted perfectly -an inherent "randomness/unpredictability" will always remain, exactly because $$v_n$$ is a non-degenerate random variable.

...and in your case, we are in a worse situation: we only observe the sequence of imperfect signals $$\{s_n\}$$ -an additional source of randomness/imperfection exists here.

Concepts such as asymptotic consistency usually refer to the ability, when information accrues beyond bound, of estimating perfectly an unknown constant, not a non-degenerate random variable. When the target is a random variable, then we talk about a "predictor", not an "estimator", and we can still use the concept of a "consistent predictor", although we would better make the clarification.

• Thank you for the illuminating answer. If I understand it correctly my problem is essentially that I have infinitely many distinct sources of randomness so there is as much randomness as there are signals. This is something that is not crucial to my model and that I could live with doing away with. Suppose instead that all the $v_n$ are deterministic (Borel) functions of finitely many random variables $x_i$, so that $v_n=f_n(x_1,...,x_m)$ for some $m\in \mathbb{N}$ where the $f_n$ are distinct. Could I then obtain "perfect prediction" of the $v_n$? Commented Oct 21, 2016 at 13:10
• This moves the problem on step back, because now, you must be able to obtain the values of the $x_i$'s, which are also random variables (plus the functional form of the $f$'s). What is the relation of the $x_i$'s with the available data? Commented Oct 21, 2016 at 13:15
• I think we should be able to assume that the $f_n$ are indicator functions for events like $(x_1,...,x_m)\geq (a,b, ...,d)$. Ideally, I wouldn't want to make any assumptions on the $x_i$ but for the sake of arguments, lets say they are normally or log-normally distributed. Commented Oct 21, 2016 at 13:24
• OK, so I realize this question has evolved quite a bit during these discussions (which I am very thankful for). Do you think it would be better to start a new more specific question? I feel that your answer deserves to be accepted since it answers what I originally had in mind. Commented Oct 21, 2016 at 15:30
• @ZIM I am glad I could help a bit. Yes, I think that you should attempt to formulate a more specific question. Just keep in mind that what you describe is a monster: the existence of interdependence that does not fade away not even asymptotically, is a situation where very few results exist to my knowledge. Commented Oct 21, 2016 at 15:36