Consider the setting where $X_1,X_2,...$ are i.i.d. real-valued random variables with $\mathbb{E}[X_i] = \theta$ and let the random variable $\tau$ be an associated stopping time. I'm wondering what can be said about the bias of the estimator (of $\theta$) $$ \hat{\theta} = \frac{1}{\tau}\sum_{i=1}^{\tau} X_i $$ when certain conditions are placed on the stopping rules. In particular, can we bound this bias by enforcing conditions of the type $\mathbb{P}\{N_0 < \tau < N_1\} = 1$? That is, the stopping rule must take draw at least $N_0$ samples and/or no more than $N_1$.

Intuitively (and empirically) it seems that as $N_0$ increases or as $N_1 - N_0$ decreases the bias $\mathbb{E}[\hat{\theta} - \theta]$ decreases to zero.

I've tried to make use of Wald's general equation which can tell us that, with reasonable assumptions on $\tau$, for the sum $S = \sum_{i=1}^\tau X_i$ we have $\mathbb{E}[S] = \mathbb{E}[\tau]\mathbb{E}[X]$, without much luck.

Edit: Some interesting work has been done looking at stopping rules that maximize this bias pioneered by Robbins and Chow (1965) and extended by Basu and Chow (1977) for a modern review of the general problem see here.

  • $\begingroup$ Why do you think that Wald's general equation applies in your case? In the work you reference by Robbins, Basu, and Chow, I don't believe that Wald's general equation will apply for the types of stopping rules they are interested in. It would help if you clarified your stopping rule which results in $P(N_0<\tau<N_1)=1$. $\endgroup$ – jsk Oct 20 '14 at 17:46
  • $\begingroup$ @jsk I'm interested in general stopping rules, with the only real assumption that they can't anticipate the future. AFAIK the assumptions for Wald's equation essentially translate to this (namely assumption #2 on the wiki page) though it is not particularly straightforward. $\endgroup$ – fairidox Oct 20 '14 at 22:42
  • $\begingroup$ You could try using $$\mathbb{E}[\hat{\theta}] = \mathbb{E}_\tau\Big\{ \frac{1}{\tau} \mathbb{E}\big[ \sum_{i=1}^{\tau} X_i\mid\tau\big]\Big\}$$ and see if your stopping rule $\tau$ allows for a closed-form inner expectation. $\endgroup$ – Xi'an Dec 31 '14 at 10:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.