1
$\begingroup$

Say we have $X \sim Unif\{1, \ldots , \theta\}$ and we want to find the uniformly minimum variance unbiased estimator for $\theta$.

My first assumption was $X_{(n)}$. Which I managed to show is complete sufficient and unbiased. Can I conclude that this is UMVUE? Is there a theorem which would say that:

If a statistic is sufficient, complete and unbiased, then it is UMVUE?

Thank you for the information!

CORRECTION: As noted in comment, $X_{(n)}$ is biased.

$\endgroup$
  • 2
    $\begingroup$ Think carefully about whether it is unbiased, given that for any $X_{(n)} < \theta$, your estimate will be low, but there are no (possible) circumstances under which it will be high. $\endgroup$ – jbowman Apr 2 '18 at 1:15
  • $\begingroup$ @jbowman Indeed, the $X_{(n)}$ is bias, however it converges towards $\theta$ almost surely as the sample size increases. What would be a unbiased estimator in this case? $\endgroup$ – rannoudanames Apr 2 '18 at 1:42
  • 1
    $\begingroup$ My hint was pretty useless, my apologies. Try the "real" way: construct an unbiased estimator, not worrying about whether it uses $X_{(n)}$, then calculate its expected value given $X_{(n)}$. A good place to start is seeing if you can construct an unbiased estimator using the sample mean. $\endgroup$ – jbowman Apr 2 '18 at 2:47
  • 1
    $\begingroup$ An important point to note: If your sample size is large enough, asymptotic unbiasedness maybe good enough as the "difference" will go to zero...case and point -- the (1/N) vs (1/N-1) scaling of the sample variance $\endgroup$ – Sid Apr 2 '18 at 3:53
  • 1
    $\begingroup$ A similar idea is at play when considering the asymptotic properties of the MLE $\endgroup$ – Sid Apr 2 '18 at 3:54
2
$\begingroup$

Let's take Sid's answer and work with it. We'll start out with a simple unbiased estimator of $\theta$ and work out its expectation given $X_{(n)}$.

Our simple unbiased estimator will be based on the sample mean, whose expectation is:

$$\mathbb{E}\bar{X} = {\theta + 1 \over 2}$$

We can construct our unbiased estimator $T$ as:

$$T = 2\bar{X}-1$$

Now, given a sample of size $n$ and $X_{(n)}$, the expectation of $T$ can be calculated by noting that, conditional upon $X_{(n)}$,

  1. One of the data points is known to be $X_{(n)}$,
  2. The other $n-1$ data points are distributed according to a discrete uniform distribution on $\{1, \dots, X_{(n)}\}$, which has expectation $(X_{(n)} + 1) / 2$.

So, starting out by calculating $\mathbb{E}[\bar{X}|X_{(n)}]$, which is a valid approach as $T$ is linear in $\bar{X}$,

$$\mathbb{E}\bar{X} = {X_{(n)} + (n-1){X_{(n)} + 1 \over 2} \over n}$$

Rearranging terms gives us:

$$\mathbb{E}\bar{X} = {(n+1)X_{(n)}+(n-1) \over 2n}$$

and

$$\mathbb{E}T = {(n+1)X_{(n)}+(n-1) \over n} - 1$$

resulting, with some further simplification, in our UMVUE $T^*$:

$$T^* = {(n+1)X_{(n)} - 1 \over n}$$

Having done all this, I am sorry to note that this estimator will not, in general, output integer values for its estimates, even though we (I presume) know $\theta$ is an integer. "Unbiased" and "variance" are not terms that take this into account, however, so were we to be in a real-world situation where we had to provide integer estimates of $\theta$, we'd have to do something else - if only (possibly) round off the estimates we get this way.

ETA: Finding the conditional distribution of $X|X_{(n)}$. Given $X_{(n)}$, we know $X \in \{1, \dots, X_{(n)}\}$. The conditional distribution $p(x|x \leq X_{(n)})$ is:

$$ p(x|x \leq X_{(n)}) = {p(x) \over P(X_{(n)})}$$

where $P(X_{(n)})$ is the probability that $x \leq X_{(n)}$.

Now, $P(X_{(n)}) = X_{(n)}/\theta$. Substituting gives:

$$ p(x|x \leq X_{(n)}) = {1/\theta \over (X_{(n)}/\theta)} = {1 \over X_{(n)}}$$

This, combined with the fact that $X \in \{1, \dots, X_{(n)}\}$, is sufficient for us to conclude that $x|x\leq X_{(n)}$ is distributed according to a discrete uniform distribution on $\{1, \dots, X_{(n)}\}$.

$\endgroup$
  • $\begingroup$ Thanks for the VERY comprehensive answer, I am however confused as to how you find the conditional distribution of $P(X|X_{(n)})$. $\endgroup$ – rannoudanames Apr 2 '18 at 4:49
  • 1
    $\begingroup$ Added an edit to address this point. $\endgroup$ – jbowman Apr 2 '18 at 5:06
  • $\begingroup$ Thank You very much for the insight on this, surprising how uniforms on discrete sets can be so tricky! $\endgroup$ – rannoudanames Apr 2 '18 at 5:22
  • $\begingroup$ My calculations of the UMVUE give $\hat\theta=\frac{X_{(n)}^{n+1}-(X_{(n)}-1)^{n+1}}{X_{(n)}^n-(X_{(n)}-1)^n}$ (math.stackexchange.com/questions/2944781/…). Could you recheck? $\endgroup$ – StubbornAtom Oct 2 at 7:46
  • $\begingroup$ In your answer you have assumed sampling is done without replacement (so the $X_i$s are not independent), but if $X_1,\ldots,X_n$ are i.i.d uniform on $\{1,2,\ldots,\theta\}$, then the analysis changes. I think this point should be made clear. $\endgroup$ – StubbornAtom Oct 2 at 8:22
3
$\begingroup$

Please see the Lehmann Scheffe theorem (Theorem 2 in http://www.stat.unc.edu/faculty/cji/lecture9.pdf), which states that if $U$ is a complete sufficient statistic, and $T$ is an unbiased estimator, than $E(T|U)$ is UMVUE. In your case, you have $ T = U $.

Please check your claims about unbiasedness: A variant of your problem is addressed in https://math.stackexchange.com/questions/150586/expected-value-of-max-of-iid-variables?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa, where one of the answers show that the estimate is in fact biased

$\endgroup$
  • 1
    $\begingroup$ At present this is more of a comment than an answer. You could expand it, perhaps by giving a summary of the information at the link, or you can convert it into a comment. $\endgroup$ – gung Apr 2 '18 at 1:17
  • $\begingroup$ Thank you, I have added more information. The reason I had left it out in the beginning, because the theorem answers the question almost exactly in the form in which it is posed. $\endgroup$ – Sid Apr 2 '18 at 1:28
  • $\begingroup$ ... except for the fact that $X_{(n)}$ is not unbiased! $\endgroup$ – jbowman Apr 2 '18 at 1:32
  • $\begingroup$ Ah! My apologies...I took the author's claim " I managed to show ... is complete sufficient and unbiased" at face value :) $\endgroup$ – Sid Apr 2 '18 at 1:36
  • $\begingroup$ Well, you're correct on the merits of the question the OP actually asked, even if he's not going to get the answer right when he turns it in! $\endgroup$ – jbowman Apr 2 '18 at 1:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.