# UMVUE for $\theta$ where $X \sim Unif\{1 ,\ldots, \theta\}$

Say we have $$X \sim Unif\{1, \ldots , \theta\}$$ and we want to find the uniformly minimum variance unbiased estimator for $$\theta$$.

My first assumption was $$X_{(n)}$$. Which I managed to show is complete sufficient and unbiased. Can I conclude that this is UMVUE? Is there a theorem which would say that:

If a statistic is sufficient, complete and unbiased, then it is UMVUE?

Thank you for the information!

CORRECTION: As noted in comment, $$X_{(n)}$$ is biased.

• Think carefully about whether it is unbiased, given that for any $X_{(n)} < \theta$, your estimate will be low, but there are no (possible) circumstances under which it will be high. – jbowman Apr 2 '18 at 1:15
• @jbowman Indeed, the $X_{(n)}$ is bias, however it converges towards $\theta$ almost surely as the sample size increases. What would be a unbiased estimator in this case? – rannoudanames Apr 2 '18 at 1:42
• My hint was pretty useless, my apologies. Try the "real" way: construct an unbiased estimator, not worrying about whether it uses $X_{(n)}$, then calculate its expected value given $X_{(n)}$. A good place to start is seeing if you can construct an unbiased estimator using the sample mean. – jbowman Apr 2 '18 at 2:47
• An important point to note: If your sample size is large enough, asymptotic unbiasedness maybe good enough as the "difference" will go to zero...case and point -- the (1/N) vs (1/N-1) scaling of the sample variance – Sid Apr 2 '18 at 3:53
• A similar idea is at play when considering the asymptotic properties of the MLE – Sid Apr 2 '18 at 3:54

Let's take Sid's answer and work with it. We'll start out with a simple unbiased estimator of $\theta$ and work out its expectation given $X_{(n)}$.

Our simple unbiased estimator will be based on the sample mean, whose expectation is:

$$\mathbb{E}\bar{X} = {\theta + 1 \over 2}$$

We can construct our unbiased estimator $T$ as:

$$T = 2\bar{X}-1$$

Now, given a sample of size $n$ and $X_{(n)}$, the expectation of $T$ can be calculated by noting that, conditional upon $X_{(n)}$,

1. One of the data points is known to be $X_{(n)}$,
2. The other $n-1$ data points are distributed according to a discrete uniform distribution on $\{1, \dots, X_{(n)}\}$, which has expectation $(X_{(n)} + 1) / 2$.

So, starting out by calculating $\mathbb{E}[\bar{X}|X_{(n)}]$, which is a valid approach as $T$ is linear in $\bar{X}$,

$$\mathbb{E}\bar{X} = {X_{(n)} + (n-1){X_{(n)} + 1 \over 2} \over n}$$

Rearranging terms gives us:

$$\mathbb{E}\bar{X} = {(n+1)X_{(n)}+(n-1) \over 2n}$$

and

$$\mathbb{E}T = {(n+1)X_{(n)}+(n-1) \over n} - 1$$

resulting, with some further simplification, in our UMVUE $T^*$:

$$T^* = {(n+1)X_{(n)} - 1 \over n}$$

Having done all this, I am sorry to note that this estimator will not, in general, output integer values for its estimates, even though we (I presume) know $\theta$ is an integer. "Unbiased" and "variance" are not terms that take this into account, however, so were we to be in a real-world situation where we had to provide integer estimates of $\theta$, we'd have to do something else - if only (possibly) round off the estimates we get this way.

ETA: Finding the conditional distribution of $X|X_{(n)}$. Given $X_{(n)}$, we know $X \in \{1, \dots, X_{(n)}\}$. The conditional distribution $p(x|x \leq X_{(n)})$ is:

$$p(x|x \leq X_{(n)}) = {p(x) \over P(X_{(n)})}$$

where $P(X_{(n)})$ is the probability that $x \leq X_{(n)}$.

Now, $P(X_{(n)}) = X_{(n)}/\theta$. Substituting gives:

$$p(x|x \leq X_{(n)}) = {1/\theta \over (X_{(n)}/\theta)} = {1 \over X_{(n)}}$$

This, combined with the fact that $X \in \{1, \dots, X_{(n)}\}$, is sufficient for us to conclude that $x|x\leq X_{(n)}$ is distributed according to a discrete uniform distribution on $\{1, \dots, X_{(n)}\}$.

• Thanks for the VERY comprehensive answer, I am however confused as to how you find the conditional distribution of $P(X|X_{(n)})$. – rannoudanames Apr 2 '18 at 4:49
• Added an edit to address this point. – jbowman Apr 2 '18 at 5:06
• Thank You very much for the insight on this, surprising how uniforms on discrete sets can be so tricky! – rannoudanames Apr 2 '18 at 5:22
• My calculations of the UMVUE give $\hat\theta=\frac{X_{(n)}^{n+1}-(X_{(n)}-1)^{n+1}}{X_{(n)}^n-(X_{(n)}-1)^n}$ (math.stackexchange.com/questions/2944781/…). Could you recheck? – StubbornAtom Oct 2 '19 at 7:46
• In your answer you have assumed sampling is done without replacement (so the $X_i$s are not independent), but if $X_1,\ldots,X_n$ are i.i.d uniform on $\{1,2,\ldots,\theta\}$, then the analysis changes. I think this point should be made clear. – StubbornAtom Oct 2 '19 at 8:22

Please see the Lehmann Scheffe theorem (Theorem 2 in http://www.stat.unc.edu/faculty/cji/lecture9.pdf), which states that if $U$ is a complete sufficient statistic, and $T$ is an unbiased estimator, than $E(T|U)$ is UMVUE. In your case, you have $T = U$.

• ... except for the fact that $X_{(n)}$ is not unbiased! – jbowman Apr 2 '18 at 1:32