Problem 8.7 From Van der Vaart's Asymptotic Statistics:

Given a sample of size $n$ from the uniform distribution on $[0,\theta]$, the maximum $X_{(n)}$ of the observations is biased downwards. Because $\text{E}[\theta-X_{(n)}] = \text{E}[X_{(1)}]$, the bias can be removed by adding the minimum of the observations. Is $X_{(1)} + X_{(n)}$ a good estimator for $\theta$ from an asymptotic point of view?


The chapter is on efficiency of estimators (e.g. convolution theorem, relative efficiency) so my first thought was to calculate the asymptotic variance of $X_{(1)} + X_{(n)}$. From previous results in the class I have $$\text{Pr}\left\{ nX_{(1)} < x\right\} \to 1 - e^{-x/\theta}\qquad \text{Pr}\left\{ -n(X_{(n)}-\theta) < x\right\} \to 1-e^{-x/\theta}$$ that is, the asymptotic marginal distributions of the minimum and the maximum are exponential distributions. However, to get the asymptotic distribution of the sum I would need the joint asymptotic distribution. I'm not sure how to proceed.


Based on the linked question, if $X_{(1)}$ and $X_{(n)}$ are asymptotically independent then the asymptotic variance of $X_{(1)} + X_{(n)}$ would simply be $$\theta^2+\theta^2 + 2(0) = 2\theta^2$$ which is larger than the variance of the MLE ($X_{(n)}$). But the MLE is biased. Also, I have not found a way to show that the minimum and maximum are asymptotically independent.

Related: with no satisfactory answer Asymptotic distribution of uniform order statistics

Also somewhat related to the famous german tank problem, but that is for discrete uniform distribution.

  • $\begingroup$ You might compare the asymptotic variance of $X_{(1)}+X_{(n)}$ with that of $\frac{n+1}{n}X_n$ $\endgroup$ – Glen_b Dec 10 '14 at 5:25
  • $\begingroup$ Could you elaborate? By $X_n$ do you just mean the last observation or the maximum? $\endgroup$ – bdeonovic Dec 10 '14 at 5:32
  • $\begingroup$ A typo. I mean $X_{(n)}$. Your complaint about $X_{(n)}$ was that it was biased (my reaction would be 'so?'); I was suggesting an obvious simple modification of it. [Personally I don't see why MMSE would not be preferable to forcing oneself to consider only unbiased estimators and then looking for small variance, but that's not so relevant to your question.] --- this should probably carry a self-study tag, btw $\endgroup$ – Glen_b Dec 10 '14 at 5:34
  • $\begingroup$ whats MMSE? Is this problem basically illustrating another example of bias/variance tradeoff? $\endgroup$ – bdeonovic Dec 10 '14 at 5:45
  • $\begingroup$ "minimum mean square error". I don't know what your problem is trying to illustrate, but my comment about MMSE certainly relates to it. $\endgroup$ – Glen_b Dec 10 '14 at 6:25

Since this is for self study, I've decided to make this intentionally terse. First, when $\theta = 1$, show that $$ (X_{(1)}, X_{(n)} - X_{(1)},1 - X_{(n)}) \sim \operatorname{Dirichlet}(1, n-1, 1). $$ More generally, you might show $$ (X_{(1)}, X_{(2)} - X_{(1)}, \ldots, X_{(n)} - X_{(n-1)}, 1 - X_{(n)}) \sim \operatorname{Dirichlet(1, 1, \ldots, 1, 1).} $$ Then, we have $$\hat \theta = -(1 - X_{(n)}) + X_{(1)} + 1.$$ Using the Dirichlet distribution given above, we have immediately that $$\operatorname{Var}(\hat \theta) = \frac{2}{(n+1)(n+2)}.$$ Hence, for arbitrary $\theta$, $$\operatorname{Var}(\hat \theta) = \frac{2\theta^2}{(n+1)(n+2)}$$

To assess whether this is "good" or not, compare it to the UMVUE $\tilde \theta = [(n+1)/n]X_{(n)}$.

| cite | improve this answer | |
  • $\begingroup$ $\hat\theta$ is not $(1 - X_{(n)}) + X_{(1)} - 1$ but $X_{(n)}) + X_{(1)}$. $\endgroup$ – Xi'an Dec 10 '14 at 6:44
  • $\begingroup$ @Xi'an whoops, I forgot a minus sign; should be $\hat \theta = -(1 - X_{(n)}) + X_{(1)} + 1 = X_{(n)} + X_{(1)}$. I also initially had the Dirichlet distribution wrong. $\endgroup$ – guy Dec 10 '14 at 6:48
  • $\begingroup$ This looks like a really interesting approach! A few questions: what inspired the connection to the Dirichlet? Is that something that holds true only in this particular case or more generally? How do you make the jump from $\theta=1$ to arbitrary $\theta$? $\endgroup$ – bdeonovic Dec 10 '14 at 13:05
  • $\begingroup$ @Benjamin I knew the order statistics were (scaled) Beta random variables, 1 - Beta flips the parameters, and then guessed that the "spacings" had a joint Dirichlet distribution because it gave the Beta result. It turns out this is well-known though. I don't think it is more general, but I'm not sure. To jump from $\theta = 1$ to arbitrary, just multiply and divide all the order statistics by $\theta$ since $X_{(j)}/\theta$ is the order statistic of a uniform on $[0,1]$ $\endgroup$ – guy Dec 10 '14 at 18:12
  • $\begingroup$ Excellent, this was a very fun exercise thank you! $\endgroup$ – bdeonovic Dec 11 '14 at 3:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.