# MSE of the Jackknife Estimator for the Uniform distribution

The Jackknife is a resampling method, a predecessor of the Bootstrap, which is useful for estimating the bias and variance of a statistic. This can also be used to apply a "bias correction" to an existing estimator.

Given the estimand $$\theta$$ and an estimator $$\hat\theta \equiv \hat\theta(X_1, X_2, \cdots X_n)$$, the Jackknife estimator (with respect to $$\hat\theta$$) is defined as $$\hat\theta_J = \hat\theta + (n-1)\left(\hat\theta - \frac{1}{n}\sum_{i=1}^n\hat\theta_{-i}\right),$$

where the $$\hat\theta_{-i}$$ terms denote the estimated value ($$\hat\theta$$) after "holding out" the $$i^{th}$$ observation.

Let $$X_1, X_2, \cdots X_n \stackrel{\text{iid}}{\sim} \text{Unif}(0, \theta)$$ and consider the estimator $$\hat\theta = X_{(n)}$$ (i.e. the maximum value, also the MLE). Note that

$$\hat\theta_{-i} = \begin{cases} X_{(n-1)}, & X_i = X_{(n)} \\[1.2ex] X_{(n)}, & X_i \neq X_{(n)} \end{cases}.$$

Thus the Jackknife estimator here can be written as a linear combination of the two largest values

\begin{align*} \hat\theta_J &= X_{(n)} + \frac{n-1}{n}\left(X_{(n)} - X_{(n-1)}\right) \\[1.3ex] &= \frac{2n-1}{n}X_{(n)} - \frac{n-1}{n} X_{(n-1)}. \end{align*}

What is the bias, variance and mean square error ?

• One thing to bear in mind here is that $\hat{\theta} = X_{(n)}$ is a terrible estimator to begin with. It is more usual to use a bias-corrected version of the original estimator. For that reason, if you are going to employ the jackknife technique, I would recommend applying it to the bias-corrected version of the initial estimator. A jackknife estimator based on an unbiased estimator is also unbiased, so that makes the problem significantly easier (and improves the estimator).
– Ben
Apr 6, 2020 at 23:36
• @Ben Thanks for pointing that out - I'll add this to answer. For the record, I'm not actually planning to use this estimator. Needed this Q/A as a reference for some notes I'm writing. Apr 7, 2020 at 2:52
• @Ben, Question: If I recall correctly, the bias-adjusted MLE for this problem is the UMVUE. So in what sense does the Jackknife give an improved estimator for this problem? Oct 4, 2021 at 20:18

It is well known that the order statistics, sampled from a uniform distribution, are Beta-distributed random variables (when properly scaled). $$\frac{X_{(j)}}{\theta} \sim \text{Beta}(j, n+1-j)$$ Using standard properties of the Beta distribution we can obtain the mean and variance of $$X_{(n)}$$ and $$X_{(n-1)}$$.

### Bias

\begin{align*} E\left(\hat\theta_J\right) &= \frac{2n-1}{n}E(X_{(n)}) - \frac{n-1}{n}E(X_{(n-1)}) \\[1.3ex] &= \frac{2n-1}{n}\frac{n}{n+1}\theta - \frac{n-1}{n}\frac{n-1}{n+1}\theta \\[1.3ex] &= \frac{n(n+1) - 1}{n(n+1)} \theta \end{align*}

Therefore the bias of $$\hat\theta_J$$ is given by $$\text{Bias}_\theta(\hat\theta_J) = \frac{-\theta}{n(n+1)}$$

### Variance

Note: Covariance is derived here. \begin{align*} \text{Var}\left(\hat\theta_J\right) &= \frac{(2n-1)^2}{n^2}\text{Var}(X_{(n)}) + \frac{(n-1)^2}{n^2}\text{Var}(X_{(n-1)}) - 2 \frac{2n-1}{n}\frac{n-1}{n}\text{Cov}(X_{(n)}, X_{(n-1)}) \\[1.3ex] &= \frac{(2n-1)^2}{n^2}\frac{n\theta^2}{(n+1)^2(n+2)} + \frac{(n-1)^2}{n^2}\frac{2(n-1)\theta^2}{(n+1)^2(n+2)} - \frac{2(2n-1)(n-1)}{n^2}\frac{(n-1)\theta^2}{(n+1)^2(n+2)} \\[1.5ex] \end{align*} $$= \frac{(2n^2-1)\theta^2}{n(n+1)^2(n+2)}$$

### MSE

Using the decomposition $$\text{MSE}_\theta(\hat\theta) = \text{Bias}^2_\theta(\hat\theta) + \text{Var}(\hat\theta)$$, we have

\begin{align*} \text{MSE}_\theta(\hat\theta_J) &= \left(\frac{-\theta}{n(n+1)}\right)^2 + \frac{(2n^2-1)\theta^2}{n(n+1)^2(n+2)} \\[1.3ex] &= \frac{2(n-1+1/n)\theta^2}{n(n+1)(n+2)}\\[1.3ex] &= \mathcal O(n^{-2}) \end{align*}