3
$\begingroup$

The Jackknife is a resampling method, a predecessor of the Bootstrap, which is useful for estimating the bias and variance of a statistic. This can also be used to apply a "bias correction" to an existing estimator.

Given the estimand $\theta$ and an estimator $\hat\theta \equiv \hat\theta(X_1, X_2, \cdots X_n)$, the Jackknife estimator (with respect to $\hat\theta$) is defined as $$\hat\theta_J = \hat\theta + (n-1)\left(\hat\theta - \frac{1}{n}\sum_{i=1}^n\hat\theta_{-i}\right),$$

where the $\hat\theta_{-i}$ terms denote the estimated value ($\hat\theta$) after "holding out" the $i^{th}$ observation.


Let $X_1, X_2, \cdots X_n \stackrel{\text{iid}}{\sim} \text{Unif}(0, \theta)$ and consider the estimator $\hat\theta = X_{(n)}$ (i.e. the maximum value, also the MLE). Note that

$$\hat\theta_{-i} = \begin{cases} X_{(n-1)}, & X_i = X_{(n)} \\[1.2ex] X_{(n)}, & X_i \neq X_{(n)} \end{cases}.$$

Thus the Jackknife estimator here can be written as a linear combination of the two largest values

\begin{align*} \hat\theta_J &= X_{(n)} + \frac{n-1}{n}\left(X_{(n)} - X_{(n-1)}\right) \\[1.3ex] &= \frac{2n-1}{n}X_{(n)} - \frac{n-1}{n} X_{(n-1)}. \end{align*}

What is the bias, variance and mean square error ?

$\endgroup$
  • $\begingroup$ One thing to bear in mind here is that $\hat{\theta} = X_{(n)}$ is a terrible estimator to begin with. It is more usual to use a bias-corrected version of the original estimator. For that reason, if you are going to employ the jackknife technique, I would recommend applying it to the bias-corrected version of the initial estimator. A jackknife estimator based on an unbiased estimator is also unbiased, so that makes the problem significantly easier (and improves the estimator). $\endgroup$ – Ben - Reinstate Monica Apr 6 at 23:36
  • 1
    $\begingroup$ @Ben Thanks for pointing that out - I'll add this to answer. For the record, I'm not actually planning to use this estimator. Needed this Q/A as a reference for some notes I'm writing. $\endgroup$ – knrumsey Apr 7 at 2:52
3
$\begingroup$

It is well known that the order statistics, sampled from a uniform distribution, are Beta-distributed random variables (when properly scaled). $$\frac{X_{(j)}}{\theta} \sim \text{Beta}(j, n+1-j)$$ Using standard properties of the Beta distribution we can obtain the mean and variance of $X_{(n)}$ and $X_{(n-1)}$.

Bias

\begin{align*} E\left(\hat\theta_J\right) &= \frac{2n-1}{n}E(X_{(n)}) - \frac{n-1}{n}E(X_{(n-1)}) \\[1.3ex] &= \frac{2n-1}{n}\frac{n}{n+1}\theta - \frac{n-1}{n}\frac{n-1}{n+1}\theta \\[1.3ex] &= \frac{n(n+1) - 1}{n(n+1)} \theta \end{align*}

Therefore the bias of $\hat\theta_J$ is given by $$\text{Bias}_\theta(\hat\theta_J) = \frac{-\theta}{n(n+1)}$$

Variance

Note: Covariance is derived here. \begin{align*} \text{Var}\left(\hat\theta_J\right) &= \frac{(2n-1)^2}{n^2}\text{Var}(X_{(n)}) + \frac{(n-1)^2}{n^2}\text{Var}(X_{(n-1)}) - 2 \frac{2n-1}{n}\frac{n-1}{n}\text{Cov}(X_{(n)}, X_{(n-1)}) \\[1.3ex] &= \frac{(2n-1)^2}{n^2}\frac{n\theta^2}{(n+1)^2(n+2)} + \frac{(n-1)^2}{n^2}\frac{2(n-1)\theta^2}{(n+1)^2(n+2)} - \frac{2(2n-1)(n-1)}{n^2}\frac{(n-1)\theta^2}{(n+1)^2(n+2)} \\[1.5ex] \end{align*} $$= \frac{(2n^2-1)\theta^2}{n(n+1)^2(n+2)}$$

MSE

Using the decomposition $\text{MSE}_\theta(\hat\theta) = \text{Bias}^2_\theta(\hat\theta) + \text{Var}(\hat\theta)$, we have

\begin{align*} \text{MSE}_\theta(\hat\theta_J) &= \left(\frac{-\theta}{n(n+1)}\right)^2 + \frac{(2n^2-1)\theta^2}{n(n+1)^2(n+2)} \\[1.3ex] &= \frac{2(n-1+1/n)\theta^2}{n(n+1)(n+2)}\\[1.3ex] &= \mathcal O(n^{-2}) \end{align*}

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.