Estimating the total with jackknife

Question

At a homework assignment, we are supposed to estimate the total in a population using a Horvitz Thompson estimator, and then estimate the variance using the jackknife technique.

As a first step, I wanted to use the jackknife to get a point estimate.

To use the jackknife technique, one should delete one observation at a time, and then calculate the estimate based on the sample without that observation.

However, when estimating the total using horvitz-thompson without a specific observation, it will of course necessarily be less than the total calculated with that observation. So, since we are summing n-1 observations instead of n, it seems that according to common sense the jackknife point estimate is not measuring the right thing.

Could you tell me where am I wrong?

Answer 1

As I understand it, the jackknife estimator is as follows:

$$\hat\theta_J = n\hat\theta - \frac{n-1}{n}\sum_{i=1}^n\hat\theta_{-i}$$

Where the $\hat\theta_{-i}$ is the estimator with the i'th observation deleted. Thus, even though the $\hat\theta_{-i}$ may be less than the "full" calculated estimate, the jackknife estimator still relies on the "full" estimate and just uses the deleted estimates to help estimate and eventually reduce the bias.

---

Response:

Unless I'm seriously misunderstanding what your goal is, what you propose doesn't make sense in general. The entire point of the Jackknife estimator is that the $\hat\theta_{(\cdot)}$ (in your notation) is not on the same scale as $\hat\theta$. It's purpose is bias reduction. The "pumped" jackknife estimator you propose will not give you a good estimate of your parameter in general (in fact it should always give you something close to zero...).

Here's a simple example. Suppose $X_1, X_2, \cdots, X_n$ are a random sample from a $U(0, \theta)$ distribution. We can perform jackknifing on the mle, and obtain an (almost) unbiased estimate. I'll also use your version of the jackknife, and we can compare the results via simulation. I'm setting $n=20$ and $\theta = 1$ for simulation purposes.

The red histogram represents the jackknife estimate as it is given above. The blue histogram is your version. As you can see it is missing the mark entirely.

To reiterate, the jackknife estimator uses $\hat\theta$ to get an estimate, and uses $\hat\theta_{(\cdot)}$ only to reduce the bias. This is why they're on a different scale, and this is why I believe you shouldn't be too concerned about the fact that $\hat\theta_{-i}$ being inherently off the mark.

If I am misunderstanding your goal, please let me know.

Answer 2

I think I have found the solution to my own question.

The jackknife estimator is

The last expression is the same as Big Agnes has written, where theta_hat(.) stands for the average of the estimators with the i-th observation deleted.

If one looks at the middle expression however: it is obvious that if the original estimator is unbiased, the only way for the jackknife estimator to be unbiased is when theta_hat(.) equals theta_hat, in case that the original estimate is unbiased - the jackknife estimator is biased.

I have found however, that if one "pumps" theta_hat(.) by multiplying it by n/n-1 then jackknife estimator works just fine, and gives the same results as the ordinary estimator.

I don't have an algebraic proof. Intuitively, it makes sense that if one removes one observation, and sums all the rest, then in order for the estimate to be in the same "order of magnitude" as the previous one, one should multiply it by n/n-1...

Therefore, IMHO, when one estimates a total using the jackknife, one should use

---

Edit : An example :

Suppose we have four observations :

1,2,3,4

We take $\hat\theta$ to be their sum and assume it is unbiased.

The four estimates of $\hat\theta_{-i}$ are

9,8,7,6

$\hat\theta_{(\cdot)}$ is 7.5

and

$ \hat\theta_J = n\hat\theta - (n-1)\hat\theta_{(\cdot)} = 17.5 $

Which is biased, since we assumed that the ordinary sum estimate is unbiased.

However, if we "pump" $\hat\theta_{(\cdot)}$ by n/n-1, the formula becomes

$ \hat\theta_J = n\hat\theta - n\hat\theta_{(\cdot)} = 10 $

---

Update : In class, the professor has solved the exercise, and actually he did what I have suggested. However, I did not not obtain a more rigorous explanation for the correction from him.

---

You have said that

If you are assuming independent observations from a common population, there is no parameter which we will estimate with the sum.

I will try to think of a way to make my assumptions more explicit, hoping that it will help.

This is all done in the context of sampling from a finite universe. The units in the populations are not random. However, the specific sample that we take is random.

Suppose the universe consists of {1,2,3} and you can only take a sample of the size of one OR a sample which is identical to the whole population. $\pi_i$ is defined for each unit of the population, and its definition is "the probability that the unit i will be included in the sample that we take". We can take 4 samples all in all, and I want to assume we choose among them with equal probability.

So $\pi_i$ , the probability that each unit will be selected is 2/4 (There are four possible samples. Look at unit 1 for example. It will be selected if (1) is chosen, or if (1,2,3) is chosen ).

So, $\pi_i $ = 1/2 for each of the three members of the population.

The horvitz-thompson (HT) estimator is $ \sum\frac{X_i}{\pi_i} $

As far as I understand, the way to show it is unbiased for this specific case is the following : We need to show that if we take the result the estimator gives for each of the possible samples, multiply it by the probability that this specific sample will occur, we will get the desired result.

HT(1) = 2

HT(2) = 4

HT(3) = 6

HT(1,2,3) = (1*2+2*2+3*2) = 12

When we multiply each of the results above by the probability that the specific sample will be chosen (1/4), and add them, we get exactly what we wanted : the total of the population.

This is just a demonstration that in the context of sampling from finite populations, you can have an unbiased estimate of the total.

I hope I got it right, this is the first course I am taking on sampling theory.