At a homework assignment, we are supposed to estimate the total in a population using a Horvitz Thompson estimator, and then estimate the variance using the jackknife technique.

As a first step, I wanted to use the jackknife to get a point estimate.

To use the jackknife technique, one should delete one observation at a time, and then calculate the estimate based on the sample without that observation.

However, when estimating the total using horvitz-thompson without a specific observation, it will of course necessarily be less than the total calculated with that observation. So, since we are summing n-1 observations instead of n, it seems that according to common sense the jackknife point estimate is not measuring the right thing.

Could you tell me where am I wrong?


As I understand it, the jackknife estimator is as follows:

$$\hat\theta_J = n\hat\theta - \frac{n-1}{n}\sum_{i=1}^n\hat\theta_{-i}$$

Where the $\hat\theta_{-i}$ is the estimator with the i'th observation deleted. Thus, even though the $\hat\theta_{-i}$ may be less than the "full" calculated estimate, the jackknife estimator still relies on the "full" estimate and just uses the deleted estimates to help estimate and eventually reduce the bias.


Unless I'm seriously misunderstanding what your goal is, what you propose doesn't make sense in general. The entire point of the Jackknife estimator is that the $\hat\theta_{(\cdot)}$ (in your notation) is not on the same scale as $\hat\theta$. It's purpose is bias reduction. The "pumped" jackknife estimator you propose will not give you a good estimate of your parameter in general (in fact it should always give you something close to zero...).

Here's a simple example. Suppose $X_1, X_2, \cdots, X_n$ are a random sample from a $U(0, \theta)$ distribution. We can perform jackknifing on the mle, and obtain an (almost) unbiased estimate. I'll also use your version of the jackknife, and we can compare the results via simulation. I'm setting $n=20$ and $\theta = 1$ for simulation purposes.

enter image description here

The red histogram represents the jackknife estimate as it is given above. The blue histogram is your version. As you can see it is missing the mark entirely.

To reiterate, the jackknife estimator uses $\hat\theta$ to get an estimate, and uses $\hat\theta_{(\cdot)}$ only to reduce the bias. This is why they're on a different scale, and this is why I believe you shouldn't be too concerned about the fact that $\hat\theta_{-i}$ being inherently off the mark.

If I am misunderstanding your goal, please let me know.

  • $\begingroup$ I don't think you misunderstand my goal. In case that $\hat\theta$ is unbiased, then in such a case $\hat\theta_{(\cdot)}$ should be on the same scale, since it is the only way algebraically that $\hat\theta_J = n\hat\theta - (n-1)\hat\theta_{(\cdot)}$ will give the same result as the unbiased $\hat\theta$ ... I added a simple example to my answer. $\endgroup$ – Sam Jun 4 '17 at 5:45
  • $\begingroup$ I see what you are getting at, but here's my problem. If you are assuming independent observations from a common population, there is no parameter which we will estimate with the sum. For instance, the sum of 4 observations may be unbiased for $\theta$, but what if you have a $5^{th}$ observation? It won't be unbiased anymore. It makes more sense to say the average is unbiased. Using your example data, we now get $\hat\theta_J = 2.333$ for the traditional jackknife and $\hat\theta_{J^*} = -0.22$ for your version. Do you see the issue? $\endgroup$ – knrumsey Jun 4 '17 at 21:11
  • 1
    $\begingroup$ Thank you, I start to see the problem. The "unbiased" estimator I had in mind is the horvitz-thompson estimator for the total. The estimator is $\sum\frac{Y_i}{\pi_i}$ , where $\pi_i$ is the probability of including the i-th unit in the sample. But it may very well be that I am misusing it. I will ask the professor, and get back to you (perhaps it will be on the next weekend). $\endgroup$ – Sam Jun 5 '17 at 7:44
  • $\begingroup$ Sounds good. One more comment... while i'm unfamiliar with the horvitz-thompson estimator, if you believe it is unbiased already, you don't really need to use the jackknife to correct for bias. You may be better of sticking with your point estimate $\hat\theta = \sum\frac{Y_i}{\pi_i}$, and using the jackknife to estimate the variance of said estimator. $\endgroup$ – knrumsey Jun 5 '17 at 16:19
  • 1
    $\begingroup$ I do not really need to use the jackknife to correct for bias, that is correct. I am using it for estimating the variance indeed. However, the jackknife estimator should in theory give the same estimate as the horvitz thompson (obviously - they are both unbiased), so I was just checking myself. $\endgroup$ – Sam Jun 24 '17 at 16:52

I think I have found the solution to my own question.

The jackknife estimator is enter image description here

The last expression is the same as Big Agnes has written, where theta_hat(.) stands for the average of the estimators with the i-th observation deleted.

If one looks at the middle expression however: it is obvious that if the original estimator is unbiased, the only way for the jackknife estimator to be unbiased is when theta_hat(.) equals theta_hat, in case that the original estimate is unbiased - the jackknife estimator is biased.

I have found however, that if one "pumps" theta_hat(.) by multiplying it by n/n-1 then jackknife estimator works just fine, and gives the same results as the ordinary estimator.

I don't have an algebraic proof. Intuitively, it makes sense that if one removes one observation, and sums all the rest, then in order for the estimate to be in the same "order of magnitude" as the previous one, one should multiply it by n/n-1...

Therefore, IMHO, when one estimates a total using the jackknife, one should use enter image description here

Edit : An example :

Suppose we have four observations :


We take $\hat\theta$ to be their sum and assume it is unbiased.

The four estimates of $\hat\theta_{-i}$ are


$\hat\theta_{(\cdot)}$ is 7.5


$ \hat\theta_J = n\hat\theta - (n-1)\hat\theta_{(\cdot)} = 17.5 $

Which is biased, since we assumed that the ordinary sum estimate is unbiased.

However, if we "pump" $\hat\theta_{(\cdot)}$ by n/n-1, the formula becomes

$ \hat\theta_J = n\hat\theta - n\hat\theta_{(\cdot)} = 10 $

Update : In class, the professor has solved the exercise, and actually he did what I have suggested. However, I did not not obtain a more rigorous explanation for the correction from him.

You have said that

If you are assuming independent observations from a common population, there is no parameter which we will estimate with the sum.

I will try to think of a way to make my assumptions more explicit, hoping that it will help.

This is all done in the context of sampling from a finite universe. The units in the populations are not random. However, the specific sample that we take is random.

Suppose the universe consists of {1,2,3} and you can only take a sample of the size of one OR a sample which is identical to the whole population. $\pi_i$ is defined for each unit of the population, and its definition is "the probability that the unit i will be included in the sample that we take". We can take 4 samples all in all, and I want to assume we choose among them with equal probability.

So $\pi_i$ , the probability that each unit will be selected is 2/4 (There are four possible samples. Look at unit 1 for example. It will be selected if (1) is chosen, or if (1,2,3) is chosen ).

So, $\pi_i $ = 1/2 for each of the three members of the population.

The horvitz-thompson (HT) estimator is $ \sum\frac{X_i}{\pi_i} $

As far as I understand, the way to show it is unbiased for this specific case is the following : We need to show that if we take the result the estimator gives for each of the possible samples, multiply it by the probability that this specific sample will occur, we will get the desired result.

enter image description here

HT(1) = 2

HT(2) = 4

HT(3) = 6

HT(1,2,3) = (1*2+2*2+3*2) = 12

When we multiply each of the results above by the probability that the specific sample will be chosen (1/4), and add them, we get exactly what we wanted : the total of the population.

This is just a demonstration that in the context of sampling from finite populations, you can have an unbiased estimate of the total.

I hope I got it right, this is the first course I am taking on sampling theory.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.