20
$\begingroup$

Is there really any difference between the jackknife and leave one out cross validation? The procedure seems identical am I missing something?

$\endgroup$
14
$\begingroup$

In cross-validation you compute a statistic on the left-out sample(s). Most often, you predict the left-out sample(s) by a model built on the kept samples. In jackknifing, you compute a statistic from the kept samples only.

$\endgroup$
1
  • 5
    $\begingroup$ I do not understand how this answer speaks to the LOOCV in the original question. In what sense can one "compute a statistic" on a single left-out observation? $\endgroup$ – Alexis Feb 18 '19 at 16:27
15
$\begingroup$

Jackknife often refers to 2 related but different processes, both of which rely on a leave-one-out approach -- leading to this very confusion.

In one context, jackknife can be used to estimate population parameters and their standards errors. For example, to use a jackknife approach to estimate the slope and intercept of a simple regression model one would:

  1. Estimate the slope and intercept using all available data.
  2. Leave out 1 observation and estimate the slope and intercept (also known as the "partial estimate" of the coefficients).
  3. Calculate the difference between the "partial estimate" and the "all data" estimate of the slope and the intercept (also know as the "pseudo value" of the coefficients).
  4. Repeat steps 2 & 3 for the entire data set.
  5. Compute the mean of the pseudo values for each coefficient -- these are the jackknife estimates of the slope and intercept

The pseudo values and the jackknife estimates of the coefficients can also be used to determine the standard errors and thus confidence intervals. Typically this approach gives wider confidence intervals for the coefficients because it's a better, more conservative, measure of uncertainty. Also, this approach can be used to get a jackknife estimate of bias for the coefficients too.

In the other context, jackknife is used to evaluate model performance. In this case jackknife = leave-one-out cross validation. Both refer to leaving one observation out of the calibration data set, recalibrating the model, and predicting the observation that was left out. Essentially, each observation is being predicted using its "partial estimates" of the predictors.

Here's a nice little write-up about jackknife I found online: https://www.utdallas.edu/~herve/abdi-Jackknife2010-pretty.pdf

$\endgroup$
2
  • 1
    $\begingroup$ Unless I am mistaken (and I may well be), your first context describes leave-one-out cross validation. $\endgroup$ – Alexis Feb 18 '19 at 16:31
  • 2
    $\begingroup$ I was just separating the ideas of estimating parameters using LOO versus estimating the value that was left out (as in LOOCV). I see them as two related but slightly different processes, but perhaps both can be referred to as LOOCV? I could also be mistaken. $\endgroup$ – jcmb Feb 26 '19 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.