8
$\begingroup$

I'm having trouble understanding what cross-validation is.

Also, what is the connection between cross-validation and the issue of model overfitting?

$\endgroup$
  • 1
    $\begingroup$ What is your understanding so far, say from the Wikipedia article? $\endgroup$ – Alexis Dec 11 '14 at 6:36
  • 1
    $\begingroup$ You could check also this and this, and the references quoted in the answers. $\endgroup$ – Tim Dec 11 '14 at 7:29
11
$\begingroup$

"Cross-validating" refers to attempts to check ("validate") whether the outcome of a statistical analysis will generalise to another sample. Typically, these involve running the same analyses ("cross") on comparable subsets of your data: e.g., you might divide your dataset into two (a "training" set and a "validation" set) and run the analysis on both.

An overfitted model is one which adequately, albeit artificially describes your current dataset, but performs poorly on yet unseen (but equivalent) datasets. This can happen if your model was derived ad hocly. Cross-validation is one way of checking against overfitting.

$\endgroup$
  • 2
    $\begingroup$ Helpful answer, and nice use of "ad hocly." $\endgroup$ – rolando2 Dec 11 '14 at 9:58
  • 1
    $\begingroup$ I believe "...derived ad hoc" would be reasonably grammatical (I think it's literally 'to this', but sometimes/usually rendered as 'for this') $\endgroup$ – Glen_b -Reinstate Monica Dec 11 '14 at 10:19
  • 2
    $\begingroup$ (+1) But it might be useful to (1) explain deriving models "ad hocly", & (2) note that estimating too many parameters can lead to overfitting even with the most unadhocly procedures. $\endgroup$ – Scortchi - Reinstate Monica Dec 11 '14 at 12:10
  • 1
    $\begingroup$ I laughed too hard at unadhocly. $\endgroup$ – Marc Claesen Dec 11 '14 at 14:43

Not the answer you're looking for? Browse other questions tagged or ask your own question.