2
$\begingroup$

I read the Wikipedia article on the Goodness of fit(GOF) and it explains GOF as

The goodness of fit of a statistical model describes how well it fits a set of observations. Measures of goodness of fit typically summarize the discrepancy between observed values and the values expected under the model in question.

I look at the description and I think that the cross-validation does the same thing as GOF. But I never heard of anyone describing the cross-validation as the GOF test.

Could cross-validation be considered as a GOF test and if it's not, what is the reason?

$\endgroup$
2
  • $\begingroup$ I thought cross-validation was a model tuning or hyperparameter estimation technique. It might also be use to predict future out-of-model errors but will tend to be biased, and you might want to regard those predictions as a goodness-of-fit measure $\endgroup$
    – Henry
    Jun 11 '20 at 0:42
  • $\begingroup$ @Henry While cv is used for the model selection, it performs the task by calculating the cost function which is the measure of fit. I understand your point. People may think of cv as more than the goodness of fit. But in the end, it's a model validation tool, I think, as the name indicates. $\endgroup$ Jun 11 '20 at 0:58
2
$\begingroup$

No. The main difference is that CV attempts to emulate out of sample performance of the model while GoF tests are in sample. Roughly speaking GoF looks at the fit to entire sample while CV fits to a subset then compares the fit with the other part of dataset

$\endgroup$
2
  • $\begingroup$ IMHO attempts needs to be emphasized. One of the most frequent mistakes I encounter with CV are incorrect splitting procedures (that do not take into account e.g. repeated measurements) - these will effectively degrage the CV result to an elaborate goodness of fit test. $\endgroup$ Jun 12 '20 at 8:00
  • 1
    $\begingroup$ @cbeleitesunhappywithSX in fact the way most people use CV turn it into an extension of estimation. when you run thousands of different models and use CV on each of them to select best, it's not much different from the main path of estimation. hence this practice undermines stated objective of CV to address overfitting. using CV this way one can overfit as well as without CV. however, this is not what OP was asking about $\endgroup$
    – Aksakal
    Jun 12 '20 at 14:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.