0
$\begingroup$

I am interested in performing GLM regression on a data with sample size of 37. After splitting the data into training (30 obs.) and testing (7 obs.) sets, I'm wondering wether it's better to build one GLM model using the whole training set consisting of 30 obs., or to make cross validation - building multiple models and then average the coefficients?

What is the common practice in such cases and will you recommend one-leave-out cross validation as the sample size of a training set is relatively small?

$\endgroup$
3
$\begingroup$

build one GLM model using the whole training set consisting of 30 obs., or to make cross validation - building multiple models and then average the coefficients?

Building multiple (linear) models and average their coefficients is not what is usually called cross validation. It is a particular way of aggregating models.

Aggregated (aka ensemble) models perform better than the individual models of the ensemble iff the individual models suffer from variance, i.e. are unstable. Otherwise, aggregation does not help.

will you recommend one-leave-out cross validation as the sample size of a training set is relatively small?

No*, particularly with small sample sizes I'd recommend to use either out-of-bootstrap or iterated/repeated $k$-fold (or equivalent, leave-$n$-out) cross validation. The many iterations will allow you to judge stability, which yields a basis for the decision for or against model aggregation.

* That is, unless the sample is so small that there isn't any chance to leave out more than 1 case. Roughly speaking, I'd recommend LOO for $n =$ 3 or 4. For $n \le$ 15, I'd evaluate all combinations of leaving out 2 cases, after that I'd go for iterated $k$-fold or out of bootstrap with a total number of surrogate models in the order of magnitude of 100 - 1000.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.