0
$\begingroup$

I have two regression models, each of which has an associated root mean squared error of cross validation (RMSECV). I would like to combine the results of the models using a weighted average to get a combined prediction. My question is how to report the accuracy of the resulting combined result: can RMSECV be propagated like any other uncertainty (e.g. adding in quadrature) or does it need to be handled differently?

$\endgroup$
1
$\begingroup$

I'd suggeest you directly measure the $RMSE_{CV}$ of the combined model. Error propagation of the $RMSE_{CV}$s of the submodels needs to take into account the covariance between the predictions of the submodels.

In other words, you can be anywhere between both models always yielding the same prediction (i.e. the error comes fromt the measurement) or yielding rather different (independent) predictions, i.e. substantial error due to model instability.
BTW, in the first case the aggregated (combined) model won't be any better than the individual models.

You can do a first check on what to expect by plotting the predictions of the two models against each other. If they're not tightly on the x = y line, and particularly if you observe a scattering around this line, you may gain by combining the models.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.