2
$\begingroup$

Suppose I have two models, each of which have the purpose of estimating a sequence of magnitudes. Suppose further that there are $N$ such magnitudes to be estimated, and that each magnitude is know to me, as the creator of the models, such that I can compute RMSE for each model across the $N$ magnitudes.

My question is the following:

When the two models have estimated the $N$ magnitudes, and I have computed the RMSE for each model, what statistic can I then use to determine if the RMSE generated by one model is significantly smaller than the RMSE generated by the other?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The standard asymptotic test is the ; see its tag wiki and literature cited there.

In your particular case, you have , since each model predicts $N$ different magnitudes. A model accounting for this effect would be appropriate, e.g., a .

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.