0
$\begingroup$

I have ~ 120 different datasets (different scales, sample size etc) and for each dataset, I predict ONE statistical parameter (doesn't matter what for my question) with different methods. To compare the different methods I use RMSE and MAE across the different datasets. I.e. if $m$ is the method than the RMSE is calculated in the following manner:

$RMSE_m = \sqrt{\frac{1}{n}\sum_{i\in D}(\hat{y}_{im}-y_i)^2}$

where $D$ are the datasets and $n$ the number of datasets. Is this approach suitable to evaluated between different methods $m$ or did I miss something?

Thanks

Improve this question
$\endgroup$
2
  • $\begingroup$ Are you trying to say that method 1 (say a neural net) has better performance than method 2 (say random forest) across the 120 different data sets? $\endgroup$
    – Dave
    Commented Feb 11, 2020 at 17:52
  • $\begingroup$ @Dave I’m not using any Machine Learning methods, I’m predicting e.g the variance using a own method which mainly consists out of integrations. But I don’t think that this plays a role does it ? $\endgroup$
    – T. Tim
    Commented Feb 11, 2020 at 18:18

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.