0
$\begingroup$

I have two very different empirical models that estimate a quantity $Z$ from two quantities ($x$ and $y$). Lets call the two models (highly non linear) $F_1$ and $F_2$

$$ Z = a_1F_1(x,y)$$

$$ Z = a_2F_2(x,y) $$

I trained the model coefficient $a$ over a first dataset (cross-validation) and I am testing on a separate dataset. Model F1 gives smaller absolute error than the other, and also smaller RMSD (root mean square deviation). If I try to overfit $a$ to each realization of Z, the standard deviation of the overfitted $a_1$ is also smaller than for the overfitted $a_2$. To me, that indicates that the process is better described by F1 cause the same coefficient works great for all cases. However, I wonder if I am right to compare the spread (Standard deviation) of coefficients with different units? Should I use some dimensionless quantity such as the Coefficient of Variation (CV)? I see in the Wiki example that has problems when comparing the spread of Celsius versus Faranheit (different units) so I a bit wary. Which is the best option?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.