I have loaded the data set Animals2 from the package library (robustbase) and I have ploted three estimated regression lines (least-squares line, the LMS line, and the line of Siegel). I wanted to know which of the estimated regression lines is most suitable for this data sets.
$\begingroup$
$\endgroup$
6
1 Answer
$\begingroup$
$\endgroup$
3
One way to decide which model fits best to your data, is to you use the Mean Squared Error (MSE). The model with lowest MSE can be considered the best fitting model. However, there is more to it. For example, how many parameters more does the better fitting model have compared to the other models. You would want the best fitting model with the fewest parameters.
Also MSE is just one Metric to measure the goodness of fit. Generally, speaking OLS will provide the lowest MSE. Hence the name. However, one can consider a situation, where outliers have a large effect on the estimation of the OSL. You could, for example, estimate your model only on parts of the data and then evaluate the model on the left over data. This could evaluate how well your model works for unseen data.
-
1$\begingroup$ This answer is circular, because proposing MSE is equivalent to just asserting OLS is the right solution--but it offers no independent reason for doing so. All three of the proposed fits in this question use the same number of parameters, so that offers no help. $\endgroup$– whuber ♦Commented May 23, 2022 at 14:17
-
1$\begingroup$ The lowest sum of squared residuals will always come from the OLS ("LM") model. $\endgroup$– DaveCommented May 23, 2022 at 14:17
-
$\begingroup$ True that. But assuming that the way the question is asked he might not aware of that fact. However, I will edit my answer to make that clear. Also that is not completely true, when we talk about non-linear models for example. $\endgroup$– JanoschCommented May 23, 2022 at 14:38
Most suitable regression line... for what purpose? $\endgroup$