0
$\begingroup$

I have 5 data sets fitting exactly the same ols model y = 5 + 0.4x. What is the best way to assess model's goodness of fit for each of the dataset and determine whether to use that model ?

$\endgroup$
2
  • $\begingroup$ What exactly is the goal? To accept or reject a model, based on its performance on the given data? Can you clarify what you want by "fit". Are you comparing the models between them? Something else? $\endgroup$ Commented Mar 23, 2020 at 13:22
  • 1
    $\begingroup$ Should all 5 datasets fitted to same ols model be treated equal ? If not, what could determine whether any of those 5 datasets warrant a different model instead. $\endgroup$
    – meag_a
    Commented Mar 23, 2020 at 23:26

3 Answers 3

0
$\begingroup$

Your question suggests that you already have your model determined: y = 5 + 0.4x. If this is the case, then you are decidedly outside of model estimation and comparison approaches. What remains is simply quantification of the fit of the model, which can be done using a variety of complimentary methods, including R^2 and RMSE.

$\endgroup$
1
  • $\begingroup$ Correct but when I do calculate r-sq or rmse, they turn out to be identical even though independent & dependent variables have different values in each of those five datasets and scatterplot look unique. They are perhaps different at 3rd or 4th decimal points. Looking to answer whether this model is good for all these 5 datasets. Thoughts? $\endgroup$
    – meag_a
    Commented Mar 23, 2020 at 11:52
0
$\begingroup$

You could use a likelihood test, to test whether your model performs better than a null model (intercept only), or to compare two models between them (on the same data).

But I do not know of any tests which would tell you whether there is a better model for your data. A Tukey test for nonadditivity ("lack-of-fit test") would test if it would benefit to add quadratic terms or interactions in the model.

$\endgroup$
0
$\begingroup$

Visualise the data points in scatter graph and compare these graphs between themselves. It might be that those "y = 5 + 0.4x" models are same, but have different data points distribution around regression line.

Also a meta question: Isn't there a bug? To have 5 data sets fitting -exactly- the same model seems highly unprobable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.