2
$\begingroup$

Given a dataset and liner model, how can I verify its sufficient quality?

> summary(ll)

Call:
lm(formula = dataset$speedup ~ dataset$cores + I(dataset$cores^2))

Residuals:
    Min      1Q  Median      3Q     Max 
-2.2220 -0.3854 -0.0917  0.3252  7.1028 

Coefficients:
                    Estimate Std. Error t value Pr(>|t|)    
(Intercept)        -0.669882   0.126153   -5.31 1.68e-07 ***
dataset$cores       1.201956   0.011876  101.20  < 2e-16 ***
I(dataset$cores^2) -0.012727   0.000235  -54.16  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.8832 on 477 degrees of freedom
Multiple R-squared:  0.9889,    Adjusted R-squared:  0.9888 
F-statistic: 2.122e+04 on 2 and 477 DF,  p-value: < 2.2e-16

Can I conclude, that model fits well since Multiple R-squared = 0.9889?

Alternatively, having the residuals printed, can I conclude, that they are random enough, hence the model fits well?

enter image description here

Can I test is somehow? Or is the intuition sufficient?

$\endgroup$
  • $\begingroup$ As I read in the past, you have to look at the p-value of F-statistics rather than at R-squared value. The error is normally distributed in a very nice way but, of course, you can check it using, for example, Kolmogorov-Smirnov test. It is not real dataset, isn't it? =) $\endgroup$ – Kirill May 8 '15 at 8:24
  • $\begingroup$ @Kirill it is a dataset I measured on the server. So what is the conclusion? Since the p-value is less then (let's say) 0.01, the model is good enough? $\endgroup$ – petrbel May 8 '15 at 8:29
  • 2
    $\begingroup$ Despite what the current answers assert, this plot tells you little about whether the model is a good fit, apart from revealing four obvious high outlying residuals--which ultimately might be the most interesting aspects to investigate. For better information use standard diagnostics, such as a residual vs fit plot, available from the command plot(ll). $\endgroup$ – whuber May 8 '15 at 14:21
1
$\begingroup$

I think your approach is correct and your model explains the data well, without too much bias (so it should generalize well for predictions).

The $R^2$ measure alone tells you how well the model explains all the variability of the data around the mean. Yours does.

Looking at residual plot tells you if your coefficient estimates are biased (something the $R^2$ does not tell you). Yours are not.

By googling I just stumbled upon this blog post which has some nice further explanations about these topics.

$\endgroup$
3
$\begingroup$

From the output, I can say that your model provides an excellent fit to the data (it sounds like you are dealing with simulated data). Moreover, the pvalue related to the F-statistic confirms that your model is extremely better than the one with a constant term only (In my opionion this pvalue is not that useful). You might check also the residuals against the covariates and see if you spot any pattern (see plot(ll)). If not, then the model may be ok.

Of course, you can do a lot of other things. For instance, you might also consider a GAM (Generalized Additive Model) and check if the quadratic form (the one you are using) is better than a GAM. This book, http://www-bcf.usc.edu/~gareth/ISL/getbook.html (you can get a free pdf copy for the site) might be helpful. Hope this helps.

$\endgroup$
  • $\begingroup$ The data is not simulated, I really measured them all for my thesis. However, can I state sth like "From the output, we conclude that the model provides an excellent fit to the data" in the academic paper? Isn't it a little bit vague? I mean, I was provided similar answers at the statistics lectures - that's why I decided to ask the community here. I seek few numbers from which I can deduce the conclusion. Simply by looking at the residuals I see they're OK but how can I express it? Anyway thanks for the interest :) $\endgroup$ – petrbel May 8 '15 at 9:32
  • $\begingroup$ It may be ok for reporting, for an academic paper that's another story. See this guide on writing papers ifs.tuwien.ac.at/~silvia/research-tips/NASA-64-sp7010.pdf. However, it depends on you data. If your data are time series, then such a high fit may not be that surprising. You need to know what, if any, other people had done with this kind of data. Anyway, in general we are happy when we've found the best model among the reasonable ones. So in a paper I would note the value of $R^2$, but I wouldn't put so much evidence on that. $\endgroup$ – utobi May 8 '15 at 10:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.