10
$\begingroup$

One way to find accuracy of the logistic regression model using 'glm' is to find AUC plot. How to check the same for regression model found with continuous response variable (family = 'gaussian')?

What methods are used to check how well does my regression model fit the data?

|cite|improve this question
$\endgroup$
  • $\begingroup$ You may want to have a look at the r-squared tag and the goodness-of-fit tag.. $\endgroup$ – Macro Feb 28 '13 at 15:36
  • 2
    $\begingroup$ The "Gaussian" family with a linear link is just ordinary least squares (OLS) regression; methods to check such fits are probably discussed in a thousand questions on this site (I do not exaggerate). $\endgroup$ – whuber♦ Feb 28 '13 at 17:13
  • $\begingroup$ This thread is relevant: stats.stackexchange.com/q/414349/121522 $\endgroup$ – mkt - Reinstate Monica Jun 25 at 5:48
15
$\begingroup$

I would suggest a brief search on "linear regression model diagnostics" as a start. But here are some that I would suggest you to check:

Make sure the assumptions are satisfactorily met

  • Use scatterplot or component plus residual plot to examine the linear relationship between the independent predictor(s) and the dependent variable.

  • Compose a plot with standardized residual versus predicted value and ensure there isn't extreme point with very high residual, and the spread of the residual is largely similar along the predicted value, as well as spreading largely equally above and below the mean of residual, zero.

  • You can also change the y-axis to residual$^2$. This plot helps identifying unequal variance.

  • Re-examine the study design to ensure the assumption of independence is reasonable.

  • Retrieve the variance inflation factor (VIF) or tolerance statistics to examine possible collinearity.

Examine potential influential point(s)

  • Check statistics such as Cook's D, DFits, or DF Beta to find out if a certain data point is drastically changing your regression results. You can find more here.

Examine the change in $R^2$ and Adjusted $R^2$ statistics

  • Being the ratio of regression sum of squares to total sum of squares, $R^2$ can tell you how many % of variability in your dependent variable are explained by the model.
  • Adjusted $R^2$ can be used to check if the extra sum of squares brought about my the additional predictor(s) is really worth the degrees of freedom they'll take.

Check necessary interaction

  • If there is a main independent predictor, before you make any interpretation of its independent effect, check if it is interacting with other independent variables. Interaction, if left unadjusted, can bias your estimate.

Apply your model to another data set and check its performance

  • You can also apply the regression formula to other separate data and see how well it predicts. Graph like scatter plot and statistics like % difference from the observed value can serve as a good start.
|cite|improve this answer
$\endgroup$
  • 2
    $\begingroup$ (+1): Very complete answer! If you're using R, plot.lm can give you most of the diagnostic plots Penguin_Knight mentions. $\endgroup$ – Zach Feb 28 '13 at 16:31
4
$\begingroup$

I like to cross-validate my regression models to see how well they generalize to new data. My metric of choice is mean absolute error on the cross-validated data, but root mean squared error is more common and equally useful.

I don't find R2 to be a good metric of how well your model fits the training data, as almost any error metric calculated on the training data will be prone to over fitting. If you must calculate R2 on the training set, I suggest using adjusted R2.

|cite|improve this answer
$\endgroup$
1
$\begingroup$

You can use $R^2$ to examine how well your model fits the training data. This will tell you what percentage of the variance in the data are explained by the model.

I suggest using RMSE (root mean square error) of your predictions on your test set when compared to the actual value. This is a standard method of reporting prediction error of a continuous variable.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ @Macro But the question originally asked for a performance metric for an OLS Regression with gaussian errors. He is coming from logistic regression. $\endgroup$ – Erik Feb 28 '13 at 15:16
  • $\begingroup$ @Erik, thanks, I misread. Anyway, regarding the first part, I don't think $R^2$, in isolation, can be used to "check if my regression model is good", to use the OP's words. Your model could fail miserably to predict effectively on vast majority of the data while still having a high $R^2$. See here for an example - in example (1), there's almost no predictive power but $R^2$ is still high. $\endgroup$ – Macro Feb 28 '13 at 15:24
  • $\begingroup$ @Macro, I agree with your comments but was aiming for a simple explanation to point the OP in the right direction $\endgroup$ – BGreene Feb 28 '13 at 16:03
0
$\begingroup$

I am used to check the functional form of my parameter estimator by plotting a non-parametric (e.g. a kernel regression) or semi-parametric estimation and comparing it to the parametric fitted curve. I think this is in the first step often faster (and perhaps more insightful) than including interaction terms or higher-orders terms.

The R package np provides many nice non-parametric and semi-parametric functions, and its Vignette is well written: http://cran.r-project.org/web/packages/np/vignettes/np.pdf

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.