4
$\begingroup$

If this is too elementary a question for CrossValidated and should go elsewhere please let me know.

I've been given a linear model in R similar to:

model = lm(y ~ x1 + x2 + x3, data=sampleset)

...that when fed a set of data and plotted produces the graph below:

Predicted vs observed

The light blue center line represents the linear regression prediction. The blue area represents 2 standard deviations from expected in either direction, using "standard error of residuals":

# se_: number of standard errors away from the predicted value
se_thresh = 2

# standard error of residuals
res_se = sd(model$residuals)

# for each member in the sample
if(model$fitted.values[j] < (i - se_thresh*res_se)){
  dataset$mark_good[j] = TRUE
}

I've been told that we can infer which members of the sample are unusually "good" or "bad" on the basis of whether the actual scores are 2 SEs above or below the predicted value.

I'm wondering:

  • why comparing members on the basis of the SE of residuals is better/more valid than simply comparing n with the standard deviation, which is simpler and would not require the model?
  • what standard error of residuals represents intuitively in the model
$\endgroup$
1
$\begingroup$

The authority here is just "I've been told". Despite that authority, you can think about the evidence you have: the graph itself tells you that lines at +/- 2 SE do not separate outliers from a mass of data points in the middle: they cut through zones with several data points.

It appears that your observed average score takes on distinct values 1, 2, 3, 3.5, 4, 4.5, 5. (The extra variation shown on the graph I take to be just graphical jittering.) At a guess each observed average score is just the average of two scores; otherwise the number of distinct values would be greater.

If you were to make decisions based on the SD of y alone, you would have to classify e.g. all the 1s in the same way or all the 5s in the same way. It is not clear how that you would help you in any manner whatsoever. The extreme values of 1 and 5 are not outlying in any strong or even weak sense. No value of 1 and no value of 5 is exceptional.

This graph alone doesn't suggest strong outliers, but it doesn't help much to identify them either. To check for outliers you would have to look at all the variables, e.g. in a scatter plot matrix. There is perhaps some evidence of asymmetry in that your residuals range (by eye) from almost -4 to less than 2.5, but then again that may be inevitable as it seems that most people get 4 or 5 and the model is predicting close to that for most of them.

Rather than worrying about outliers your main concern should be whether a simple linear model is the best you can do and whether there is scope for improving matters through a nonlinear model or a transformation.

A specific graphical point is that open symbols tolerate overlap better than the solid or closed symbols you are using.

$\endgroup$
1
$\begingroup$

I think you have to interpret things in terms of the structural model of linear regression. Suppose that $$ E \left[ Y | \beta \right] = \beta_0 + \beta_1X. $$

By imposing this model, you are imposing a connectedness between all the points. So, in your example, the data points at $2$ do have some influence on the values at $1$. Therefore, you should take into account all this data when minimizing variance, and not just the values at a given $x$. This constrained variance minimization problem (minimize variance, such that the means at each x-value have to lie on a line) is what standard errors are.

$\endgroup$
0
$\begingroup$

To answer your questions:

We compare data in a regression based on standard error of residuals and not solely on the standard deviation of Y because the regression can explain part of the standard deviation of Y.

Adjusted R^2, the (adjusted) coefficient of determination, tells us what proportion of the variation in Y is explained by the linear model. When R^2 is relatively high, we can say that part of the standard deviation for Y is dependent on the interaction between Y and the Xi's. Standard error takes this into account.

The standard deviation of Y is the spread of Y about its mean.

The standard error of the regression is the spread of Y about the regression line.

On a related note, an ANOVA analyzes the relationship between the variance of Y and the variance of the residuals to help us determine if the linear model used is a good fit (F-test).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.