3
$\begingroup$

I am wondering if anyone could explain why there are some states where Studentized residuals are undefined.

For example I got the following R code:

y <- c(0, 0, 0, 0, 0, 1) 
x <- 1:6
xx <- (1:6 - 3.5)^2
> rstudent(lm(y ~ x))
     1          2          3          4          5          6 
0.7559289  0.1428571 -0.2672612 -0.7142857 -1.5118579      NaN 

rstudent(lm(y ~ xx))
        1             2             3             4             5 
-3.008915e+00 -2.834734e-01  2.123977e-01  2.123977e-01 -2.834734e-01 
        6 
1.203157e+08 

As you can see the residual for point "6" is NaN and therefore undefined.

Plotting data I get the following plots:

plot(x,y): plot(rstudent(lm(y~x))

plot(xx,y):enter image description here

I can't figure out by myself why the residuals should not be defined in the case of lm(y ~ x).

$\endgroup$

1 Answer 1

4
$\begingroup$

Read ?rstudent. The error variance estimate used in the calculation of each Studentized residual is got by re-fitting the model on all the other observations. (These are often called deletion residuals, or externally Studentized residuals.) So what estimate of error variance do you get when excluding the sixth observation in this example?

[Raw residuals can be standardized by dividing each by an estimate of its standard deviation

$$s_i = \frac{e_i}{\hat\sigma \sqrt{1-h_{ii}}}$$

where $\hat\sigma$ is the estimate of residual standard error, & $h_{ii}$ the $i$th diagonal element of the hat matrix. If you want to detect outliers, then, reasoning that the estimate of residual standard error will be too high if the $i$th observation is indeed an outlier, you can use deletion residuals:

$$s_{-i} = \frac{e_i}{\hat\sigma_{-i} \sqrt{1-h_{ii}}}$$

where $\hat\sigma_{-i}$ is the estimate of residual standard error got by fitting the model on all observations except the $i$th.

As the first five observations of the response are identical, $\hat\sigma_{-6}$=0. So you're seeing NaN or $1.2\times10^8$ for the sixth deletion residual because you're dividing by zero, but this is just an extreme case of deletion residuals doing their job. It wouldn't arise in practice because you wouldn't be treating this kind of response as a continuous variable.]

$\endgroup$
6
  • $\begingroup$ when I leave out the 6th observation the error residuals all turn into NaNs... $\endgroup$
    – Phil S.
    Commented Nov 10, 2014 at 11:33
  • $\begingroup$ Yes - & the residual standard error? summary(lm(y[1:5]~x[1:5])) $\endgroup$ Commented Nov 10, 2014 at 11:46
  • $\begingroup$ Residual standard error: 1.813e-16 on 3 degrees of freedom. Also it states that lm(y[1:5] ~ x[1:5]) is essentially a perfect fit... which is also indicated by the really small error.. $\endgroup$
    – Phil S.
    Commented Nov 10, 2014 at 11:51
  • $\begingroup$ So zero then, to within machine precision. Not surprising as the first 5 $y$ values are all the same. $\endgroup$ Commented Nov 10, 2014 at 11:59
  • $\begingroup$ Ok so this is what I would expect: that you can model it perfectly when every target variable is the same. But why does it fail if one of the targets is different from all others (being the same)? Basically because this one destroys the otherwise perfect fit? $\endgroup$
    – Phil S.
    Commented Nov 10, 2014 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.