3
$\begingroup$

I have the following residual plot. Can I detect outliers from residual plot? I want to remove 200 outliers in my data set, but I do not know how should I do that in R ?

residual plots:

enter image description here

scatter plots:

enter image description here

$\endgroup$
  • 3
    $\begingroup$ That plot is difficult to read. I read it as implying that you have one (1) outlier, or rather there is one outlying point on the graph, which might represent arbitrarily many tied observations. The idea that you know you should remove 200 outliers in advance is eerily like the idea that you know before investigation that 200 people are guilty of trying to undermine a state. $\endgroup$ – Nick Cox Oct 4 '14 at 17:55
  • $\begingroup$ Thanks a lot. How can I detect what is that outlier? is there a code in R that shows what that outlier is? $\endgroup$ – PSS Oct 4 '14 at 18:42
  • $\begingroup$ We now have 11 plots that are difficult to read. Still looks like one (1) outlier, as above. It should be easy to identify as having e.g. the largest negative residual. $\endgroup$ – Nick Cox Oct 4 '14 at 18:44
  • 1
    $\begingroup$ If you know how many outliers you have (200, though I don't know how you could know that) and you have some definite criterion for what makes an observation more outlying than another, then you simply order the observations by that criterion and take the 200 largest ones. So what do you mean by 'outlier'? Define that only well enough to order the observations and you seem to be done. $\endgroup$ – Glen_b Oct 4 '14 at 21:39
  • $\begingroup$ Thanks a lot for your reply. When I write this code : NegAdvSqrtIncome.mod<-lm(NegAdvSqrtIncome~ LogAssets+NegAdvInvest+AdvLogDelinq+LogCMembers+NegAdvSqrtBranches, data=HW3) and residualPlots(NegAdvSqrtIncome.mod, id.n=3) and qqPlot(NegAdvSqrtIncome.mod, id.n=3)it tags the outliers with numbers (5270 5925 7687 )! what are these numbers? my dependent variable does not have these values. $\endgroup$ – PSS Oct 5 '14 at 0:20
5
$\begingroup$

In general you can define outliers differently, depending on what exactly you are trying to achieve. For example, a presence of observations with very high leverage won't necessarily indicate that they are effecting the regression at all. On the other hand, presence of values with high Cook Distance, can certainly do. It is also possible that some values will have both. High Studentized residuals can indicate Heteroscedasticity. Here's an illustration of how you can identify/inspect each when compared to your original data and fitted regression line

Create some dummy data set and fit a linear regression model

set.seed(11)
df <- data.frame(x = rnorm(200), y = rnorm(200, 10, 5))
fit <- lm(y ~ x, data = df)
# summary(fit)

We will use influencePlot from car package in order to identify outliers and plot them, when

  1. x axis are hat values
  2. y axis are Studentized residuals
  3. Circles representing the observations proportional to Cooks distances

    library(car)
    (outs <- influencePlot(fit))
    #        StudRes         Hat      CookD
    # 62  -2.3075152 0.035229039 0.30844382
    # 73   2.7848421 0.008209828 0.17618044
    # 196  0.5258255 0.047410106 0.08310058
    

enter image description here

Now, we can get the corresponding row names of the, for example, 2 highest values in each

n <- 2
Cooksdist <- as.numeric(tail(row.names(outs[order(outs$CookD), ]), n))
Lev <- as.numeric(tail(row.names(outs[order(outs$Hat), ]), n))
StdRes <- as.numeric(tail(row.names(outs[order(outs$StudRes), ]), n))

And plot them over the fitted regression line

plot(df$x, df$y)
abline(fit, col = "blue")
points(df$x[Cooksdist], df$y[Cooksdist], col = "red", pch = 0, lwd = 15)
points(df$x[Lev], df$y[Lev], col = "blue", pch = 25, lwd = 8)
points(df$x[StdRes], df$y[StdRes], col = "green", pch = 20, lwd = 5)
text(df$x[as.numeric(row.names(outs))], 
     df$y[as.numeric(row.names(outs))], 
     labels = round(df$y[as.numeric(row.names(outs))], 3),
     pos = 1)

enter image description here

You can clearly see that some of the outliers are overlapping, when the leverage ones (the blue triangles) can sometimes affect the regression line while in other occasions be almost on it, while the red squares (Cook Distance) always have high effect on the regression line.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ What is the meaning of the points that influencePlot(fit) point out? Some have large Cook's distance but others don't how do you interpret this? $\endgroup$ – guy Jun 8 '17 at 18:58
  • $\begingroup$ @tbone they are proportional to Cooks Distance. Try res <- influencePlot(fit, id.n = 2) ; res[order(-res[, "CookD"]), "CookD", drop = FALSE] and compare to the plot. $\endgroup$ – David Arenburg Jun 8 '17 at 19:08
  • $\begingroup$ I don't understand what you mean. So identified points are not proportional to Cook's distance? I just want to understand why they are flagged. $\endgroup$ – guy Jun 8 '17 at 19:14
  • $\begingroup$ @tbone it returns the value(s) (depending on id.n) with the highest CookD, the highest Hat and the highest STDRes (while adding all the rest of the values). See setNames(c(max(cooks.distance(fit)), max(abs(rstudent(fit))), max(hatvalues(fit))), c(which.max(cooks.distance(fit)), which.max(abs(rstudent(fit))), which.max(hatvalues(fit)))) $\endgroup$ – David Arenburg Jun 8 '17 at 19:55
  • $\begingroup$ Ahh, got it now, so it flags the point(s) with the highest Cook distance, the highest estimated value (fitted value / hat value) and the highest studentized residual. $\endgroup$ – guy Jun 8 '17 at 20:18
2
$\begingroup$

If you want to find the 200 most extreme points, you might do a z score transformation to see which have the highest |z|. A rough guide is to look at |z|>3.

But I echo @Nick Cox . In decades of statistical practice I've never been in a situation where I knew how many outliers there were.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you.What is your recommendation? I do not know how to determine outliers? and how to remove them? I would be very thankful if you help $\endgroup$ – PSS Oct 4 '14 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.