How to read Cook's distance plots?

Does anyone know how to work out whether points 7, 16 and 29 are influential points or not? I read somewhere that because Cook's distance is lower than 1, they are not. Am, I right?

-
There are various opinions. Some of them relate to the number of observations or to the number of parameters. These are sketched at en.wikipedia.org/wiki/…. –  whuber Feb 2 '12 at 15:02
@whuber Thanks. This is always a grey area when performing data exploration for me. Data point 16 above massively influences model outcomes, thus increasing Type I errors. –  Platypezid Feb 2 '12 at 15:11
One might argue that it increases "Type III" errors as well, which (generically and informally) are errors related to inapplicability of the underlying probability model. –  whuber Feb 2 '12 at 15:22
@whuber yes, very true! –  Platypezid Feb 2 '12 at 15:31

Some texts tell you that points for which Cook's distance is higher than 1 are to be considered as influential. Other texts give you a threshold of $4/N$ or $4/(N - k - 1)$, where $N$ is the number of observations and $k$ the number of explanatory variables. In your case the latter formula should yield a threshold around 0.1 .

John Fox (1), in his booklet on regression diagnostics is rather cautious when it comes to giving numerical thresholds. He advises the use of graphics and to examine in closer details the points with "values of D that are substantially larger than the rest". According to Fox, thresholds should just be used to enhance graphical displays.

In your case the observations 7 and 16 could be considered as influential. Well, I would at least have a closer look at them. The observation 29 is not substantially different from a couple of other observations.

(1) Fox, John. (1991). Regression Diagnostics: An Introduction. Sage Publications.

-
+1 Clear summary. I would add that influential cases are not usually a problem when their removal from the dataset would leave the parameter estimates essentially unchanged: the ones we worry about are those whose presence really does change the results. –  whuber Feb 2 '12 at 15:24
@lejohn Very appreciative of your response. Whuber is right excellent clarity in your answer. This is very informative. Might I suggest you highlight Fox's and your opinions in the wikipedia page! –  Platypezid Feb 2 '12 at 15:36

+1 to both @lejohn and @whuber. I wanted to expand a little on @whuber's comment. Cook's distance can be contrasted with dfbeta. Cook's distance refers to how far, on average, predicted y-values will move if the observation in question is dropped from the data set. dfbeta refers to how much a parameter estimate changes if the observation in question is dropped from the data set. Note that with $k$ covariates, there will be $k+1$ dfbetas (the intercept, $\beta_0$, and 1 $\beta$ for each covariate). Cook's distance is presumably more important to you if you are doing predictive modeling, whereas dfbeta is more important in explanatory modeling.

There is one other point worth making here. In observational research, it is often difficult to sample uniformly across the predictor space, and you might have just a few points in a given area. Such points can diverge from the rest. Having a few, distinct cases can be discomfiting, but merit considerable thought before being relegated outliers. There may legitimately be an interaction amongst the predictors, or the system may shift to behave differently when predictor values become extreme. In addition, they may be able to help you untangle the effects of colinear predictors. Influential points could be a blessing in disguise.

-
+1 "Cook's distance is presumably more important to you if you are doing predictive modeling, whereas dfbeta is more important in explanatory modeling": this is very useful advice. –  Anne Z. Feb 5 '12 at 0:45
Hi - interesting discussion. But couldn't it be rational to integrate a dummy-variable to measure the effect from for example observation 16? –  Pantera Feb 8 '12 at 9:04
@Pantera I removed 16 and compared the pre & post ommission models –  Platypezid Feb 8 '12 at 12:43
Hi - if you remove observations, you should make sure that you have "good" argument to do it, for example that the observation is wrongly measured. If we throw out observation because they just make some statistical trouble, then we're close to data-mining. –  Pantera Feb 9 '12 at 13:12