I'm new to regression and diagnostics so if this seems a bit basic/unnecessarily long-winded that's why.

I perform a multiple regression of a response variable on four predictor variables. There are 300 data points. I'm asked to find the case with the highest influence. I understand that studentized residuals, leverage and Cook's distances are involved.

I know that the concept of what makes a particular case an outlier or a case of high leverage or whatever is kind of subjective but for the sake of this question, I have a cutoff point defined for certain things (I don't know if these are standard or not so you may already know this):

  1. If the absolute value of the studentized residual is greater than 2, then the corresponding data point is considered an outlier.
  2. If the leverage value is greater than twice the number of predictors divided by the number of data points (in this case this value is $\frac{1}{30}$), then the corresponding case has high leverage.

The Cook's distance for each case is calculated as follows:$$D_i = \frac{1}{p}r_i^2\left(\frac{h_{ii}}{1-h_{ii}}\right)$$ where $D_i$ is the distance for the i$^{th}$ case, $p$ is the number of preditors, $r_i$ is the studentized residual for the i$^{th}$ case, and $h_{ii}$ is the leverage for the i$^{th}$ case. There is no cutoff for Cook's Distance.

My instinct is to just find the data point with the largest Cook's Distance and choose that. This prompts my main question:

-Is it possible that the data point with the largest Cook's distance is not the case with the highest influence?

-If so, how likely is that to be true, and how would I find out which data point actually does have the highest influence?

If it helps, I'm using RStudio to calculate values, interpret plots etc. Thanks

  • $\begingroup$ The leverage $h_{ii}$ measures how each input parameter $\vec{x}$ influences the fit model. In contrast, the Cook's distance includes also the influence of the output parameter $y$ -- more accurate: it includes the residuals $r_i$. Thus both are measures of influence. So the question is, in which influence are you interested in? $\endgroup$ – Semoi Nov 1 '19 at 16:37
  • $\begingroup$ The lecturer in my regression course gave the definition that a case has high influence if omitting the case from the regression causes a large change in the least-squares estimates. That's the influence I'm interested in. $\endgroup$ – sandbag66 Nov 1 '19 at 16:46

Let $\hat{\beta}$ be the estimators of the linear regression model $\hat{y} = x \hat{\beta}$. And let $\hat{\beta}_{(-i)}$ be the estimators of the same parameters if we omit the $i^{th}$ datapoint, $(x_{i,1}, x_{i,2}, \ldots, x_{i,p}, y_i)$. A "natural" way to quantify the influence of each datapoint onto the estimators is the quantity $|\hat{y} - \hat{y}_{(-i)}|^2$. The Cook's distance is just a rescaled version of this quantity. It is defined as $$D_i = \frac{|\hat{y} - \hat{y}_{(-i)}|^2}{(p+1) \hat{\sigma}^2}$$ where the nominator is usually written in a vector notation. Thus in this sense the Cook's distance measures the influence on each datapoint onto the estimators.

| cite | improve this answer | |
  • $\begingroup$ This is a good explanation of what Cook's Distance is, but it doesn't really address the questions I asked at the bottom of the post. $\endgroup$ – sandbag66 Nov 1 '19 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.