Finding the Case with the Highest Influence

Question

I'm new to regression and diagnostics so if this seems a bit basic/unnecessarily long-winded that's why.

I perform a multiple regression of a response variable on four predictor variables. There are 300 data points. I'm asked to find the case with the highest influence. I understand that studentized residuals, leverage and Cook's distances are involved.

I know that the concept of what makes a particular case an outlier or a case of high leverage or whatever is kind of subjective but for the sake of this question, I have a cutoff point defined for certain things (I don't know if these are standard or not so you may already know this):

1. If the absolute value of the studentized residual is greater than 2, then the corresponding data point is considered an outlier.
2. If the leverage value is greater than twice the number of predictors divided by the number of data points (in this case this value is $\frac{1}{30}$), then the corresponding case has high leverage.

The Cook's distance for each case is calculated as follows:$$D_i = \frac{1}{p}r_i^2\left(\frac{h_{ii}}{1-h_{ii}}\right)$$ where $D_i$ is the distance for the i$^{th}$ case, $p$ is the number of preditors, $r_i$ is the studentized residual for the i$^{th}$ case, and $h_{ii}$ is the leverage for the i$^{th}$ case. There is no cutoff for Cook's Distance.

My instinct is to just find the data point with the largest Cook's Distance and choose that. This prompts my main question:

-Is it possible that the data point with the largest Cook's distance is not the case with the highest influence?

-If so, how likely is that to be true, and how would I find out which data point actually does have the highest influence?

If it helps, I'm using RStudio to calculate values, interpret plots etc. Thanks

Answer 1

Let $\hat{\beta}$ be the estimators of the linear regression model $\hat{y} = x \hat{\beta}$. And let $\hat{\beta}_{(-i)}$ be the estimators of the same parameters if we omit the $i^{th}$ datapoint, $(x_{i,1}, x_{i,2}, \ldots, x_{i,p}, y_i)$. A "natural" way to quantify the influence of each datapoint onto the estimators is the quantity $|\hat{y} - \hat{y}_{(-i)}|^2$. The Cook's distance is just a rescaled version of this quantity. It is defined as $$D_i = \frac{|\hat{y} - \hat{y}_{(-i)}|^2}{(p+1) \hat{\sigma}^2}$$ where the nominator is usually written in a vector notation. Thus in this sense the Cook's distance measures the influence on each datapoint onto the estimators.