Leverage and Influence

Question

Is it possible that an outlier is neither influential nor does it have high leverage?

Or can it happen that an observation with high leverage is not an outlier and is neither influential?

Answer 1

How do you define outlier? It generally means a data point which do not conform with the majority of the data points. That can of course be interpreted in many ways, and just as in society, where nonconformists can be very valuable (a literary reference), and sometimes are right when the majority goes wrong, outliers in data doesn't need to be "bad". But they might need some special thought or treatment. In chemical industry outliers sometimes lead to new patents!

But your question is about leverage, which is mostly used with linear models. We could say that a high-leverage point is an outlier in $X$-space.

Or can it happen that an observation with high leverage is not an outlier and is neither influential?

Leverage can be calculated from only the design matrix $X$, so before any $Y$ is observed! while residuals (which would be used with any definition of outlier) obviously need $Y$. That means that, just from the mathematics, any value of the residual can go with any value of leverage. With really extreme leverage values, that design point will have so much influence in the estimation that a large residual would be rare (that is what influence means, the regression line gravitate towards the influential point.) So a low-leverage point could have a large residual (and be considered an outlier), or a low residual while being in error (sometimes called an inlier, see Difference between Outlier and Inlier.) And a high-leverage point could have a small residual, but still being in error. So not an outlier, if that definition is based on a large residual.

You also ask if a point can have high leverage and not being influential? That question is difficult to understand, since leverage is taken to mean having high influence. So you should clarify.