I'm currently working on an Econometrics project and I've come to a point where I've dropped a high leverage point as identified by cook's distance and a leverage plot (had observations that were twelve standard deviations away from mean and a nonsensical entry). My regression model before I dropped the high leverage outlier (Sample Size=408) had every regressor be statistically significant at the 95% confidence level. However, after I drop this one observation (New Sample Size=407), one of my variables becomes slightly statistically insignificant (New P=0.051 where Old P=0.045). What is the explanation for this?
First, a change in p from 0.045 to 0.051 is essentially meaningless. It simply points out, yet again, how using a simple cutoff of p < .05 leads to silliness.
Second, a high leverage point forces the regression line to be near the point, even if it is a data entry error or an extremely odd case. Thus, it can radically change the regression line and, therefore, change all the associated statistics.
Third, since in your case the high leverage point was "nonsensical" your first regression is not to be taken seriously and should be ignored. If that high leverage point was "real" - that is, simply an unusual case - then there might be more to say.