Is the high leverage sample observation considered a subset of the population, forcing its squared residual to be less than or equal to the population variance? Or am I misinterpreting the question? I'm trying to determine why the answer is false to the below question.
-
$\begingroup$ Do you know what leverage measures, in an intuitive sense? $\endgroup$– Glen_bFeb 9, 2014 at 21:30
-
1$\begingroup$ Yes, horizontal distance of an observation from the mean of the data. So is the answer that a high leverage point could have an squared residual of 0 which could be less than population variance ? $\endgroup$– Info5ekFeb 9, 2014 at 21:43
-
$\begingroup$ "Distance from the mean" is not intuitively "leverage". Why is that called "leverage"? $\endgroup$– Glen_bFeb 9, 2014 at 22:33
-
$\begingroup$ That's how it was explained to me. Intuition is subjective. $\endgroup$– Info5ekFeb 9, 2014 at 23:25
-
$\begingroup$ Again, why leverage? What is this quantity designed to measure? Note the second paragraph on the Wikipedia page on leverage $\endgroup$– Glen_bFeb 10, 2014 at 0:05
1 Answer
and
Which precisely matches what I guessed initially, that any squared residual must be less than or equal to the population variance. These formulas also suggest that higher leverage will result in a lower squared residual.
-
1$\begingroup$ This answer begins with a useful and relevant observation but then appears to conflate a "squared residual" with variance. The two differ; a squared residual can be a lot larger than its variance even for influential data. $\endgroup$– whuber ♦Feb 9, 2014 at 23:02
-
$\begingroup$ aren't squared residuals an estimator for the population variance? $\endgroup$– Info5ekFeb 9, 2014 at 23:14
-
1$\begingroup$ The estimator and the estimand are different things. In this case we must clearly distinguish $e_i^2$ (the squared residual itself) from $\mathbb{E}(e_i^2)$ (its expectation--a population property) from $\sigma^2$ (another population property). $\endgroup$– whuber ♦Feb 9, 2014 at 23:32
-
$\begingroup$ In that case what's the proper way to describe the answer using the above formulas? $\endgroup$– Info5ekFeb 9, 2014 at 23:36
-
$\begingroup$ My understanding is that $E (e_i)=0$ because the above analysis is (implicitly) conditioning on the configuration of the "X matrix". So $E (e_i^2)=var (e_i) $ holds in this case. $\endgroup$ Apr 24, 2016 at 16:07