I am interested in proving the following fact for locally weighted polynomial regression from The Elements of Statistical Learning by Hastie et. al.

It can be shown that $||l(x_0)||$ increases with $d$...

The same fact is also mentioned in Local Regression and Likelihood by Loader:

... the influence function infl(x) increases as the degree of the local polynomial increases.

Where $d$ is the degree of local polynomial and

$$ ||l(x_0)||^2 = b(x_0)^T(B^TWB)^{-1}B^TWW^TB(B^TWB)^{-1}b(x_0)$$ where $b(x)^T = (1, x, x^2, ..., x^{d-1})$, $B$ is the $N \times d$ regression matrix with ith row $b(x_i)^T$, and $W(x_0)$ is an $N \times N$ diagonal weight matrix.

This question's accepted solution solves a related variance problem in the special case when $W = I_{N \times N}$, however this doesn't cover the more general case as it uses the fact that $I_{N \times N} \times I_{N \times N} = I_{N \times N}$ which is not the case for $W$.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.