2
$\begingroup$

I'm working with the Nadaraya Watson estimator and read the book of Wand and Jones (Kernel Smoothing, 1995) for introduction. On page 117 (for those who have the book) there is written that with larger bandwidth h the estimate tends towards the least squares line through the data.

I would have said that with an increasing h tends the estimate to the average but not necessarily to the least squares line?

$\endgroup$
  • $\begingroup$ (I unintentionally deleted my first post) $\endgroup$ – To Mate Dec 11 '18 at 13:56
3
$\begingroup$

You are indeed right that the Nadaraya–Watson estimator would tend to the average since

$$ {\widehat {m}}_{h}(x)={\frac {\sum _{i=1}^{n}K_{h}(x-X_{i})Y_{i}}{\sum _{j=1}^{n}K_{h}(x-X_{j})}} $$

also can be written as

$$ {\widehat {m}}_{h}(x) = \arg\min_{\beta_0(x)\in\mathbb{R}} \sum_{i=1}^nK_h(x-X_i)(\beta_0(x)-Y_i)^2. $$

That is, we are locally fitting a constant term at each $x$. It is the bandwidth $h$ what shows how local we are: the larger $h$, the more points are taken into account. Hence, as you said, as $h\to\infty$, ${\widehat {m}}_{h}(x)$ tends to the average of $Y$ for each $x$.

A natural extension of this is to consider fitting something more flexible than a constant. In particular, a locally linear case would be

$$ {\widehat {m}}_{1,h}(x) = \arg\min_{\beta_0(x)\in\mathbb{R}} \sum_{i=1}^nK_h(x-X_i)(\beta_0(x)+\beta_1(x-X_i)-Y_i)^2. $$

(See equation (5.4) in your referred book for a closed-form solution.) Now it at each $x$ fits a model with a constant and a linear term, where again it is $h$ what shows how "locally" we look at $X_i$: the larger $h$, the less smooth the fitted line will be. In particular, as $h\to\infty$, we take all the points with equal weights and indeed get the OLS line. That is what is discussed at your referred book, page 117.

Here's an illustration of my answer:

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.