Nonparametric estimation of regression function: kernel estimation vs series estimation

Question

I am working on a small research project trying to estimate regression function nonparametrically when I have only one regressor. Basically, I am trying to estimate the regression function $$r(x)=E[Y∣X=x] $$ when I have i.i.d. pairs $(X_i,Y_i)$, $i=1,\ldots,n$.

I looked through the literature and I found that there are two main nonparametric techniques employed:

• using kernels
• series estimation using polynomials

My questions are:

• Are there any any rules of thumb that I shall use when selecting one of these nonparametric procedures?
• Does either of the above two approaches have some well known statistical or approximation advantages over the other?

Answer 1

Before talking about the two main factors for the choice between a kernel and a series method, it is important that you think about the intuition behind each method.

With the kernel method, you are basically calculating locally weighted averages of your data points. Thus, there is kind of a local fitting. In contrast, the series method aims to do kind of a global fitting as the functions (in your case polynomials) usually live on the entire real line.

So when deciding about which method to use, it is crucial to know the characteristics of your data. In particular, the decision depends on (a) outliers in the data and (b) the local scarcity of the data.

(a) Outliers : if you have many outliers in your dataset, it is probably better to use the kernel method as only the local average at a particular outlier point is influenced by that outlier. In comparison, if you would use a series method, that particular outlier influences a whole function (e.g. the coefficient for your $x^2$).

(b) Local scarcity : if there are ranges in your sample for which you do not have a lot of data, it is probably better to use the series method since your functions (again, your polynomials) usually live and hence are fitted on the whole real line. On the contrary, if you were to use the kernel method, the local scarcity of data would deteriorate your estimate in that particular range as the local average is calculated with only a few data points.

In terms of your second question, I would, at least for the approximation advantages, again refer to what I stated above. For statistical advantages such as techniques for asymptotic proofs, the two approaches are just completely different and to my knowledge, there aren't any hard facts about "advantages".