As the title says, is kernel regression a parametric or non-parametric method, and how can this be motivated/explained?
3
-
2$\begingroup$ What is it you want explained? The distinction between parametric and nonparametric models? Kernel regression? $\endgroup$– whuber ♦Mar 9, 2022 at 22:08
-
$\begingroup$ I want to know why Kernel Regression is considered a non-parametric regression method (Wikipedia)? I heard from someone that both the case of non-parametric and parametric can be argued for Kernel Regression, which made me curious. I am sorry if the question is not clear, maybe that is because of my own lack of understanding. $\endgroup$– AlexanderMar 9, 2022 at 22:13
-
1$\begingroup$ Thank you for the clarification. You might find material on "semi-parametric" methods will illuminate some of the issues. $\endgroup$– whuber ♦Mar 9, 2022 at 22:16
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Kernel regression is considered non-parametric.
It is tempting to think of the amount of optimal smoothing as a "parameter", nevertheless on that aspect to quote from Shalizi's Advanced Data Analysis from an Elementary Point of View Chapt. 4 "Using Nonparametric Smoothing in Regression": "Strictly speaking, parameters are properties of the data-generating process alone, so the optimal amount of smoothing is not really a parameter."
Our optimal amount of smoothing depends on our smoothing method, kernel choice, but also on how much data we have. As such as the number of data is our "parameters" in a kernel regression setting, kernel regression is non-parametric in itself. This fully aligns with the definition of a non-parametric model as a model "that cannot be parametrized by a finite sample of parameters" (from Wasserman's All of Statistics Chapt. 7 "Models, Statistical Inference and Learning").
