# Difference between kernel linear regression and non-parametric regression

A quick perplexity popped up in my mind while reading about non-parametric linear regression.

In linear regression, we model our response $$\textbf{y} \sim \mathcal{N}(X\beta, \sigma^2I)$$ so basically we try to estimate a linear function of the form

$$f_\beta(\textbf{x}_i) =\textbf{x}_{i,1}\beta_1, \dots, \textbf{x}_{i,p}\beta _p$$

while in non-parametric regression we allow more possibilities for the structure of $$f$$ and the response is modeled as

$$\textbf{y} \sim \mathcal{N}(f(x), \sigma^2I)$$

with $$f$$ respecting some smoothness assumptions.

What it's not too clear to me is what is the main difference between kernel linear regression and non-parametric one. It is well known that the word linear in linear regression refers to the parameters, so one typically applies a non-linear feature transformation $$\phi : \mathbb{R}^p \rightarrow \mathbb{R}^d$$ to the features and then searches for some hyperplane fitting the data (brought in higher dimension by the map $$\phi$$).

• Your characterizations of linear regression and non-parametric regression are not what one would expect: they both appear to be (extremely) limited ones. Thus, reviewing some of our posts about non-parametric models might resolve this question for you.
– whuber
Commented Oct 4, 2021 at 12:22
• Aren’t they exactly what is meant for linear and non parametric regression? Could you provide more details please? Commented Oct 4, 2021 at 12:35
• "Non-parametric" refers to distributional assumptions. Your characterization would correspond either to semi-parametric regression (if a wide set of possible $f$ is contemplated) or a parametric regression (if $f$ is finitely parameterized). Your characterization of linear regression is unusually restrictive in specifying Normal responses.
– whuber
Commented Oct 4, 2021 at 12:38

Kernel regression is one of the non-parametric regression models, so it cannot differ from non-parametric models. It is a model that uses kernels to approximate the expected value of the distribution of the data. Other non-parametric models may use different ways of achieving this, for example in case of $$k$$-NN regression the predicted mean would be just an average of the $$k$$ closest neighbors of the datapoint.