# Is kernel ridge regression the same as kernel regression?

I know the Nadaraya-Watson kernel regression. What is new to me is the Kernel ridge regression from scitkit-learn's KernelRidge with kernel='rbf'. It mentions that the kernel trick is used, which is not done in the Nadaraya-Watson kernel regression, so it seems to me they are different concepts. Am I right, or are they the same afterall?

## migrated from stackoverflow.comNov 9 '17 at 21:17

This question came from our site for professional and enthusiast programmers.

Yeah, you are right. You practically replace the square matrix $X^TX$ with a Kernel $K$ when you estimate your coefficients.
• I dropped it since it seems to the distinction seems to be confusing. Parametric regression models, I would argue, establish a clear dependency of Y on X given by the coefficients (dimension of coefficients is finite). This is also what kernelized ridge regression does, since your $beta$ captures this. Non-parametric models do not establish this dependency in the same way. Reading your comment I am not sure if you still can say that KRR is parametric, because the dependency Y on X is not so clear anymore due to the kernel. – Simon Dirmeier Nov 15 '17 at 10:00
• No, with beta I mean the coefficients you are estimating. The estimator for beta for classical ridge is $beta = (X^T X + alpha I)^{-1} X^T y$ which you can also solve like this: $beta = X^T (XX^T + alpha I)^{-1} y$. Beta is p-dimensional, as the number of covariates. In kernel ridge regression you replace $X X^T$ with a kernel. So, I would argue that this is still a parametric approach, if we take the definition of a parametric model being one that has coefficients with of finite dimensionality (p). However, since you operate in some high-dimensional feature space now, I might be wrong. – Simon Dirmeier Nov 15 '17 at 14:55