Is kernel ridge regression the same as kernel regression? I know the Nadaraya-Watson kernel regression. What is new to me is the Kernel ridge regression from scitkit-learn's KernelRidge with kernel='rbf'. It mentions that the kernel trick is used, which is not done in the Nadaraya-Watson kernel regression, so it seems to me they are different concepts. Am I right, or are they the same afterall?
 A: Yeah, you are right. You practically replace the square matrix $X^TX$ with a Kernel $K$ when you estimate your coefficients.
A: No, they are not the same as algorithms, though you might be able to find pairs of kernel where they give the same answer (model) in terms of predictors.
Kernel Regression (simplest form) is a density estimator with mean prediction:
$$
\begin{align*}
\mu_{\text{kernel-regression}} &= \sum_i w_i y_i, \quad w_i = \frac{K(X^*, X_i)}{\sum_j K(X^*, X_j)} \\
\end{align*}
$$
while Kernel Ridge Regression is a regression (least-squares type inversion) with mean prediction:
$$
\mu_{\text{kernel-ridge-regression}}  = \sum_i w_i y_i, \quad w_i = \sum_j K(X^*, X_j) K_{ji}^{-1}(X, X) 
$$
And as noted before, the "kernel" part of kernel ridge regression is that you practically replace the square matrix $X^TX$ with a Kernel $K$ when you estimate your coefficients.
I have no idea why "kernel regression" means "kernel density esimation" ... if someone knows the history, or knows of complete results on mapping between the two methods please update this answer.
This is related:
Is Kernel Regression similar to Gaussian Process Regression?
You can see discussiona about his in Kevin Murphy's (newer) book as well as the older Rasmussen book.
