Viewing kernel regression in a Bayesian framework If one wanted to use Kernel Regression in a Bayesian Framework, any ideas on how one would go about it? 
Kernel Regression
 A: Gaussian processes might be something worth looking at (although in machine learning kernel methods mean something slightly different).  Essentially if you use a squared exponential covariance function, you end up with something like a Bayesian radial basis function regression model, with a prior over the function implemented by the model rather than its parameters.  There is a very nice book (with MATLAB software) by Rasmussen and Williams.
A: All I can do is an educated guess: on ICML this year, there is a paper Support Vector Machines as Probabilistic Models by Vojtech Franc, Alexander Zien, Bernhard Schölkopf. You might find something in there on how to formulate SVR as a probabilistic model and thus use it in a Bayesian framework.
Looks like a tough road, though.
A: In addition to answers based on Gaussian processes given by other answers, it is possible to do something which resembles (to a certain extent) Nadaraya-Watson style regression using Dirichlet process mixture models. See, for example, this paper by Mueller et al. They discuss this connection very briefly on page 70. The analogy here isn't perfect. And the method suffers some pretty serious drawbacks (due to the fact that one must implicitly model the covariates), such that Bayesians don't really like using this approach anymore.
A: Check out Bayesian Kernel Machine Regression (BKMR) and its many adaptations. 
There is a fantastic R package to support BKMR with great documentation. It is simply titled 'bkmr'. 
Check out this citation to learn more:
Bobb, Jennifer F., et al. "Bayesian kernel machine regression for estimating the health effects of multi-pollutant mixtures." Biostatistics 16.3 (2014): 493-508.
