22
$\begingroup$

I'm know that Gaussian Process Regression (GPR) is an alternative to using splines for fitting flexible nonlinear models. I would like to know in which situations would one be more suitable than the other, especially in the Bayesian regression framework.

I've already looked at What are the advantages / disadvantages of using splines, smoothed splines, and gaussian process emulators? but there does not seem to be anything on GPR in this post.

Improve this question
$\endgroup$
1
  • $\begingroup$ I would say that GP is a more data-driven approach to fitting a non-linear function. The splines are typically restricted to n-th polynomial. The GPs can model more complex functions than polynomials (not 100% sure though). $\endgroup$
    – Vladislavs Dovgalecs
    Commented Apr 3, 2015 at 7:01

3 Answers 3

Reset to default
24
$\begingroup$

I agree with @j__'s answer.

However, I would like to highlight the fact that splines are just a special case of Gaussian Process regression/kriging.

If you take a certain type of kernel in Gaussian process regression, you exactly obtain the spline fitting model.

This fact is proven in this paper by Kimeldorf and Wahba (1970). It is rather technical, as it uses the link between the kernels used in kriging and Reproducing Kernel Hilbert Spaces (RKHS).

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ As an example, in the one-dimensional case, the GP model for the famous smoothing spline is simply a doubly integrated Gaussian White Noise. This has been used by Craig Ansley and Robert Kohn to design efficient algorithms at the end of the 1980s. I believe that this equivalence can be partially understood without diving in the deep maths of RKHS. $\endgroup$
    – Yves
    Commented May 6, 2016 at 10:27
  • $\begingroup$ This is a very good answer. $\endgroup$
    – Astrid
    Commented Jan 29, 2017 at 23:49
6
$\begingroup$

It is a very interesting question: The equivalent between Gaussian processes and smoothing splines has been shown in Kimeldorf and Wahba 1970. The generalization of this correspondence in the case of constrained interpolation has been developed in Bay et al. 2016.

Bay et al. 2016. Generalization of the Kimeldorf-Wahba Correspondence for constrained interpolation. Electronic Journal of Statistics.

In this paper, the advantage of the Bayesian approach has been discussed.

Improve this answer
$\endgroup$
2
$\begingroup$

I agree with @xeon's comment also GPR puts a probability distribution over infinite number of possible functions and the mean function (which is spline like) is only the MAP estimate but you also have a variance about that. This allows for great opportunities such as experimental design (choosing input data which is maximally informative). Also if you want to performs integration (quadrature) of the model a GP will have a gaussian result which allows you to give confidence to your result. At least with standard spline models this is not possible.

In practice GPR gives a more informative result (in my experience) but spline models seem to be quicker in my experience.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.