2
$\begingroup$

I understand that a stochastic process or function is considered a Gaussian process if sampling from it at any point some set of times yields a set of observations that match a Gaussian random variable, and an other way to view this is that it can be described by a mean function with some kind of covariance function specifying the variance in each dimension between pairs of inputs. But I lack an intuition of what this really means or looks like.

Does saying something is a Gaussian process imply that it is a smooth function? Is there a good way to visualize making predictions with kriging in low-dimension, say a 3D plot with two dimensions the inputs and the third the model's guesses?

Edit: This part of this video is really helpful for the visualization question.

$\endgroup$
1
  • 2
    $\begingroup$ Smoothness is determined by the smoothness of the covariance (or variogram) at the origin. $\endgroup$
    – whuber
    Nov 29, 2017 at 17:29

1 Answer 1

7
$\begingroup$

Not all choices of kernel function yield a smooth function. The exponential kernel $K(x_i, x_j) = \exp\left(-\gamma d(x_i, x_j)\right)$ for $\gamma > 0$ and $d$ a valid distance is the covariance function to the Orenstein-Uhlenbeck process; the result is not a smooth function. More information can be found in Rassmussen and Williams, Gaussian Processes for Machine Learning.

$\endgroup$
2
  • 1
    $\begingroup$ How does the shape of our belief about the process reform when points are sampled and a non-smooth kernel is used? This image seems to capture the effects of choosing a few, but I lack an intuition of what sampling does with this kernel. It's not like plopping down a basis function as in b-splines, is it? Moreover, why choose these other kernels? What assumptions does each implicitly make about the underlying function? $\endgroup$ Nov 29, 2017 at 17:48
  • 2
    $\begingroup$ If you assume that your function is infinitely smooth, then the standard RBF kernel is reasonable. But real-world functions tend not to be so smooth, so a less-smooth kernel would be appropriate if modeling that is important to you. R&W discuss the Matérn kernel as admitting the exponential and RBF kernels as special cases, and suggest the choice of $\nu = \frac{5}{2}$ as a reasonable default choice between the two extremes. Chapter 4 of GPML discusses this in more detail; I think it would be very helpful for you to read that chapter. $\endgroup$
    – Sycorax
    Nov 29, 2017 at 18:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.