2
$\begingroup$

I understand that a stochastic process or function is considered a Gaussian process if sampling from it at any point some set of times yields a set of observations that match a Gaussian random variable, and an other way to view this is that it can be described by a mean function with some kind of covariance function specifying the variance in each dimension between pairs of inputs. But I lack an intuition of what this really means or looks like.

Does saying something is a Gaussian process imply that it is a smooth function? Is there a good way to visualize making predictions with kriging in low-dimension, say a 3D plot with two dimensions the inputs and the third the model's guesses?

Edit: This part of this video is really helpful for the visualization question.

$\endgroup$
1
  • 2
    $\begingroup$ Smoothness is determined by the smoothness of the covariance (or variogram) at the origin. $\endgroup$ – whuber Nov 29 '17 at 17:29
7
$\begingroup$

Not all choices of kernel function yield a smooth function. The exponential kernel $K(x_i, x_j) = \exp\left(-\gamma d(x_i, x_j)\right)$ for $\gamma > 0$ and $d$ a valid distance is the covariance function to the Orenstein-Uhlenbeck process; the result is not a smooth function. More information can be found in Rassmussen and Williams, Gaussian Processes for Machine Learning.

$\endgroup$
2
  • 1
    $\begingroup$ How does the shape of our belief about the process reform when points are sampled and a non-smooth kernel is used? This image seems to capture the effects of choosing a few, but I lack an intuition of what sampling does with this kernel. It's not like plopping down a basis function as in b-splines, is it? Moreover, why choose these other kernels? What assumptions does each implicitly make about the underlying function? $\endgroup$ – Pavel Komarov Nov 29 '17 at 17:48
  • 2
    $\begingroup$ If you assume that your function is infinitely smooth, then the standard RBF kernel is reasonable. But real-world functions tend not to be so smooth, so a less-smooth kernel would be appropriate if modeling that is important to you. R&W discuss the Matérn kernel as admitting the exponential and RBF kernels as special cases, and suggest the choice of $\nu = \frac{5}{2}$ as a reasonable default choice between the two extremes. Chapter 4 of GPML discusses this in more detail; I think it would be very helpful for you to read that chapter. $\endgroup$ – Sycorax Nov 29 '17 at 18:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.