# Is a function describable by a Gaussian process smooth?

I understand that a stochastic process or function is considered a Gaussian process if sampling from it at any point some set of times yields a set of observations that match a Gaussian random variable, and an other way to view this is that it can be described by a mean function with some kind of covariance function specifying the variance in each dimension between pairs of inputs. But I lack an intuition of what this really means or looks like.

Does saying something is a Gaussian process imply that it is a smooth function? Is there a good way to visualize making predictions with kriging in low-dimension, say a 3D plot with two dimensions the inputs and the third the model's guesses?

Edit: This part of this video is really helpful for the visualization question.

• Smoothness is determined by the smoothness of the covariance (or variogram) at the origin. – whuber Nov 29 '17 at 17:29

## 1 Answer

Not all choices of kernel function yield a smooth function. The exponential kernel $K(x_i, x_j) = \exp\left(-\gamma d(x_i, x_j)\right)$ for $\gamma > 0$ and $d$ a valid distance is the covariance function to the Orenstein-Uhlenbeck process; the result is not a smooth function. More information can be found in Rassmussen and Williams, Gaussian Processes for Machine Learning.

• How does the shape of our belief about the process reform when points are sampled and a non-smooth kernel is used? This image seems to capture the effects of choosing a few, but I lack an intuition of what sampling does with this kernel. It's not like plopping down a basis function as in b-splines, is it? Moreover, why choose these other kernels? What assumptions does each implicitly make about the underlying function? – Pavel Komarov Nov 29 '17 at 17:48
• If you assume that your function is infinitely smooth, then the standard RBF kernel is reasonable. But real-world functions tend not to be so smooth, so a less-smooth kernel would be appropriate if modeling that is important to you. R&W discuss the Matérn kernel as admitting the exponential and RBF kernels as special cases, and suggest the choice of $\nu = \frac{5}{2}$ as a reasonable default choice between the two extremes. Chapter 4 of GPML discusses this in more detail; I think it would be very helpful for you to read that chapter. – Sycorax Nov 29 '17 at 18:07