Why does a sample from multivariate Gaussian distribution look continuous?

Question

I am familiarizing myself with multivariate Gaussian distributions and Gaussian processes using george, and I am trying to understand how sampling from a multivariate Gaussian distribution works.

The following is a sample from the prior of a multivariate Gaussian distribution, using numpy.random.multivariate_normal. Each value of x on the graph corresponds to one dimension in the distribution - the graph was generated with 500 values of x equally spaced between 0 and 10. The multivariate Gaussian distribution was generated with mean at 0.0 across all 500 dimensions, and a covariance matrix generated using a squared exponential kernel.

As can be seen above, the sample appears "continuous".

My understanding is that adjacent points on the graph (corresponding to the values sampled from each dimension) are related to each other purely by the covariance matrix - so the fact that the sample looks like a continuous function stems from the covariance causing values in "adjacent" dimensions (?) to be closer to each other. Is my understanding correct, or is this because of the particular way that numpy samples from a multivariate_normal, or maybe because of some other reasons?

Answer 1

As you use a squared exponential kernel i.e. $$ k(x, x,') = \exp(-\theta (x - x')^2), $$ then you get bigger covariance if $x$ and $x'$ are closer. If we take the limit, then we get $$ \lim_{x \rightarrow x'} k(x, x') = 1. $$ So our intuition says to us that difference between values of realizations of multivariate Gaussian for close points should be small.

More formally, as our covariance function is infinitely differentiable, then for Gaussian random process we get infinitely differentiable realizations.

Also from the figures you can see how values of $\theta$ affects continuity of the process (there are three random realizations for each value of $\theta$)