Let's assume $\{X_t\}_{t \geq0} $ is a Gaussian Process with a certain covariance matrix. In order for this process to be stationary the mean vector should not depend on $t$ and the covariance function should not depend on $t$. However, what additional restrictions should the covariance have?

In order for the process to be stationary, the variance of each realization should be the same. So,that means that the diago$nal elements of the covariance matrix should all be equal to the commn variance. Am I wrong? In general what does the covariance matrix of a stationary Gaussian process look like?


2 Answers 2


Here is a somewhat informal explanation:

The covariance matrix for a Gaussian process is a gram matrix obtained by evaluating some kernel function $k(x, x')$ pairwise between a set of observations.

Stationary in the context of a Gaussian process implies that the covariance between two points, say $x$ and $x'$, would be identical to the covariance between the same two points perturbed by some $\Delta$, i.e:

$k(x, x') = k(x+\Delta, x' + \Delta)$

This implies that the hyper-parameters of $k$ (if they exist) do not vary across the index (here $x$). As an example, the popular exponetiated quadratic (also called the squared exponential, or "RBF") kernel is stationary:

$k(x,x') = \alpha^2 e^{ \frac{- (x - x')^2}{2 l^2} } + \delta_{ij} \sigma_0^2$

($\delta_{ij}$ being the kronecker delta)

Because the hyperparameters ($\alpha, l, \sigma_0$) have no dependency on the index, $x$.

If, for example, the lengthscale $l$ would be permitted to vary over $x$, the covariance between any two points $k(x,x')$ would not necessarily be the same as $k(x + \Delta,x'+ \Delta)$.

If the covariance kernel is stationary, one can see that for the diagonal elements of the covariance matrix we will be evaluating $k(x,x)$. Since the pairwise distance between $x$ and itself is zero, in the case of the kernel above, the covariance collapses to:

$k(x,x) = \alpha^2 + \sigma_0^2$

Implying all diagonal elements of the unconditional covariance matrix will be identical, as neither $\alpha$ nor $\sigma_0$ depend on $x$.


The variance covariance matrix of a stationary Gaussian process should have the same value for all its diagonal elements. Its auto-covariance should also be the same through time. In other words, the off-diagonal lines that are parallel to the diagonal should have the same values in each line. And obviously, the matrix needs to be positive semi-definite.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.