# What does the covariance matrix of a Gaussian Process look like?

Let's assume $\{X_t\}_{t \geq0}$ is a Gaussian Process with a certain covariance matrix. In order for this process to be stationary the mean vector should not depend on $t$ and the covariance function should not depend on $t$. However, what additional restrictions should the covariance have?

In order for the process to be stationary, the variance of each realization should be the same. So,that means that the diago$nal elements of the covariance matrix should all be equal to the commn variance. Am I wrong? In general what does the covariance matrix of a stationary Gaussian process look like? ## 2 Answers Here is a somewhat informal explanation: The covariance matrix for a Gaussian process is a gram matrix obtained by evaluating some kernel function$k(x, x')$pairwise between a set of observations. Stationary in the context of a Gaussian process implies that the covariance between two points, say$x$and$x'$, would be identical to the covariance between the same two points perturbed by some$\Delta$, i.e:$k(x, x') = k(x+\Delta, x' + \Delta)$This implies that the hyper-parameters of$k$(if they exist) do not vary across the index (here$x$). As an example, the popular exponetiated quadratic (also called the squared exponential, or "RBF") kernel is stationary:$k(x,x') = \alpha^2 e^{ \frac{- (x - x')^2}{2 l^2} } + \delta_{ij} \sigma_0^2$($\delta_{ij}$being the kronecker delta) Because the hyperparameters ($\alpha, l, \sigma_0$) have no dependency on the index,$x$. If, for example, the lengthscale$l$would be permitted to vary over$x$, the covariance between any two points$k(x,x')$would not necessarily be the same as$k(x + \Delta,x'+ \Delta)$. If the covariance kernel is stationary, one can see that for the diagonal elements of the covariance matrix we will be evaluating$k(x,x)$. Since the pairwise distance between$x$and itself is zero, in the case of the kernel above, the covariance collapses to:$k(x,x) = \alpha^2 + \sigma_0^2$Implying all diagonal elements of the unconditional covariance matrix will be identical, as neither$\alpha$nor$\sigma_0$depend on$x\$.

The variance covariance matrix of a stationary Gaussian process should have the same value for all its diagonal elements. Its auto-covariance should also be the same through time. In other words, the off-diagonal lines that are parallel to the diagonal should have the same values in each line. And obviously, the matrix needs to be positive semi-definite.