# Can the covariance matrix in a Gaussian Process be non-symmetric?

I was watching a lecture on Gaussian Process and when the covariance matrix was introduced, the tutor explained that the matrix is $$(n \times n)$$ because every point is covered twice - we include the information about the covariance of $$(x_1, x_2)$$ and about the covariance of $$(x_2, x_1)$$. She then said: that's because your covariance can vary in different directions.

How is it possible that the covariances can vary in different directions inside the GP covariance matrix? Could you give me an example of when that could be the case?

Update:

After giving it some thought, I realized that it is not $$cov(x_1, x_2)$$ or $$cov(x_2, x_1)$$ (as computed from the definition of covariance) that go into the GP covariance matrix, but instead (as was shown in the lecture as well), the covariance matrix is populated by a covariance kernel $$k(x, y)$$ that acts/is interpreted as a covariance, but it is some function of the distance between $$x$$ and $$y$$.

I could therefore imagine, that we might have a covariance kernel that is a function of $$(x - y)^p$$ where $$p$$ is an odd power. In such instance, it would indeed make $$k(x_1, x_2) \neq k(x_2, x_1)$$. But would this be a valid kernel?

Could you clarify if my thinking about the covariance kernel is reasonable? Could you explain if the covariance matrix in Gaussian Process can be non-symmetric? If yes, could you give an example of a dataset where it would make sense to make covariance different in different directions, i.e. where we would like $$k(x_1, x_2)$$ to be different from $$k(x_2, x_1)$$?

• I am giving it some thought and I have a possible explanation why it might make sense. The covariance matrix does not really contain $cov(x, y)$ but instead, it is populated by a function $k(x, y)$ that "acts" as a covariance (it is later interpreted as a covariance between $x$ and $y$, however, the function can indeed take a different value for $k(x, y)$ and $k(y, x)$). Since in the lecture it was said that the covariance kernel is a function of distance between $x$ and $y$, it might indeed make sense for kernels where there is $x - y$ raised to an odd power. Does my reasoning make sense? Nov 4, 2018 at 19:26
• In a gaussian process, the matrix $K(x_1, x_2)$ is always used as a covariance matrix for a multivariate normal, hence, it must be symmetric. Nov 5, 2018 at 10:12
• @InfProbSciX Don't overlook Wikipedia's qualifier "For most applications..." There's no inherent mathematical necessity that kernels be symmetric. For instance, an exponentially weighted moving average (EWMA) of a time series is a windowed mean using an asymmetric kernel.
– whuber
Feb 1, 2019 at 15:12
• @Mathtick You seem to confuse two separate concepts: In a Gaussian process, all finite-dimensional marginal distributions are Gaussian; but the kernel describes how those distributions vary with the spatial configuration. That's a separate issue.
– whuber
Oct 1, 2021 at 14:05
• @mathtick This might be a matter of what community one works within. In the spatial statistics literature such asymmetric "kernels" are contemplated, but often the simplifying assumption of symmetry is quickly made. Spatio-temporal modeling is a natural area of applications where asymmetric kernels can be expected, so ruling them out would just be unconstructive.
– whuber
Oct 1, 2021 at 16:47