# Covariance matrix for Gaussian Process and Wishart distribution

I'm reading through this paper on Generalised Wishart Processes (GWP). The paper calculates the covariances between different random variables (following Gaussian Process) using squared exponential covariance function, i.e., $K(x,x') = \exp\left(-\frac{|(x-x')|^2}{2l^2}\right)$. It then says that this covariance matrix follows GWP.

I used to think that a covariance matrix computed from linear covariance function ($K(x,x') = x^Tx'$), follows Wishart Distribution with appropriate parameters.

My question is, how can we still assume the covariance to follow a Wishart distribution with squared exponential covariance function? Also, in general, what is the necessary condition for a covariance function to produce a Wishart distributed covariance matrix?

If $\mathbf{X} \sim \mathcal{N}(0, \Sigma)$ is a $p$-dimensional Gaussian random variable (a column vector) with mean 0 and covariance matrix $\Sigma$, the distribution of $\mathbf{W} = \mathbf{X} \mathbf{X}^T$ is a Wishart distribution $W_p(\Sigma, 1)$. Note that $\mathbf{W}$ is a $p \times p$ matrix. This is a general result about how the quadratic form $$\mathbf{x} \mapsto \mathbf{x} \mathbf{x}^T$$ transforms a Gaussian distribution to a Wishart distribution. It holds for any choice of positive definite covariance matrix $\Sigma$. If you have i.i.d. observations $\mathbf{X}_1, \ldots, \mathbf{X}_n$ then with $\mathbf{W}_i = \mathbf{X}_i \mathbf{X}_i^T$ the distribution of $$\mathbf{W}_{1} + \ldots + \mathbf{W}_n$$ is a Wishart $W_p(\Sigma, n)$-distribution. Dividing by $n$ we get the empirical covariance matrix $-$ an estimate of $\Sigma$.
For Gaussian processes there is an ambient space, lets say for illustration that it is $\mathbb{R}$, such that the random variables considered are indexed by elements in the ambient space. That is, we consider a process $(X(x))_{x \in \mathbb{R}}$. It is Gaussian (and for simplicity, here with mean 0) if its finite dimensional marginal distributions are Gaussian, that is, if $$\mathbf{X}(x_1, \ldots, x_p) := (X(x_1), \ldots, X(x_p))^T \sim \mathcal{N}(0, \Sigma(x_1, \ldots, x_p))$$ for all $x_1, \ldots, x_p \in \mathbb{R}$. The choice of covariance function, as mentioned by the OP, determines the covariance matrix, that is, $$\text{cov}(X(x_i), X(x_j)) = \Sigma(x_1, \ldots, x_p)_{i,j} = K(x_i, x_j).$$ Disregarding the choice of $K$ the distribution of $$\mathbf{X}(x_1, \ldots, x_p) \mathbf{X}(x_1, \ldots, x_p)^T$$ will be a Wishart $W_p(\Sigma(x_1, \ldots, x_p), 1)$-distribution.
• @steadyfish, yes, the indices $i$ and $j$ refer to the points $x_i$ and $x_j$ in the ambient space, and for a temporal process to two time points. Covariance matrices are always positive (semi)definite. The formulation was not intended to restrict the result in any way, but rather to emphasize that it holds for any choice of $\Sigma$ $-$ as long as $\Sigma$ is a covariance matrix. I left out the possibility that $\Sigma$ can be just semidefinite to avoid cluttering the answer with irrelevant issues on singular normal distributions etc. – NRH Apr 5 '13 at 17:46
• Thanks @NRH. I get the point about ambient space. About Covariance matrix, my question was whether there is any other way of defining covariance matrix apart from $x^{T}x$ (and not about positive definite or positive semidefinite property). (I hope the question is clear this time around!) – steadyfish Apr 5 '13 at 18:19