12
$\begingroup$

I'm reading through this paper on Generalised Wishart Processes (GWP). The paper calculates the covariances between different random variables (following Gaussian Process) using squared exponential covariance function, i.e., $K(x,x') = \exp\left(-\frac{|(x-x')|^2}{2l^2}\right)$. It then says that this covariance matrix follows GWP.

I used to think that a covariance matrix computed from linear covariance function ($K(x,x') = x^Tx'$), follows Wishart Distribution with appropriate parameters.

My question is, how can we still assume the covariance to follow a Wishart distribution with squared exponential covariance function? Also, in general, what is the necessary condition for a covariance function to produce a Wishart distributed covariance matrix?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
10
+50
$\begingroup$

What is mixed up is the covariance specification in terms of the ambient space on which the Gaussian process is defined, and the operation that transforms a finite dimensional Gaussian random variable to yield a Wishart distribution.

If $\mathbf{X} \sim \mathcal{N}(0, \Sigma)$ is a $p$-dimensional Gaussian random variable (a column vector) with mean 0 and covariance matrix $\Sigma$, the distribution of $\mathbf{W} = \mathbf{X} \mathbf{X}^T$ is a Wishart distribution $W_p(\Sigma, 1)$. Note that $\mathbf{W}$ is a $p \times p$ matrix. This is a general result about how the quadratic form $$\mathbf{x} \mapsto \mathbf{x} \mathbf{x}^T$$ transforms a Gaussian distribution to a Wishart distribution. It holds for any choice of positive definite covariance matrix $\Sigma$. If you have i.i.d. observations $\mathbf{X}_1, \ldots, \mathbf{X}_n$ then with $\mathbf{W}_i = \mathbf{X}_i \mathbf{X}_i^T$ the distribution of $$\mathbf{W}_{1} + \ldots + \mathbf{W}_n$$ is a Wishart $W_p(\Sigma, n)$-distribution. Dividing by $n$ we get the empirical covariance matrix $-$ an estimate of $\Sigma$.

For Gaussian processes there is an ambient space, lets say for illustration that it is $\mathbb{R}$, such that the random variables considered are indexed by elements in the ambient space. That is, we consider a process $(X(x))_{x \in \mathbb{R}}$. It is Gaussian (and for simplicity, here with mean 0) if its finite dimensional marginal distributions are Gaussian, that is, if $$\mathbf{X}(x_1, \ldots, x_p) := (X(x_1), \ldots, X(x_p))^T \sim \mathcal{N}(0, \Sigma(x_1, \ldots, x_p))$$ for all $x_1, \ldots, x_p \in \mathbb{R}$. The choice of covariance function, as mentioned by the OP, determines the covariance matrix, that is, $$\text{cov}(X(x_i), X(x_j)) = \Sigma(x_1, \ldots, x_p)_{i,j} = K(x_i, x_j).$$ Disregarding the choice of $K$ the distribution of $$\mathbf{X}(x_1, \ldots, x_p) \mathbf{X}(x_1, \ldots, x_p)^T$$ will be a Wishart $W_p(\Sigma(x_1, \ldots, x_p), 1)$-distribution.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for answering this. I've a few questions, reg. your answer -When you say the transformation that transforms Gaussian dist to Wishart dist holds for any choice of +ve definite cov matrix, what diff choices do we have for this cov matrix? Also, just to clarify- for the cov matrix defined by cov function, i and j indicate elements in the ambient space of Gaussian Process (for e.g. if it's a temporal process then, time instants t_1 and t_2)? $\endgroup$
    – steadyfish
    Commented Apr 5, 2013 at 16:47
  • $\begingroup$ @steadyfish, yes, the indices $i$ and $j$ refer to the points $x_i$ and $x_j$ in the ambient space, and for a temporal process to two time points. Covariance matrices are always positive (semi)definite. The formulation was not intended to restrict the result in any way, but rather to emphasize that it holds for any choice of $\Sigma$ $-$ as long as $\Sigma$ is a covariance matrix. I left out the possibility that $\Sigma$ can be just semidefinite to avoid cluttering the answer with irrelevant issues on singular normal distributions etc. $\endgroup$
    – NRH
    Commented Apr 5, 2013 at 17:46
  • $\begingroup$ Thanks @NRH. I get the point about ambient space. About Covariance matrix, my question was whether there is any other way of defining covariance matrix apart from $x^{T}x$ (and not about positive definite or positive semidefinite property). (I hope the question is clear this time around!) $\endgroup$
    – steadyfish
    Commented Apr 5, 2013 at 18:19
  • $\begingroup$ @steadyfish, oh, I see. In fact, I was sloppy with the transpositions and whether the vectors were row or column vectors. I have made that precise now and added a little about the relation between the empirical covariance matrix and the theoretical covariance matrix. The theoretical is not defined in terms of the observations. $\endgroup$
    – NRH
    Commented Apr 5, 2013 at 19:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.