Log marginal likelihood of Gaussian Process for multiple-output regression

Question

The log marginal likelihood for Gaussian Process regression is calculated according to Chapter 5 of the Rasmussen and Williams GPML book:

$log\ p(y|X,\theta) = -\frac{1}{2}y^TK_y^{-1}y-\frac{1}{2}log|K_y|-\frac{n}{2}log2\pi$

It is straightforward to get a single log marginal likelihood value when the regression output is one dimension. But when it comes to multiple-output regression, the first term on the right side of the above equation is actually a matrix instead of a single value:

$-\frac{1}{2}y^TK_y^{-1}y$

So I am wondering how to handle this situation. I notice in many Gaussian Process implementations, people get the log marginal likelihood by summing the posterior pdf of each training sample, which is similar to the "LOO-CV" concept mentioned in Chapter 5 of the GPML book, but without really leaving one out during training. Is this a reasonable way to calculate the log marginal likelihood in Gaussian Process context? Thanks for any feedback in advance!

Answer 1

Terms like $y^tMy$ are scalar. It's known as a quadratic form. In your case, $M=K_y^{-1}$.

I assume that $K$ is a function of $\theta$, the vector of unknown parameters. You grab the partial derivatives with respect to the $\theta$ vector and solve for zero. Alternatively, one could maximize the log likelihood directly by using a numeric optimization algorithm.