# Marginal likelihood of a Gaussian Process?

I've been studying Gaussian processes (GP), this resource implements GP from scratch and has been very useful in visualizing what's happening. So far all has made sense to me except for the below equation (eq 11 in link), the log marginal likelihood of the GP:

$$-1/2 [Y^{T} K_y^{-1}Y] -1/2 [log(|K_y|)] - N/2[log(2 \pi)]$$

The author explains that this step is necessary to optimize the hyperparameters of the kernel function. I've used some algebra and found that this is simply the log multivariate gaussian PDF:

$$\frac{exp(-1/2 Y^{T}K_y^{-1}Y)}{|K_y|^{1/2}(2\pi)^{N/2}}$$

My question is- what has been marginalized out? I gather it's either the old data (training) or the new data (at inference). But I'm not sure which. Intuition and context lead me to believe that we're optimizing kernel hyperparamters for the new data (else why not just accomplish this step during in the prior?)

However, the author defines {$$K,K_*, K_{**}$$} as $$K(old, old)$$, $$K(old, new)$$, and $$K(new, new)$$. I'm not sure which of the above $$K_y$$ is supposed to represent, what has been marginalized out, and some intuitions as to why it's been marginalized out. Could anyone comment on the what/why of what's going on here?

$$K_y$$ represents $$K(old, old)$$. You are marginalizing out the Prior over function values, which is explained in the "Prior" section of the blog. By marginalizing out the prior over function values you are able to optimize parameters of kernel function(in the case of RBF kernel it is $$l$$ and $$\sigma_f$$) based on the information you acquire from the training data.
• This optimization technique seems akin to MLE. I'm not seeing anything particularly Bayesian in the author's write up, other than the use of words, prior & posterior. In a Bayesian design, would the parameters, {$l, \sigma_f$} be sampled? (Or am I confused?) Jan 2, 2021 at 15:46
• I think you are confused because author does not derive the results in functions 4 and 5. Those results deriven from posterior over function values, with the assumption poserior follows a gaussian distrb. Parameters $\{l, \sigma_f \}$ are estimated by MLE because they are required to calculate mean and covariance which are obtained from posterior. "95% confidence interval" can be used in bayesian setting too as long as you are able to estimate distrb, difference is in the estimation of distrb. not the usage of confidence interval. Chapter 2 of the book explains your questions in detail. Jan 2, 2021 at 16:44