# Predictive Distribution in Gaussian Process Derivation

In Gaussian Process for Machine Learning (Rasmussen and Williams), pg 11, we are given the following predictive distribution:

$$p\left(f_{*} | \mathbf{x}_{*}, X, \mathbf{y}\right)=\int p\left(f_{*} | \mathbf{x}_{*}, \mathbf{w}\right) p(\mathbf{w} | X, \mathbf{y}) d \mathbf{w}$$

We are also given the definition: $$f_{*} \triangleq f\left(\mathbf{x}_{*}\right)$$ where $$f(\mathrm{x})=\mathrm{x}^{\top} \mathrm{w}$$ and $$y=f(\mathbf{x})+\varepsilon$$.

I understand that one may compute the distribution shown above by doing the following:

$$E[f_{*}] = \mathrm{x}^{\top}E[\mathrm{w}]$$ $$V[f_{*}] = \mathrm{x}^{\top}V[\mathrm{w}]\mathrm{x}$$

We then plug in the expressions for the mean and variance of our posterior over w. Essentially, we are computing the mean and variance of a scaled Gaussian random variable as we already established that the posterior distribution over w is Gaussian.

However, I have a problem with the integral shown above. I understand how to arrive at this expression (sum rule, product rule, and conditional independence). However, the $$p\left(f_{*} | \mathbf{x}_{*}, \mathbf{w}\right)$$ term is bothering me.

I have seen derivations where this term is replaced with $$p\left(y_{*} | \mathbf{x}_{*}, \mathbf{w}\right)$$. This makes more sense to me as, due to the Gaussian noise, this term would be a Gaussian distribution and the integral of the product of two Gaussian distributions is a Gaussian distribution (provided proper normalisation). However, wouldn't $$p\left(f_{*} | \mathbf{x}_{*}, \mathbf{w}\right)$$ be a delta function at $$f_{*} = \mathrm{x}^{\top}_{*} \mathrm{w}$$? Am I missing something here?

Any advice would be greatly welcomed! Cheers!

• I think you should add the self study tag. May 26, 2019 at 3:45
• @MichaelChernick Just added it, thanks for the suggestion.
– JKB
May 26, 2019 at 15:08