I am learning gaussian processes from the book Bayesian Reasoning and Machine Learning, Chapter 19, page 400, section 19.1.2. A snapshot from the text, snapshot_from_book

I failed to understand the use of delta function here, why, the equation can't simply be written as, $$p(\mathbf{y}|\mathbf{x}) = \int_\mathbf{w} \mathbf{\Phi w} p(\mathbf{w})$$

Am I missing something trivial?


1 Answer 1


The integral is actually the following: $$p(\mathbf{y}|\mathbf{x})=\int p(\mathbf{y|\mathbf{x},\mathbf{w}})p(\mathbf{w})d\mathbf{w}$$ Here, $p(\mathbf{y|\mathbf{x},\mathbf{w}})$ is the distribution of $\mathbf{y}$ when we know $\mathbf{x}$ and $\mathbf{w}$, in which the relation is actually deterministic, i.e. $\mathbf{y}=\mathbf{\Phi(x)}\mathbf{w}$. So, $\mathbf{y}$ can only be single value, whose distribution is represented by dirac-delta functions to be able to do further analysis. This deterministic relation is automatically enforced because the delta function will ensure that no $\mathbf{w}$ value will be integrated if $\mathbf{y}\neq \mathbf{\Phi(x)}\mathbf{w}$, since $\delta(\mathbf{y}-\mathbf{\Phi(x)}\mathbf{w})$ is $0$ when the two are not equal. This is not the same as replacing $p(\mathbf{y|\mathbf{x},\mathbf{w}})$ with $\mathbf{\Phi(x)w}$.

  • $\begingroup$ So in other words we are taking a subset of $\mathbf{w}$'s that satisfied the condition 𝐲=𝚽(𝐱)𝐰 , but is not it true that whatever the w is y will always be equal to 𝚽(𝐱)𝐰 ? That's how we wrote the equation, and created it deterministic. $\endgroup$ Sep 21, 2019 at 21:09
  • $\begingroup$ 𝚽(𝐱)𝐰 gives you a y but what if you ask for the density of another y given the same 𝐱,𝐰? $\endgroup$
    – gunes
    Sep 21, 2019 at 21:14
  • $\begingroup$ Ahh, I see, because it has to be joint distribution so all possible (x,y) combination has to be described. Thanks, I think I understand it now. $\endgroup$ Sep 21, 2019 at 21:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.