Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I am learning gaussian processes from the book Bayesian Reasoning and Machine Learning, Chapter 19, page 400, section 19.1.2. A snapshot from the text,
I failed to understand the use of delta function here, why, the equation can't simply be written as, $$p(\mathbf{y}|\mathbf{x}) = \int_\mathbf{w} \mathbf{\Phi w} p(\mathbf{w})$$
Am I missing something trivial?
The integral is actually the following: $$p(\mathbf{y}|\mathbf{x})=\int p(\mathbf{y|\mathbf{x},\mathbf{w}})p(\mathbf{w})d\mathbf{w}$$ Here, $p(\mathbf{y|\mathbf{x},\mathbf{w}})$ is the distribution of $\mathbf{y}$ when we know $\mathbf{x}$ and $\mathbf{w}$, in which the relation is actually deterministic, i.e. $\mathbf{y}=\mathbf{\Phi(x)}\mathbf{w}$. So, $\mathbf{y}$ can only be single value, whose distribution is represented by dirac-delta functions to be able to do further analysis. This deterministic relation is automatically enforced because the delta function will ensure that no $\mathbf{w}$ value will be integrated if $\mathbf{y}\neq \mathbf{\Phi(x)}\mathbf{w}$, since $\delta(\mathbf{y}-\mathbf{\Phi(x)}\mathbf{w})$ is $0$ when the two are not equal. This is not the same as replacing $p(\mathbf{y|\mathbf{x},\mathbf{w}})$ with $\mathbf{\Phi(x)w}$.
