# Conceptual Difference between $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$ and $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y})$

So the predictive distribution of the Gaussian process is provided as follows where $$p(\mathbf{f} \lvert \mathbf{X},\mathbf{y})$$ and $$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y})$$ is the predictive distribution, taken from Martin Krasser blog

$$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y}) = \int{p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})p(\mathbf{f} \lvert \mathbf{X},\mathbf{y})}\ d\mathbf{f} \\ = \mathcal{N}(\mathbf{f}_* \lvert \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*)$$

Now what I am confused with is that isnt $$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$$ supposed to give us the predictive distribution $$\bf{f_*}$$ conditioned on the query point and posterior distribution $$\bf{f}$$? Here we already conditioned on posterior distribution $$\bf{f}$$ so why are we multiplying again with $$p(\mathbf{f} \lvert \mathbf{X},\mathbf{y})$$ and integrating?

What is "conceptual" difference between $$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$$ and $$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y})$$?

The actual relationship by law of total probability is as follows: $$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y}) = \int{p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f},\mathbf{X}, \mathbf{y})p(\mathbf{f} \lvert \mathbf{X},\mathbf{y},\mathbf{X_*})}\ d\mathbf{f}$$

It's just within this context, we have:

$$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f},\mathbf{X}, \mathbf{y})=p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$$ because when you know $$\mathbf{f}$$, you won't need $$\mathbf{X},\mathbf{y}$$. Similarly, the second term reduces to $$p(\mathbf{f|\mathbf{X},\mathbf{y}})$$ since $$\mathbf{f}$$ is fed by $$\mathbf{X,y}$$, and having only $$\mathbf{X}_*$$ is not of any use.

• For your second comment: The relation between $\mathbf{f,X,y}$ is not deterministic, so $X,y$ are redundant because we only need $f$ to find $f_*$ from $X_*$. Otherwise $p(\mathbf{f|X,y})$ would be dirac delta. – gunes Feb 10 '20 at 15:23
• your first comment makes so much sense, if the relation between $\mathbf{f,X,y}$ deterministic, we could simply have written $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$, since it is not deterministic we write $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f},\mathbf{X}, \mathbf{y})$ right? – GENIVI-LEARNER Feb 10 '20 at 15:37
• it is similar to that, we're considering all possible f, and find the distribution of $f_*$. A distribution gives you much more info than expectation. – gunes Feb 10 '20 at 15:42
• No, the integration give you a function of $f_*,X_*,X_y$, not a single number because it's a distribution. Think about joint PDF integration for simplicity:$$f(x)=\int f(x,y)dy$$ Integral result is a function of $x$. – gunes Feb 10 '20 at 15:48
• Perfect!! cant thank you enough!! I do have one tiny confusion on GP but I post it a another question, and hopefully can request for your contribution. – GENIVI-LEARNER Feb 10 '20 at 15:50