1
$\begingroup$

So the predictive distribution of the Gaussian process is provided as follows where $p(\mathbf{f} \lvert \mathbf{X},\mathbf{y})$ and $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y})$ is the predictive distribution, taken from Martin Krasser blog

$$ p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y}) = \int{p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})p(\mathbf{f} \lvert \mathbf{X},\mathbf{y})}\ d\mathbf{f} \\ = \mathcal{N}(\mathbf{f}_* \lvert \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*)$$

Now what I am confused with is that isnt $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$ supposed to give us the predictive distribution $\bf{f_*}$ conditioned on the query point and posterior distribution $\bf{f}$? Here we already conditioned on posterior distribution $\bf{f}$ so why are we multiplying again with $p(\mathbf{f} \lvert \mathbf{X},\mathbf{y})$ and integrating?

What is "conceptual" difference between $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$ and $ p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y})$?

$\endgroup$

1 Answer 1

1
$\begingroup$

The actual relationship by law of total probability is as follows: $$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y}) = \int{p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f},\mathbf{X}, \mathbf{y})p(\mathbf{f} \lvert \mathbf{X},\mathbf{y},\mathbf{X_*})}\ d\mathbf{f}$$

It's just within this context, we have:

$$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f},\mathbf{X}, \mathbf{y})=p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$$ because when you know $\mathbf{f}$, you won't need $\mathbf{X},\mathbf{y}$. Similarly, the second term reduces to $p(\mathbf{f|\mathbf{X},\mathbf{y}})$ since $\mathbf{f}$ is fed by $\mathbf{X,y}$, and having only $\mathbf{X}_*$ is not of any use.

$\endgroup$
14
  • 1
    $\begingroup$ For your second comment: The relation between $\mathbf{f,X,y}$ is not deterministic, so $X,y$ are redundant because we only need $f$ to find $f_*$ from $X_*$. Otherwise $p(\mathbf{f|X,y})$ would be dirac delta. $\endgroup$
    – gunes
    Feb 10, 2020 at 15:23
  • 1
    $\begingroup$ your first comment makes so much sense, if the relation between $\mathbf{f,X,y}$ deterministic, we could simply have written $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$, since it is not deterministic we write $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f},\mathbf{X}, \mathbf{y})$ right? $\endgroup$ Feb 10, 2020 at 15:37
  • 1
    $\begingroup$ it is similar to that, we're considering all possible f, and find the distribution of $f_*$. A distribution gives you much more info than expectation. $\endgroup$
    – gunes
    Feb 10, 2020 at 15:42
  • 1
    $\begingroup$ No, the integration give you a function of $f_*,X_*,X_y$, not a single number because it's a distribution. Think about joint PDF integration for simplicity:$$f(x)=\int f(x,y)dy$$ Integral result is a function of $x$. $\endgroup$
    – gunes
    Feb 10, 2020 at 15:48
  • 1
    $\begingroup$ Perfect!! cant thank you enough!! I do have one tiny confusion on GP but I post it a another question, and hopefully can request for your contribution. $\endgroup$ Feb 10, 2020 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.