1
$\begingroup$

So the predictive distribution of the Gaussian process is provided as follows where $p(\mathbf{f} \lvert \mathbf{X},\mathbf{y})$ and $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y})$ is the predictive distribution, taken from Martin Krasser blog

$$ p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y}) = \int{p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})p(\mathbf{f} \lvert \mathbf{X},\mathbf{y})}\ d\mathbf{f} \\ = \mathcal{N}(\mathbf{f}_* \lvert \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*)$$

Now what I am confused with is that isnt $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$ supposed to give us the predictive distribution $\bf{f_*}$ conditioned on the query point and posterior distribution $\bf{f}$? Here we already conditioned on posterior distribution $\bf{f}$ so why are we multiplying again with $p(\mathbf{f} \lvert \mathbf{X},\mathbf{y})$ and integrating?

What is "conceptual" difference between $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$ and $ p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y})$?

$\endgroup$
1
$\begingroup$

The actual relationship by law of total probability is as follows: $$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{X},\mathbf{y}) = \int{p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f},\mathbf{X}, \mathbf{y})p(\mathbf{f} \lvert \mathbf{X},\mathbf{y},\mathbf{X_*})}\ d\mathbf{f}$$

It's just within this context, we have:

$$p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f},\mathbf{X}, \mathbf{y})=p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$$ because when you know $\mathbf{f}$, you won't need $\mathbf{X},\mathbf{y}$. Similarly, the second term reduces to $p(\mathbf{f|\mathbf{X},\mathbf{y}})$ since $\mathbf{f}$ is fed by $\mathbf{X,y}$, and having only $\mathbf{X}_*$ is not of any use.

$\endgroup$
14
  • 1
    $\begingroup$ For your second comment: The relation between $\mathbf{f,X,y}$ is not deterministic, so $X,y$ are redundant because we only need $f$ to find $f_*$ from $X_*$. Otherwise $p(\mathbf{f|X,y})$ would be dirac delta. $\endgroup$ – gunes Feb 10 '20 at 15:23
  • 1
    $\begingroup$ your first comment makes so much sense, if the relation between $\mathbf{f,X,y}$ deterministic, we could simply have written $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f})$, since it is not deterministic we write $p(\mathbf{f}_* \lvert \mathbf{X}_*,\mathbf{f},\mathbf{X}, \mathbf{y})$ right? $\endgroup$ – GENIVI-LEARNER Feb 10 '20 at 15:37
  • 1
    $\begingroup$ it is similar to that, we're considering all possible f, and find the distribution of $f_*$. A distribution gives you much more info than expectation. $\endgroup$ – gunes Feb 10 '20 at 15:42
  • 1
    $\begingroup$ No, the integration give you a function of $f_*,X_*,X_y$, not a single number because it's a distribution. Think about joint PDF integration for simplicity:$$f(x)=\int f(x,y)dy$$ Integral result is a function of $x$. $\endgroup$ – gunes Feb 10 '20 at 15:48
  • 1
    $\begingroup$ Perfect!! cant thank you enough!! I do have one tiny confusion on GP but I post it a another question, and hopefully can request for your contribution. $\endgroup$ – GENIVI-LEARNER Feb 10 '20 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.