0
$\begingroup$

Suppose we have three features $x_i \sim N(0, 1)$ for $i=1,2,3$. We then use Bayesian linear regression with interpolant $f(x, w) = wX$, such that we model y as $N(f(x, w), \beta)$, i.e., with a Gaussian likelihood. Then we set a zero mean isotropic gaussian prior on the parameters $w$ such that this prior is infinitely broad. We know that the posterior distribution in this case will be gaussian and so will the predictive distribution.

My question is, if we consider two models, one with x1 and x2, and another with x1, x2 and x3, and we generate the predictive distribution for a single test point for each, can we say anything about the covariance between these two predictive distribitions? What if x1, x2 and x3 are known to be indepednant?

Improve this question
$\endgroup$
6
  • $\begingroup$ Covariance is defined between two random variables (with a joint probability distribution), not between two distributions $\endgroup$
    – J. Delaney
    Mar 12 at 12:21
  • $\begingroup$ Yes but can we say anything about the joint distribution then? @J.Delaney $\endgroup$
    – Qazaz
    Mar 12 at 13:14
  • $\begingroup$ i.e., can we say anything about the joint predictive distribution $\endgroup$
    – Qazaz
    Mar 12 at 13:22
  • $\begingroup$ Joint distribution of what ? you can independently sample $P(y|x_1,x_2)$ and $P(y|x_1,x_2,x_3)$, but the covariance of these sample will be trivially zero as they are independent $\endgroup$
    – J. Delaney
    Mar 12 at 14:23
  • $\begingroup$ But they arent independant, since if for example x3 has no informative information in the regression problem, i.e., w3=0., then these distributions will be perfectly correlated, right? @J.Delaney $\endgroup$
    – Qazaz
    Mar 13 at 5:22

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy