Suppose we have three features $x_i \sim N(0, 1)$ for $i=1,2,3$. We then use Bayesian linear regression with interpolant $f(x, w) = wX$, such that we model y as $N(f(x, w), \beta)$, i.e., with a Gaussian likelihood. Then we set a zero mean isotropic gaussian prior on the parameters $w$ such that this prior is infinitely broad. We know that the posterior distribution in this case will be gaussian and so will the predictive distribution.
My question is, if we consider two models, one with x1 and x2, and another with x1, x2 and x3, and we generate the predictive distribution for a single test point for each, can we say anything about the covariance between these two predictive distribitions? What if x1, x2 and x3 are known to be indepednant?