Bayesian replication, but with new variables

Question

Suppose I have data I've collected containing predictor variables $X_1, X_2$, and $X_3$. I build a main effects statistical model predicting $Y$ from these predictors and estimate the relevant coefficients ($b_0, b_1, b_2, b_3$).

Now let's suppose I want to collect more data and build the same statistical model, with the addition of two new variables (e.g., $X_4$ and $X_5$). Easy. But, let's suppose I want to do a Bayesian analysis this time, using my estimates from the original model as priors. That sounds reasonable, and mayhap I could put informative priors on $b_0-b_3$, then use uninformative priors on $b_4/b_5$.

But we have a problem. The estimates of $b_0, \ldots, b_3$ were built from a model that did not have $X_4$ and $X_5$. Since I expect these new variables will be at least somewhat correlated with $X_1, \ldots, X_3$, there's no reason to suspect my original estimates will be the same.

But, I really want to use Bayesian methods with informative priors. It seems a waste to not use the prior information I've gleaned. So, here's my question: can I use informative priors from a past model on a new model, when the new model contains new predictors? If so, how?

My initial thoughts: I know the new estimates of $b_0, \ldots, b_3$ will be a function of the old values and the covariance between the old and new predictors. I'm thinking I could just do a somewhat informative prior on the covariances between $(X_1, \ldots, X_3) \& (X_4, X_5)$ and use the old values of $b_0, \ldots, b_3$ with the appropriate covariances to get a more informative prior. This might make sense and work well, assuming there are no interactions between the new/old predictors. But, there may be interactions. If so, it seems to me (initially) there's not much I can do to make those priors informative.

Answer 1

Suppose I have data I've collected containing predictor variables 𝑋1,𝑋2, and 𝑋3. I build a main effects statistical model predicting 𝑌 from these predictors and estimate the relevant coefficients (𝑏0,𝑏1,𝑏2,𝑏3).

Sure let's suppose that

$$Y = \beta_0 + \sum_{j=1}^3 \beta_j X_j + \epsilon$$

and you have obtained estimates $\hat \beta_0, \ldots, \hat \beta_3$.

Now let's suppose I want to collect more data and build the same statistical model, with the addition of two new variables (e.g., 𝑋4 and 𝑋5). Easy. But, let's suppose I want to do a Bayesian analysis this time, using my estimates from the original model as priors. That sounds reasonable, and mayhap I could put informative priors on 𝑏0−𝑏3, then use uninformative priors on 𝑏4/𝑏5.

Okay, so the second model we'll take to be

$$Y = \theta_0 + \sum_{j=1}^5 \theta_j X_j + \eta$$

to be on the same variables. It is valid to use $\hat \beta_0, \ldots, \hat \beta_3$ in constructing priors for $\theta_0, \ldots, \theta_3$. You could choose weakly informative priors, there are no truly uninformative priors, for $\theta_4$ and $\theta_5$. You could also choose to make the priors for $\theta_4$ and $\theta_5$ a bit stronger by using more background knowledge if it is available.

But we have a problem. The estimates of 𝑏0,…,𝑏3 were built from a model that did not have 𝑋4 and 𝑋5. Since I expect these new variables will be at least somewhat correlated with 𝑋1,…,𝑋3, there's no reason to suspect my original estimates will be the same.

To me this isn't an intractable problem, but rather expresses more background information that can be put into the priors. You don't need to make your priors for $\theta_0, \ldots, \theta_3$ exactly match the respective means and standard errors of $\hat \beta_0, \ldots, \hat \beta_3$. You might keep the means quite similar while making the variances of the priors a bit larger. If you think there is some statistical dependence to be accounted for between the new parameters go ahead with trying a model that incorporates that.

But, I really want to use Bayesian methods with informative priors. It seems a waste to not use the prior information I've gleaned. So, here's my question: can I use informative priors from a past model on a new model, when the new model contains new predictors? If so, how?

I think what you've already suggested out above is a pretty good start. My main two suggestions are:

1. don't feel you need to exactly copy the sampling distributions of the previous parameters over as being the priors in the second model.
2. explore and validate models using prior predictive simulations.

My initial thoughts: I know the new estimates of 𝑏0,…,𝑏3 will be a function of the old values and the covariance between the old and new predictors.

They'll be related at least.

I'm thinking I could just do a somewhat informative prior on the covariances between (𝑋1,…,𝑋3)&(𝑋4,𝑋5) and use the old values of 𝑏0,…,𝑏3 with the appropriate covariances to get a more informative prior. This might make sense and work well, assuming there are no interactions between the new/old predictors. But, there may be interactions. If so, it seems to me (initially) there's not much I can do to make those priors informative.

Modelling with covariances and/or without interactions sounds fine. It is okay to explore multiple models with differing assumptions and compare. If you have not already, I strongly recommend you read Gelman et al. 2020. I think they provide a lot of useful advice on developing Bayesian models, and you might find their discussion reduces some of your concerns about choosing appropriate priors.