For multiple least squares linear regression, can we actually specify a ratio between the coefficients as the prior? For example, for following linear model: $y = b_1*x_1 + b_2*x_2 + b_3*x_3$ ,can we specify $b1:b2:b3 = 1:2:3$?


  1. We don't know what exactly $b_1, b_2, b_3$ are, we only know they are expected to be proportional to each other.
  2. They are not exactly proportional, which means we cannot reduce it to univariate regression problem: $y = b_1*(x_1+2*x_2+3*x_3)$. The ratio between the effects are prior expectations, we want data can help adjust specific values for the coefficients.

I variant of your question is answered here: Linear model with constraints on coefficients in terms of ratios, but your question have a Bayesian twist.

First you will need some prior on $b_1$ (maybe a weak prior), then you write $b_2 = \gamma_2 b_1, b_3=\gamma_3 b_1$ and now you need a prior on $\gamma_2$ with prior expectation 2 and (maybe a small) variance, likewise for $\gamma_3$, but now with prior expectation 3.

It shouldn't be difficult to program such a model in bugs or stan.

  • 1
    $\begingroup$ Thanks a lot! Kjetlil. So for the prior on $b_1$, I'm think about using one round of univariate regression first, like: $y = b_1(x_1+2*x_2+3*x_3)$ to calibrate $b_1$ as the prior you mentioned. And then use this prior on $b_1$ to do the following multiple regression using MCMC. Is this something sound reasonable? Also can you please point me some specific parts of bugs/stan that I should look at to implement? I'm more familiar with R. are there something related too? Thanks. $\endgroup$ – wuhabjmo Aug 28 '18 at 21:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.