# Bayesian regression with independent variable drawn from distribution

I'm trying to set up a bayesian regression of the form

$y_i \sim f(\beta_0 + \beta_1 x_i)$

but rather than $x_i$ fixed, they themselves are drawn from a distribution of (known) mean $x_i \sim g(\mu_i)$. $f$ and $g$ could be any distribution suitable for regression, but will probably be poisson or negative binomial.

Is this type of regression possible, or is the problem under-defined? I've tried a few MCMC runs but they typically fail to converge. I've so far only tried metropolis-hastings as the conditional distribution of $x_i$ given the data and $\beta$ doesn't seem to have an analytic solution for gibbs sampling.

Any tips / pointers to discussions appreciated.

Edit for clarity:

Both $y_i$ and a set $x_i^{(1)},...,x_i^{(n)}$ are observed. $\{x_i\}$ are then used to fit $\mu_i$ (which could be performed in the same MCMC run) and then random samples $z_i$ are drawn from $g(\mu_i)$ to construct the mean $\beta_0 + \beta_1 z_i$ for the inference of $\beta$.

This (slightly strange) modelling choice is chosen to get a full posterior on $\beta$ that takes into account the "true" $z$ generating the mean is hugely variable and that the $x$ observations are unreliable.

• This is a regression with errors in variables or random/mixed effects. The MCMC issue should be clarified and may asked as a separate question. Do you observe the $x_i$'s? Feb 16, 2015 at 18:00
• Thanks for your reply. For each $y_i$ I have a (very noisy) set $\{x_i\}$ measured, so my idea was to fit a model to $\{x\}$ (ie find $\mu$) then construct the problem above (perhaps bad to use $x$ again, maybe $z$ since not actually observed). Of course eventually I'd like to do inference of $\mu$ in the same MCMC run as $\beta$, but trying to take small steps just now. Feb 17, 2015 at 10:43
• could you provide more details on the model you tried? Feb 18, 2015 at 7:32

As stated, if the $x_i$ are observed, the fact that they are random does not modify the conditional model $$y_i \sim f(y;\beta_0 + \beta_1 x_i)$$ and hence the posterior on $\mathbf\beta$ does not incorporate this item of information.

If you had an error in variable model, \begin{align*}y_i &\sim f(y;\beta_0 + \beta_1\mu_i)\\x_i&\sim g(x;\mu_i)\end{align*} then both $\mathbf\beta$ and $\mu$ would be treated simultaneously. Provided you can construct a prior on the $\mu_i$'s that creates interaction.

• "treated simultaneously" is pretty unclear. I feel I understand what you are saying, but I am not sure if my understanding is correct, so I imagine others could feel the same...
– Tim
Feb 18, 2015 at 7:53
• "Treated simultaneously" is just simultaneously inferred from the same MCMC run? The latter error in variable model you propose is most similar to what I have, except the mean of $y_i$ is $\beta_0 + \beta_1 z_i$ where $z_i$ is drawn from $g(x;\mu_i)$ but only the observed $x_i$ are used to fit $\mu_i$ (apologies for change of notation, I should have made it clearer in the original question). Feb 18, 2015 at 10:50
• I've added in an edit to the original question for clarity. Thanks for your help so far Feb 18, 2015 at 10:59
• treated simultaneously = handled together = analysed jointly = modelled as a pair = ... Feb 18, 2015 at 18:49
• (+1) Welcome to 10.000!
– Zen
Feb 19, 2015 at 19:29