4
$\begingroup$

I'm trying to set up a bayesian regression of the form

$y_i \sim f(\beta_0 + \beta_1 x_i)$

but rather than $x_i$ fixed, they themselves are drawn from a distribution of (known) mean $x_i \sim g(\mu_i)$. $f$ and $g$ could be any distribution suitable for regression, but will probably be poisson or negative binomial.

Is this type of regression possible, or is the problem under-defined? I've tried a few MCMC runs but they typically fail to converge. I've so far only tried metropolis-hastings as the conditional distribution of $x_i$ given the data and $\beta$ doesn't seem to have an analytic solution for gibbs sampling.

Any tips / pointers to discussions appreciated.

Edit for clarity:

Both $y_i$ and a set $x_i^{(1)},...,x_i^{(n)}$ are observed. $\{x_i\}$ are then used to fit $\mu_i$ (which could be performed in the same MCMC run) and then random samples $z_i$ are drawn from $g(\mu_i)$ to construct the mean $\beta_0 + \beta_1 z_i$ for the inference of $\beta$.

This (slightly strange) modelling choice is chosen to get a full posterior on $\beta$ that takes into account the "true" $z$ generating the mean is hugely variable and that the $x$ observations are unreliable.

Improve this question
$\endgroup$
3
  • $\begingroup$ This is a regression with errors in variables or random/mixed effects. The MCMC issue should be clarified and may asked as a separate question. Do you observe the $x_i$'s? $\endgroup$
    – Xi'an
    Feb 16, 2015 at 18:00
  • $\begingroup$ Thanks for your reply. For each $y_i$ I have a (very noisy) set $\{x_i\}$ measured, so my idea was to fit a model to $\{x\}$ (ie find $\mu$) then construct the problem above (perhaps bad to use $x$ again, maybe $z$ since not actually observed). Of course eventually I'd like to do inference of $\mu$ in the same MCMC run as $\beta$, but trying to take small steps just now. $\endgroup$
    – kezz_smc
    Feb 17, 2015 at 10:43
  • 1
    $\begingroup$ could you provide more details on the model you tried? $\endgroup$
    – Xi'an
    Feb 18, 2015 at 7:32

1 Answer 1

Reset to default
5
$\begingroup$

As stated, if the $x_i$ are observed, the fact that they are random does not modify the conditional model $$y_i \sim f(y;\beta_0 + \beta_1 x_i)$$ and hence the posterior on $\mathbf\beta$ does not incorporate this item of information.

If you had an error in variable model, \begin{align*}y_i &\sim f(y;\beta_0 + \beta_1\mu_i)\\x_i&\sim g(x;\mu_i)\end{align*} then both $\mathbf\beta$ and $\mu$ would be treated simultaneously. Provided you can construct a prior on the $\mu_i$'s that creates interaction.

Improve this answer
$\endgroup$
5
  • $\begingroup$ "treated simultaneously" is pretty unclear. I feel I understand what you are saying, but I am not sure if my understanding is correct, so I imagine others could feel the same... $\endgroup$
    – Tim ♦
    Feb 18, 2015 at 7:53
  • $\begingroup$ "Treated simultaneously" is just simultaneously inferred from the same MCMC run? The latter error in variable model you propose is most similar to what I have, except the mean of $y_i$ is $\beta_0 + \beta_1 z_i$ where $z_i$ is drawn from $g(x;\mu_i)$ but only the observed $x_i$ are used to fit $\mu_i$ (apologies for change of notation, I should have made it clearer in the original question). $\endgroup$
    – kezz_smc
    Feb 18, 2015 at 10:50
  • $\begingroup$ I've added in an edit to the original question for clarity. Thanks for your help so far $\endgroup$
    – kezz_smc
    Feb 18, 2015 at 10:59
  • $\begingroup$ treated simultaneously = handled together = analysed jointly = modelled as a pair = ... $\endgroup$
    – Xi'an
    Feb 18, 2015 at 18:49
  • 1
    $\begingroup$ (+1) Welcome to 10.000! $\endgroup$
    – Zen
    Feb 19, 2015 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.