2
$\begingroup$

I am currently reading the following paper which formulates the weighted linear regression in a Bayesian setting. In classic weighted LS, we minimise the following:

$$ \sum_{i=1}^{N} w_i (\beta^Tx_i - y_i) $$

In this paper, they try and have a Bayesian formulation of the WLS. So, it makes the following modelling choices about the probability distributions of the random variables:

$$ y_i \sim N(\beta^tx_i, \sigma^2/w_i) $$

So, here we are modelling each of the $y_i$ to have variance which can be weighted by their individual weight. There is a normal prior also over the regression parameters $\beta$.

$$ \beta \sim N(\beta_0, \Sigma_{\beta, 0}) $$

There is a Gamma prior over the weights $w_i$.

$$ w_i \sim Gamma(a_i, b_i) $$

Now, my question is that the regression problem is basically:

$$ y_i = \beta^T x_i + \epsilon_i $$

My question is why is there no prior on $\epsilon$? In this paper, they estimate $\sigma^2$ through some standard regression formula (Apologies as I have not gone far to derive it yet). However, to me it seems that $\sigma^2$ is also an unknown parameter in the model and if we follow Bayesian statistical modelling, we should specify a prior for it.

If anyone is curious, the paper is here:

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=4AAE4C2C2577844312D5EBBB60303F64?doi=10.1.1.75.9906&rep=rep1&type=pdf

$\endgroup$
  • $\begingroup$ You cannot expect readers to go through the paper to answer and even understand your question so please produce some context for your question to make sense on its own. $\endgroup$ – Xi'an Jan 15 '15 at 18:12
  • 2
    $\begingroup$ This question appears to be off-topic because it requires the reader to go through an external link to be understood. $\endgroup$ – Xi'an Jan 15 '15 at 18:13
  • $\begingroup$ Ok, I will try and edit the question later in the night today. Sorry for it. $\endgroup$ – Luca Jan 15 '15 at 18:20
  • $\begingroup$ Between the start and end of your question you changed the formulation of the model: you converted "$y_i \sim N(\beta^tx_i, \sigma^2/w_i)$" into "$y_i = \beta^T x_i + \epsilon_i$". That changed the model. (The role played by $\epsilon_i$ in the latter is replaced by the distributional assumptions on $y_i$ in the former.) So basically it looks like you're telling us "the paper assumes this, but I assume something else, so why doesn't the paper deal with my assumption?" $\endgroup$ – whuber Jan 15 '15 at 19:03
  • $\begingroup$ hmmmm... ok, I will need to go away and have a think. I will update the thread as soon as I get the confusions clear in my head. $\endgroup$ – Luca Jan 15 '15 at 19:06
1
$\begingroup$

This is just a model assumption the author made. Unfortunately, there aren't standardized procedures to "follow Bayesian statistical modelling", so while you may specify a prior for variance, it isn't a requirement for a linear regression to be bayesian.

$\endgroup$
  • $\begingroup$ Thanks for the answer. So, I guess the authors choose to only model a few of the unknown variables in the Bayesian setting. However, I guess nothing stops me from also have a prior over the noise precision as well. $\endgroup$ – Luca Jan 15 '15 at 21:48
  • $\begingroup$ I found the answer in Gelman's book Bayesian Data Analysis (Page 384). He argues that priors for variances are less important for most applications but mentions that a scaled-inverse chi square dist. is a conjugate prior in this setting. $\endgroup$ – Luca Jan 19 '15 at 12:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.