4
$\begingroup$

I am currently reading the following paper which formulates the weighted linear regression in a Bayesian setting. In classic weighted LS, we minimise the following:

$$ \sum_{i=1}^{N} w_i (\beta^Tx_i - y_i) $$

In this paper, they try and have a Bayesian formulation of the WLS. So, it makes the following modelling choices about the probability distributions of the random variables:

$$ y_i \sim N(\beta^tx_i, \sigma^2/w_i) $$

So, here we are modelling each of the $y_i$ to have variance which can be weighted by their individual weight. There is a normal prior also over the regression parameters $\beta$.

$$ \beta \sim N(\beta_0, \Sigma_{\beta, 0}) $$

There is a Gamma prior over the weights $w_i$.

$$ w_i \sim Gamma(a_i, b_i) $$

Now, my question is that the regression problem is basically:

$$ y_i = \beta^T x_i + \epsilon_i $$

My question is why is there no prior on $\epsilon$? In this paper, they estimate $\sigma^2$ through some standard regression formula (Apologies as I have not gone far to derive it yet). However, to me it seems that $\sigma^2$ is also an unknown parameter in the model and if we follow Bayesian statistical modelling, we should specify a prior for it.

If anyone is curious, the paper is here:

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=4AAE4C2C2577844312D5EBBB60303F64?doi=10.1.1.75.9906&rep=rep1&type=pdf

Improve this question
$\endgroup$
6
  • $\begingroup$ You cannot expect readers to go through the paper to answer and even understand your question so please produce some context for your question to make sense on its own. $\endgroup$
    – Xi'an
    Jan 15, 2015 at 18:12
  • 2
    $\begingroup$ This question appears to be off-topic because it requires the reader to go through an external link to be understood. $\endgroup$
    – Xi'an
    Jan 15, 2015 at 18:13
  • $\begingroup$ Ok, I will try and edit the question later in the night today. Sorry for it. $\endgroup$
    – Luca
    Jan 15, 2015 at 18:20
  • $\begingroup$ Between the start and end of your question you changed the formulation of the model: you converted "$y_i \sim N(\beta^tx_i, \sigma^2/w_i)$" into "$y_i = \beta^T x_i + \epsilon_i$". That changed the model. (The role played by $\epsilon_i$ in the latter is replaced by the distributional assumptions on $y_i$ in the former.) So basically it looks like you're telling us "the paper assumes this, but I assume something else, so why doesn't the paper deal with my assumption?" $\endgroup$
    – whuber ♦
    Jan 15, 2015 at 19:03
  • $\begingroup$ hmmmm... ok, I will need to go away and have a think. I will update the thread as soon as I get the confusions clear in my head. $\endgroup$
    – Luca
    Jan 15, 2015 at 19:06

1 Answer 1

Reset to default
1
$\begingroup$

This is just a model assumption the author made. Unfortunately, there aren't standardized procedures to "follow Bayesian statistical modelling", so while you may specify a prior for variance, it isn't a requirement for a linear regression to be bayesian.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the answer. So, I guess the authors choose to only model a few of the unknown variables in the Bayesian setting. However, I guess nothing stops me from also have a prior over the noise precision as well. $\endgroup$
    – Luca
    Jan 15, 2015 at 21:48
  • $\begingroup$ I found the answer in Gelman's book Bayesian Data Analysis (Page 384). He argues that priors for variances are less important for most applications but mentions that a scaled-inverse chi square dist. is a conjugate prior in this setting. $\endgroup$
    – Luca
    Jan 19, 2015 at 12:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.