Bayesian Weighted Linear regression - Cross Validated most recent 30 from stats.stackexchange.com 2019-08-19T01:47:48Z https://stats.stackexchange.com/feeds/question/133553 http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://stats.stackexchange.com/q/133553 3 Bayesian Weighted Linear regression Luca https://stats.stackexchange.com/users/36540 2015-01-15T13:46:18Z 2015-01-15T20:52:58Z <p>I am currently reading the following paper which formulates the weighted linear regression in a Bayesian setting. In classic weighted LS, we minimise the following:</p> <p>$$\sum_{i=1}^{N} w_i (\beta^Tx_i - y_i)$$</p> <p>In this paper, they try and have a Bayesian formulation of the WLS. So, it makes the following modelling choices about the probability distributions of the random variables:</p> <p>$$y_i \sim N(\beta^tx_i, \sigma^2/w_i)$$</p> <p>So, here we are modelling each of the $y_i$ to have variance which can be weighted by their individual weight. There is a normal prior also over the regression parameters $\beta$.</p> <p>$$\beta \sim N(\beta_0, \Sigma_{\beta, 0})$$</p> <p>There is a Gamma prior over the weights $w_i$.</p> <p>$$w_i \sim Gamma(a_i, b_i)$$</p> <p>Now, my question is that the regression problem is basically:</p> <p>$$y_i = \beta^T x_i + \epsilon_i$$</p> <p>My question is why is there no prior on $\epsilon$? In this paper, they estimate $\sigma^2$ through some standard regression formula (Apologies as I have not gone far to derive it yet). However, to me it seems that $\sigma^2$ is also an unknown parameter in the model and if we follow Bayesian statistical modelling, we should specify a prior for it.</p> <p>If anyone is curious, the paper is here:</p> <p><a href="http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=4AAE4C2C2577844312D5EBBB60303F64?doi=10.1.1.75.9906&amp;rep=rep1&amp;type=pdf" rel="nofollow">http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=4AAE4C2C2577844312D5EBBB60303F64?doi=10.1.1.75.9906&amp;rep=rep1&amp;type=pdf</a></p> https://stats.stackexchange.com/questions/133553/-/133614#133614 1 Answer by user154510 for Bayesian Weighted Linear regression user154510 https://stats.stackexchange.com/users/53475 2015-01-15T20:52:58Z 2015-01-15T20:52:58Z <p>This is just a model assumption the author made. Unfortunately, there aren't standardized procedures to "follow Bayesian statistical modelling", so while you may specify a prior for variance, it isn't a requirement for a linear regression to be bayesian.</p>