# How to estimate the $\mu$ in the Bayesian regression model?

For a simple linear regression model without intercept, that is $$y_i=ax_i+\varepsilon_i$$ where $$\varepsilon_i\sim_{iid} N(0, \tau^2), i=1,2,\dots, n$$ and $$x_i$$ is a fixed covariate. Assume that $$a|\tau \sim N(\mu, \tau^2)$$. and $$\tau$$ is fixed.

By the least squares estimator of $$a$$, we know that the minimum of $$\sum_{i=1}^n(y_i-\hat{a}x_i)^2$$ obtained at the estimator of $$a$$ $$\hat{a}=\frac{\sum_{i}x_iy_i}{\sum_i x_i^2}$$

But I do not know how to use this estimator to estimate the $$\mu$$?

• What does "$\gamma$" refer to??
– whuber
Commented Oct 21, 2020 at 13:12
• @whuber Sorry, this is $\tau$.
– user261225
Commented Oct 22, 2020 at 0:11

It looks to me like $$\mathcal{N}(\mu,\tau^2)$$ is the prior distribution of $$a$$. Thus, you don't estimate $$\mu$$, it is a parameter that describes your initial belief of $$a$$ -- often this is set to $$0$$ to encourage low weights unless you have reason to believe another mean is better.
So after you see more data, you compute the posterior mean of $$a$$ using bayes rule. Then you can either set $$a$$ as the argmax of the posterior and perform linear regression (MAP estimate), or you can integrate out $$\mu$$ to derive a predictive distribution of $$y$$, again using bayes rule.
• I don't understand the reason for setting $\mu=0,$ because it looks like all numbers are equally valid for the prior. After all, for any real number $\lambda$ this model is identical to the model $$z_i = (a-\lambda)x_i + \varepsilon_i$$ where $z_i = y_i - \lambda x_i.$ If your reasoning is correct, than in the re-expressed model we would stipulate $a-\lambda=0,$ which is the same as $a=\lambda.$ How do you resolve this apparent contradiction?