# The variance in the linear regression model

I have asked a similar question in what is the likelihood function $p(y|a,\tau)$ of simple linear regression model?, that is,

For a simple linear regression model without intercept, that is $$y_i=ax_i+\varepsilon_i$$ where $$\varepsilon_i\sim_{iid} N(0, \tau^2), i=1,2,\dots, n$$ and $$x_i$$ is a fixed covariate.

If I change $$y_i=ax_i+\varepsilon_i$$ and $$a|\tau \sim N(\mu, \tau^2)$$, is the $$y_i\sim N(ax_i, \tau^2)???$$

• this is equivalent to including a intercept $\mu$ – PedroSebe Oct 13 at 0:38
• @PedroSebe So is the distribution of $y_i$ with $N(ax_i,\tau^2)$? – user261225 Oct 13 at 3:45
• Sorry, I completely misread your question! Please, disregard what I said about intercepts. Yes, you do have $y_i|a,\tau\sim N(ax_i,\tau^2)$. – PedroSebe Oct 13 at 4:38
• @PedroSebe Sorry, how to get the likelihood $y_i|a, \tau\sim N(ax_i, \tau^2)$? It seems that $p(y_i|\beta, \tau)=p(y_i, \beta, \tau)/p(\beta,\tau)$? – user261225 Oct 13 at 22:03

So you are assuming that the coefficient in your model is also random. You can still think of $$y_i$$ as sum of two normally distributed random variables so it will also be normal. However, the parameters will depend on whether $$a$$ and $$\epsilon_i$$ are independent or not.

$$E(Y|X, \tau)=xE(a)=\mu x$$

And $$Var(Y|X, \tau) = x^2\tau^2 + \tau^2 + 2xCov(a,\epsilon)$$

Note that here I have assumed that covariance of $$a$$ and $$\epsilon_i$$ is independent of $$i$$.

Accordingly, $$Y|X \sim N(\mu x, ((x^2+1)\tau^2+2xCov(a,\epsilon)))$$

• Thanks. But if we assume that $\tau$ and $a$ are two parameters in statistics, $a|\tau\sim N(\mu, \tau^2)$ and $\tau\sim Gamma(\alpha, \beta)$. Can we just think $y_i\sim N(ax_i, \tau^2)$? Because if we think $\beta$ is a random variable, $\tau$ is also a random variable. – user261225 Oct 13 at 21:54
• Okay. Great question. No. In that case we only say $Y_i|X,\tau, a \sim N(a x_i, ...)$. Unless $a$ is realized and given, we cannot say $Y \sim N(ax_i,...)$. Just like while describing the distribution of $a$ you have assumed $\tau$ to be given, the same has to be done with $Y$ and $a$ as well if you want to make $a$ a parameter of the distribution. The distribution of $Y|X$ will be a whole more complicated now because of $\tau$. – Dayne Oct 14 at 2:16
• So $y_i|\tau, a \sim N(ax_i, \tau^2)$, right? – user261225 Oct 14 at 2:35
• yes. Also given $x$ besides $\tau, a$. A small notation issue. $y_i$ is generally used for a realized observation. $Y$ is the random variable if we are considering all $y_i$ to be observations coming from a given distribution. – Dayne Oct 14 at 4:21