I have asked a similar question in what is the likelihood function $p(y|a,\tau)$ of simple linear regression model?, that is,

For a simple linear regression model without intercept, that is $$y_i=ax_i+\varepsilon_i$$ where $\varepsilon_i\sim_{iid} N(0, \tau^2), i=1,2,\dots, n$ and $x_i$ is a fixed covariate.

If I change $y_i=ax_i+\varepsilon_i$ and $a|\tau \sim N(\mu, \tau^2)$, is the $$y_i\sim N(ax_i, \tau^2)???$$

  • $\begingroup$ this is equivalent to including a intercept $\mu$ $\endgroup$ – PedroSebe Oct 13 at 0:38
  • $\begingroup$ @PedroSebe So is the distribution of $y_i$ with $N(ax_i,\tau^2)$? $\endgroup$ – user261225 Oct 13 at 3:45
  • $\begingroup$ Sorry, I completely misread your question! Please, disregard what I said about intercepts. Yes, you do have $y_i|a,\tau\sim N(ax_i,\tau^2)$. $\endgroup$ – PedroSebe Oct 13 at 4:38
  • $\begingroup$ @PedroSebe Sorry, how to get the likelihood $y_i|a, \tau\sim N(ax_i, \tau^2)$? It seems that $p(y_i|\beta, \tau)=p(y_i, \beta, \tau)/p(\beta,\tau)$? $\endgroup$ – user261225 Oct 13 at 22:03

So you are assuming that the coefficient in your model is also random. You can still think of $y_i$ as sum of two normally distributed random variables so it will also be normal. However, the parameters will depend on whether $a$ and $\epsilon_i$ are independent or not.

$$E(Y|X, \tau)=xE(a)=\mu x$$

And $$Var(Y|X, \tau) = x^2\tau^2 + \tau^2 + 2xCov(a,\epsilon)$$

Note that here I have assumed that covariance of $a$ and $\epsilon_i$ is independent of $i$.

Accordingly, $$Y|X \sim N(\mu x, ((x^2+1)\tau^2+2xCov(a,\epsilon)))$$

| cite | improve this answer | |
  • 1
    $\begingroup$ Thanks. But if we assume that $\tau$ and $a$ are two parameters in statistics, $a|\tau\sim N(\mu, \tau^2)$ and $\tau\sim Gamma(\alpha, \beta)$. Can we just think $y_i\sim N(ax_i, \tau^2)$? Because if we think $\beta$ is a random variable, $\tau$ is also a random variable. $\endgroup$ – user261225 Oct 13 at 21:54
  • $\begingroup$ Okay. Great question. No. In that case we only say $Y_i|X,\tau, a \sim N(a x_i, ...)$. Unless $a$ is realized and given, we cannot say $Y \sim N(ax_i,...)$. Just like while describing the distribution of $a$ you have assumed $\tau$ to be given, the same has to be done with $Y$ and $a$ as well if you want to make $a$ a parameter of the distribution. The distribution of $Y|X$ will be a whole more complicated now because of $\tau$. $\endgroup$ – Dayne Oct 14 at 2:16
  • $\begingroup$ So $y_i|\tau, a \sim N(ax_i, \tau^2)$, right? $\endgroup$ – user261225 Oct 14 at 2:35
  • $\begingroup$ yes. Also given $x$ besides $\tau, a$. A small notation issue. $y_i$ is generally used for a realized observation. $Y$ is the random variable if we are considering all $y_i$ to be observations coming from a given distribution. $\endgroup$ – Dayne Oct 14 at 4:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy