I am trying to understand the confidence interval for linear regression parameters. At this link Derive Variance of regression coefficient in simple linear regression an answer is provided. However, I did not understand in the derivation fully. Do we consider the regression parameters as random variables?

That is what are the random variables in the following regression: $$ y=\beta_1 x+\beta_0 + \epsilon $$ From what I understand is that just the $\epsilon$ is random variable with mean 0 and variance $\sigma^2$. Considering $\beta_1, x, \beta_0$ as constants $y$ also becomes random variable with mean $\beta_1 x+\beta_0$ and variance $\sigma^2$, from which confidence interval can be calculated.

However,for regression coefficients confidence intervals are also calculated. Can you please clarify the random variables in regression please?



I am from a different domain, and use somewhat different language, but maybe this will help.

Imagine doing an experiment. $x$ is a set of given values, an "independent variable". Not random. For each of these values you measure a dependent variable, $y$. Presumably, $y$ depends on $x$ in a deterministic (non-random) way, but your measurements are also affected by some noise, $\epsilon$. Therefore, $y$ is a random variable.

When you calculate coefficients of linear regression, $\hat{\beta}_0$ and $\hat{\beta}_1$ (which are estimates of "true", non-random $\beta_0$ and $\beta_1$), their values depend on $y$, and therefore they are also random. If you repeat your experiment once again, for the same $x$ values, you will get (slightly, or not slightly) different $y$, and then will calculate different $\hat{\beta}_0$ and $\hat{\beta}_1$.

Finally, note that $\hat{\beta}_0$ and $\hat{\beta}_1$ are expressed in terms of $y$ linearly. Therefore, variances of $\hat{\beta}_0$ and $\hat{\beta}_1$ are proportional to $var(y) = var(\epsilon)$.

  • 1
    $\begingroup$ Itd be helpful to put some hats on your betas to emphasize that they are estimates. $\endgroup$ – Matthew Drury Apr 25 '17 at 0:46
  • 1
    $\begingroup$ @AlexFainshtein thanks for the answer, I am accepting your answer but would like to add my comment: although $\beta_1, \beta_0$ are not random variables (they are not random but unknown), the estimations of them, i.e. $\hat{\beta}_0,\hat{\beta}_1$are random variables. Thus confidence intervals for estimations can be calculated as done for a random variable. $\endgroup$ – tempx May 15 '17 at 14:13

Error is the only random variable. The X's are assumed to fixed but if you assume linearity you can generalize to other values of X. Of course extrapolation is very risky and rarely justified.

  • $\begingroup$ The parameter estimates are also random variables. $\endgroup$ – Matthew Drury Apr 25 '17 at 1:43
  • $\begingroup$ Agreed, I meant in the model. $\endgroup$ – David Lane Apr 25 '17 at 2:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.