I am reading a book about linear regression in which there is a plot showing the regression line and two curves they call a confidence band (also can be seen here). As I learn something last week that a set of samples may result in a statistical quantity called the confidence level which used to describe the "precision" of estimation of the population. However, here shows two curves of confidence level at each variable X of the fitting line. It is quite confusing why there will be a curve instead of a single number. My best guess is at each X, there could be tons of measurement so many $\hat{Y}$'s returned, and there will be confidence level for those $\hat{Y}$'s. If this is correct, why there are local minima of confidence curves. The local minima indicate there is some point at which you do the measurement, results will be more "precise"? I don't see how this happen.

My second question is how to compute the confidence curves. Though many software like R and Mplus work out the problem easily, I want to know how to compute that by hand calculation. I know the thread states the algebra and I borrow the math here

$$ s_{\hat{Y}_{X}} = s_{Y|X}\sqrt{\frac{1}{n}+\frac{\left(X-\bar{X}\right)^{2}}{\sum_{i=1}^{n}{\left(X_{i}-\bar{X}\right)^{2}}}} $$

$$ s_{Y|X} = \sqrt{\frac{\sum_{i=1}^{n}{\left(Y_{i}-\hat{Y}\right)^{2}}}{n-2}} $$

$$ \hat{Y} \pm t_{\nu=n-2, \alpha/2}s_{\hat{Y}} $$

I am trying to understand those expressions and notation. My understanding is $X$ is the independent variable we choose for the fitting line and $X_i$ is the independent variable from data. $\hat{Y}$ is the measurement on the fitted line when $X$ is given while $Y_i$ is the measurement from data. I don't understand what $Y$ stands for. From the math, it seems that we compute something called $S_{\hat{Y}_X}$ from data and fitted values then use the very last formula to compute the curve. In the last expression, the $t$ is the student distribution and $\alpha$ related to the confidence level we choose and I assume there is a typo that $S_\hat{Y}$ should be $S_{\hat{Y}_X}$?

Very last question. I don't remember which book I read before but there is an example to introduce a cross term in the regression model something like

$$ Y = a_0 + a_1X + a_2TX $$

Where $Y$, $X$ and $T$ are given as data and by regression, $a_0$, $a_1$ and $a_2$ are computed. But instead of plotting Y again X, the author concerns the plot $Z=a_2T$ against $T$ only. In this case, if we want to plot the confidence band as well, should I just repeat the same calculation but replace $\hat{Y}$ with $\hat{Z}$, $X$ with $T$ and $\overline{X}$ with $\overline{T}$?

  • 1
    $\begingroup$ It's best on this site to ask individual questions separately. Your first 2 questions are closely enough related to keep together, but the third is a completely different question that you should post separately. When you do so, note that the example for the "cross term" doesn't quite make sense, as there is no way to evaluate $a_2$ and $a_3$ separately as they appear together as a sum multiplying $TX$. Perhaps $a_2$ appeared instead in a separate term $a_2T$ and the last term should be $a_3TX$? $\endgroup$ – EdM Jan 15 '20 at 17:10
  • $\begingroup$ My bad, I correct the math it should be $a_2+a_3T$ instead $\endgroup$ – user1285419 Jan 15 '20 at 17:28
  • $\begingroup$ Now, however, you can't distinguish $a_1$ from $a_2$; if you multiply the new formula out and collect terms in $X$ you get a term $(a_1+a_2)X$. $\endgroup$ – EdM Jan 15 '20 at 17:31
  • $\begingroup$ You are right. I refine it again. $\endgroup$ – user1285419 Jan 15 '20 at 17:34
  • $\begingroup$ Please look at this question and its answers, which deals with much of what you were asking. If you still have open questions after that, please post a new question or questions as the current question has been closed due to its high similarity to the one that's linked. $\endgroup$ – EdM Jan 15 '20 at 17:58