Please go through my draft summary below and let me know if my conventions are correct, comprehensible, and non ambiguous.

Simple Linear Regression Model

Let given observed sample set be $\{(x_1,y_1),(x_2,y_2), \cdots (x_N,y_N)\}$.$X,Y$ are random variables that can take on any value $(x_i,y_i)$ within range of sample set.

Population Regression Function, PRF

Given a population $(X,Y)$ we hypothesize underlying population has a regression line as follows. The conditional expectation is

$$\begin{aligned} & E(Y|x) = \beta_0 + \beta_1x & \text{PRF} \end{aligned}$$

The above equation is called Population Regression Function, PRF. Including the error $\varepsilon$, the prediction of dependent variable would be

$$\begin{aligned} & Y = E(Y|x) + \varepsilon & \text{Prediction} \end{aligned}$$

which is called simple linear regression model for population.

  • RVs: $X, Y|x, \varepsilon$

  • Parameters $\beta_0, \beta_1, \mu_X, \sigma_X^2, \ \ \mu_{Y|x}, \sigma_{Y|x}^2\ \ \sigma^2$

Sample Regression Function, SRF

Point Estimates from single SRF

Given a sample set $(X,Y)$, we estimate underlying population has a regression line as follows.

\begin{align} & \hat{Y} = \hat{\beta_0} + \hat{\beta_1}x & \text{SRF, Estimator of RV } E(Y|x), \text{ not } Y \\ & \hat{\varepsilon} = Y - \hat{Y} & \text{Estimator of RV } \varepsilon \end{align}

For given sample $(x_i, y_i)$ from sample set $(X,Y)$, a fitted value and residual are $$\begin{aligned} & \hat{y_i} = \hat{Y}(x_i)= b_0 + b_1x_i & \text{Fitted value, Estimate of RV } E(Y|x) \text{ at } x_i \\ & \hat{\varepsilon_i} = y_i - \hat{y_i} & \text{Residual, Estimate of RV } \varepsilon \text{ at } (x_i,y_i) \end{aligned}$$

Using OLS,

$$\begin{aligned} & \hat\beta_1 = \dfrac{\sum_{(x,y)}(y - \overline{Y})(x - \overline{X}) }{\sum_{x}(x - \overline{X})^2} & \text{Slope RV, Estimator of Parameter } \beta_1 \\ & \hat\beta_0 = \overline{Y} - \hat\beta_1\overline{X} & \text{y-intercept RV, Estimator of Parameter } \beta_0 \\ & b_1 = \dfrac{\sum_{i}(y_i - \overline{y})(x_i - \overline{x}) }{\sum_{i}(x_i - \overline{x})^2} & \text{Slope constant, Estimate of Parameter } \beta_1 \\ & b_0 = \overline{y} - b_1\overline{x} & \text{y-intercept constant, Estimate of Parameter } \beta_0 \end{aligned}$$

  • $\hat{\beta_0},\hat{\beta_1}$ are estimators of $\beta_0,\beta_1$ for any sample set. $b_0,b_1$ are estimates of $\beta_0,\beta_1$ for given sample set

  • Estimators: $\hat{\beta_0}, \hat{\beta_1}, \hat{X}, \hat{Y}, \hat{\varepsilon}$

  • Estimates: $b_0, b_1, \overline{x}, s_X^2, \overline{\hat{y}}=\overline{y}, s_{Y|x}^2=s^2$

The entire draft is available here


  1. Devore's book here
  2. Gujarati's Basic Econometrics here.

Update 1: As per whuber advice, I also have started separate distinct to the point question, with first one here. Looking forward for further feedback. (For eg, we could either quash this question and continue there, or bring one of them (or first), here modifying this question)

Update 2: Specific split up questions (including first one just for entirety).
1. Doubt about Predictor Variable
2. Is residual an estimate of error
3. Characterizing estimator,estimate separately
4. Why SSE has (n-2)df - not duplicate


closed as unclear what you're asking by kjetil b halvorsen, mdewey, DeltaIV, Michael Chernick, mkt Nov 26 '18 at 15:19

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ I think this isn't an on topic question for SE. It's a bit of an XY problem. Honestly, regression terminology is hard to grapple with, but your attempt to reinvent the wheel here is unfathomable for me. A lot of unnecessary terms, and misuse, and incorrect assumptions. Sxy is, for instance, not a constant because it has variance. $\endgroup$ – AdamO Nov 20 '18 at 17:55
  • $\begingroup$ The first section of this post is addressed at stats.stackexchange.com/questions/16381/…: looking at that might be a good start. $\endgroup$ – whuber Nov 20 '18 at 18:01
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ – whuber Nov 21 '18 at 14:09