Please go through my draft summary below and let me know if my conventions are correct, comprehensible, and non ambiguous.

Simple Linear Regression Model

Let given observed sample set be $\{(x_1,y_1),(x_2,y_2), \cdots (x_N,y_N)\}$.$X,Y$ are random variables that can take on any value $(x_i,y_i)$ within range of sample set.

Population Regression Function, PRF

Given a population $(X,Y)$ we hypothesize underlying population has a regression line as follows. The conditional expectation is

$$\begin{aligned} & E(Y|x) = \beta_0 + \beta_1x & \text{PRF} \end{aligned}$$

The above equation is called Population Regression Function, PRF. Including the error $\varepsilon$, the prediction of dependent variable would be

$$\begin{aligned} & Y = E(Y|x) + \varepsilon & \text{Prediction} \end{aligned}$$

which is called simple linear regression model for population.

  • RVs: $X, Y|x, \varepsilon$

  • Parameters $\beta_0, \beta_1, \mu_X, \sigma_X^2, \ \ \mu_{Y|x}, \sigma_{Y|x}^2\ \ \sigma^2$

Sample Regression Function, SRF

Point Estimates from single SRF

Given a sample set $(X,Y)$, we estimate underlying population has a regression line as follows.

\begin{align} & \hat{Y} = \hat{\beta_0} + \hat{\beta_1}x & \text{SRF, Estimator of RV } E(Y|x), \text{ not } Y \\ & \hat{\varepsilon} = Y - \hat{Y} & \text{Estimator of RV } \varepsilon \end{align}

For given sample $(x_i, y_i)$ from sample set $(X,Y)$, a fitted value and residual are $$\begin{aligned} & \hat{y_i} = \hat{Y}(x_i)= b_0 + b_1x_i & \text{Fitted value, Estimate of RV } E(Y|x) \text{ at } x_i \\ & \hat{\varepsilon_i} = y_i - \hat{y_i} & \text{Residual, Estimate of RV } \varepsilon \text{ at } (x_i,y_i) \end{aligned}$$

Using OLS,

$$\begin{aligned} & \hat\beta_1 = \dfrac{\sum_{(x,y)}(y - \overline{Y})(x - \overline{X}) }{\sum_{x}(x - \overline{X})^2} & \text{Slope RV, Estimator of Parameter } \beta_1 \\ & \hat\beta_0 = \overline{Y} - \hat\beta_1\overline{X} & \text{y-intercept RV, Estimator of Parameter } \beta_0 \\ & b_1 = \dfrac{\sum_{i}(y_i - \overline{y})(x_i - \overline{x}) }{\sum_{i}(x_i - \overline{x})^2} & \text{Slope constant, Estimate of Parameter } \beta_1 \\ & b_0 = \overline{y} - b_1\overline{x} & \text{y-intercept constant, Estimate of Parameter } \beta_0 \end{aligned}$$

  • $\hat{\beta_0},\hat{\beta_1}$ are estimators of $\beta_0,\beta_1$ for any sample set. $b_0,b_1$ are estimates of $\beta_0,\beta_1$ for given sample set

  • Estimators: $\hat{\beta_0}, \hat{\beta_1}, \hat{X}, \hat{Y}, \hat{\varepsilon}$

  • Estimates: $b_0, b_1, \overline{x}, s_X^2, \overline{\hat{y}}=\overline{y}, s_{Y|x}^2=s^2$

The entire draft is available here


  1. Devore's book here
  2. Gujarati's Basic Econometrics here.

Update 1: As per whuber advice, I also have started separate distinct to the point question, with first one here. Looking forward for further feedback. (For eg, we could either quash this question and continue there, or bring one of them (or first), here modifying this question)

Update 2: Specific split up questions (including first one just for entirety).
1. Doubt about Predictor Variable
2. Is residual an estimate of error
3. Characterizing estimator,estimate separately
4. Why SSE has (n-2)df - not duplicate

  • 1
    $\begingroup$ I think this isn't an on topic question for SE. It's a bit of an XY problem. Honestly, regression terminology is hard to grapple with, but your attempt to reinvent the wheel here is unfathomable for me. A lot of unnecessary terms, and misuse, and incorrect assumptions. Sxy is, for instance, not a constant because it has variance. $\endgroup$
    – AdamO
    Commented Nov 20, 2018 at 17:55
  • $\begingroup$ The first section of this post is addressed at stats.stackexchange.com/questions/16381/…: looking at that might be a good start. $\endgroup$
    – whuber
    Commented Nov 20, 2018 at 18:01
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – whuber
    Commented Nov 21, 2018 at 14:09