Given a sample set $(X,Y)$, supposing $X$ is fixed and known,

Population Regression Function,PRF:
Hypothesizing the underlying population, we have,

$$\begin{aligned} & E(Y) = \beta_0 + \beta_1x & \scriptsize \text{(1) PRF} \\ & Y = E(Y) + \varepsilon & \scriptsize \text{(2) Prediction} \end{aligned}$$

where $\beta_0,\beta_1,\varepsilon$ are unknown parameters.

Sample Regression Function, SRF:
And for that given sample set, we have estimators and estimates as follows.

$$\begin{aligned} & \hat{Y} = \hat{\beta_0} + \hat{\beta_1}x & \scriptsize\text{(3) SRF, Estimator of RV } E(Y|x), \text{ not } Y \\ & \hat{\varepsilon} = Y - \hat{Y} & \scriptsize\text{(4) Estimator of RV } \varepsilon \\ & \hat\beta_1 = \dfrac{\sum_{(x,y)}(y - \overline{Y})(x - \overline{X}) }{\sum_{x}(x - \overline{X})^2} & \scriptsize\text{(5) Slope RV, Estimator of Parameter } \beta_1 \\ & \hat\beta_0 = \overline{Y} - \hat\beta_1\overline{X} & \scriptsize\text{(6) y-intercept RV, Estimator of Parameter } \beta_0 \\ \end{aligned}$$

$$\begin{aligned} & \hat{y_i} = \hat{Y}(x_i) = b_0 + b_1x_i & \scriptsize\text{(7) Fitted value constant, Estimate of RV } E(Y|x) \text{ at } x_i \\ & \hat{\varepsilon_i} = \hat{\varepsilon}(x_i,y_i) = y_i - \hat{y_i} & \scriptsize\text{(8) Residual constant, Estimate of RV } \varepsilon \text{ at } (x_i,y_i) \\ & b_1 = \dfrac{\sum_{i}(y_i - \overline{y})(x_i - \overline{x}) }{\sum_{i}(x_i - \overline{x})^2} & \scriptsize\text{(9) Slope constant, Estimate of Parameter } \beta_1 \\ & b_0 = \overline{y} - b_1\overline{x} & \scriptsize\text{(10) y-intercept constant, Estimate of Parameter } \beta_0 \end{aligned}$$

Convention Reference:

Wiki says,

  • If parameter is $\theta$, then traditionally estimator is indicated by $\hat\theta$
  • Being function of data, estimator is also a random variable
  • If X is a RV corresponds to that data, estimator could be indicated as $\hat{\theta}(X)$
  • The estimate for $X=x$ is then $\hat\theta(x)$ which is a constant.

1. Conflict with the book: The book mixes the estimators and estimates in the equation, so it becomes difficult to grasp the concept (screenshot), so I want to maintain clear distinction for latter work. Related doubts will on that come later (for eg, the ratio $\frac{S_{xx}}{S_{yy}}$ and/or its components are constans or RVs? it matters because its used later to show $\hat\beta_1$ has normal distribution). The elaboration in my equations is not to reinvent any wheel, but bring out clarity in the equations to differentiate estimator, estimate clearly at the beginning in equations so as to avoid pitfalls in understanding upcoming topics.


  1. Is it correct to say,

    $\hat\beta_0,\hat\beta_1$ are estimators of $\beta_0,\beta_1$ for any sample set. $b_0,b_1$ are estimates of $\beta_0,\beta_1$ for given sample set calculated using $\hat\beta_0,\hat\beta_1$ functions?

If not, how do we characterize $\hat\beta_0,\hat\beta_1,b_0,b_1$?

  1. Equations (7),(8) which are actually estimates are also having to be denoted by hat (from the book as it mixes estimators and estimates). I have further added index to differentiate, but do we have any better ways? For $\hat\varepsilon_i$ I could think of $e_i$, but for $\hat{y_i}$? What is usual standard here?

  2. I have used non-indexed $x,y$ in RHS for estimators and indexed $x_i,y_i$ in RHS for estimates so as to differentiate RHS, to emphasize former is a function and later is a constant. Is this ok or do we have any standard better ways of differentiation in RHS between estimators and estimates?

1. Devore's Probability and Statistics for Engineering and the Sciences (page 479 onwards)

Previous Question
Next Question


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.