In the Classical Regression Model i.e. $\big(E(y|x)=\alpha +\beta x$ and $Var(y|x)=\sigma^2\big)$ with only two coefficients for intercept $\alpha$ and slope $\beta$ of a dummy variable $x$, we can interpret $\alpha$ as the the mean of values for which $x=0$ and $\beta$ as the difference of the means of the data where $x=1$ and $x=0$ respectively. This makes intutitive sense, but how can I formally show that I can deduce those special expression from the standard definition: $$\hat{\alpha}=\bar y -\hat{\beta} \bar x$$ and $$\hat{\beta}=\frac{\frac{1}{n}\sum (x_i-\bar x)(y_i-\bar y)}{\frac{1}{n}\sum (x_i-\bar x)^2}.$$

I cannot reach the formulation in terms of group means.

  • 2
    $\begingroup$ Have you tried by substituting the $x_i$s with kind of dummy variables (taking value 0 or 1) in your formula above? Maybe starting from the formula for $\hat{\beta}$ that's a way to move forward.. $\endgroup$ Commented Jan 11, 2014 at 17:31
  • $\begingroup$ This question appears to conflate two distinct concepts: the model and its estimates. The interpretation of $\alpha$ and $\beta$ depends on the model. The role of the formulas is solely to make reasonable guesses concerning the actual values of the parameters but otherwise plays no role in their interpretation. Therefore it is illogical--and in principle could be impossible--the derive the interpretation from an analysis of the formulas for the estimates. $\endgroup$
    – whuber
    Commented Jan 11, 2014 at 18:21
  • $\begingroup$ Related: stats.stackexchange.com/questions/354803/… $\endgroup$ Commented Jul 7, 2018 at 13:37

1 Answer 1


The theoretical model is

$$E(Y\mid X)=\alpha +\beta X$$

Assuming that $X$ is a $0/1$ binary variable we notice that

$$E(Y\mid X=1) - E(Y\mid X=0)=\alpha +\beta -\alpha = \beta $$

I think that the OP asks "does the OLS estimator "mimics" this relationship, being perhaps its sample analogue?"

Let's see: we have that

$$\hat{\beta}=\frac{\frac{1}{n}\sum (x_i-\bar x)(y_i-\bar y)}{\frac{1}{n}\sum (x_i-\bar x)^2} = \frac {\operatorname{\hat Cov(Y,X)}}{\operatorname{\hat Var(X)}} $$

Now since $X$ is a binary variable, i.e. a Bernoulli random variable, we have that ${\operatorname{Var(X)} = p(1-p)}$ where $p\equiv P(X=1)$. Under a stationarity assumption, the sample estimate of this probability is simply the sample mean of $X$, denoted $\bar x$ and one can verify that indeed $$\frac{1}{n}\sum (x_i-\bar x)^2 = {\operatorname{\hat Var(X)}}=\bar x (1-\bar x) =\hat p(1-\hat p)$$

Let's turn now to the covariance. We have

$$\operatorname{\hat Cov(Y,X)}=\frac{1}{n}\sum (x_i-\bar x)(y_i-\bar y) = \frac{1}{n}\sum x_iy_i -\bar x \bar y$$

Denote $n_1$ the number of those observations for which $x_i=1$. We can write

$$\frac{1}{n}\sum x_iy_i = \frac{1}{n}\sum_{x_i=1} y_i = \frac{n_1}{n}\cdot \frac{1}{n_1}\sum_{x_i=1} y_i = \hat p\cdot (\bar y \mid X=1) = \hat p \cdot \hat E(Y\mid X=1)$$

Also $\bar y = \hat E(Y)$ and using the law of total expectations we have

$$\hat E(Y) = \hat E(Y \mid X=1) \cdot \hat p + \hat E(Y \mid X=0)\cdot (1-\hat p)$$

Inserting all these results in the expression for the sample covariance we have

$$\operatorname{\hat Cov(Y,X)}= \hat p \cdot \hat E(Y\mid X=1) - \hat p\cdot \left[\hat E(Y \mid X=1) \cdot \hat p + \hat E(Y \mid X=0)\cdot (1-\hat p)\right]$$

$$= \hat p(1-\hat p)\cdot \left[\hat E(Y \mid X=1) - \hat E(Y \mid X=0)\right]$$

Inserting all in the expression for $\hat \beta$ we have

$$=\hat{\beta} = \frac {\operatorname{\hat Cov(Y,X)}}{\operatorname{\hat Var(X)}} = \frac {\hat p(1-\hat p)\cdot \left[\hat E(Y \mid X=1) - \hat E(Y \mid X=0)\right]}{\hat p(1-\hat p)} $$

$$\Rightarrow \hat{\beta} = \hat E(Y \mid X=1) - \hat E(Y \mid X=0)$$

which is the sample analogue/feasible implementation of the theoretical relationship. I leave the demonstration related to the $\hat \alpha$ for the OP to work out.

  • $\begingroup$ would it be the same result if we have a Heteroscedasticity model? for example: variance depends on X, the estimator of beta would be the same as above? $\endgroup$ Commented Dec 19, 2020 at 11:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.