1
$\begingroup$

I am wondering how to find an OLS estimator for beta_1 in terms of population moments when the corresponding independent variable is a Bernoulli random variable equal to either 0 or 1. My professor gave the following formula during class (shown below). Any guidance would be very appreciated!

This is what I have so far, working from the formula provided:

enter image description here enter image description here

Alternatively:

enter image description here

enter image description here

enter image description here

$\endgroup$
2
  • $\begingroup$ Drawing a scatterplot will make very short work of this exercise, because it will make immediate the insight that the fitted line must pass through the two mean values. $\endgroup$
    – whuber
    Jul 15, 2020 at 20:22
  • $\begingroup$ See stats.stackexchange.com/questions/439092/… for an answer. $\endgroup$
    – whuber
    Jul 19, 2020 at 20:42

1 Answer 1

0
$\begingroup$

Since this is clearly a homework problem, I won't give away the full answer. First, I'll note that the formula you posted does not appear to be correct; the denominator should be $\sum(D_i-\bar D)^2$. $\sum(D_i-\bar D)$ is just $0$.

The formula says that a sample estimate of $\beta_1$, namely, $\hat \beta_1$, is equal to the true value of $\beta_1$ in the population plus a bias term. The bias term is a ratio of two quantities, which involve $D_i$, the binary predictor; its mean $\bar D$ (also equal to the proportion of $1$s; and $\epsilon_i$, a random disturbance. The bias term can be written in terms of population moments (e.g., mean, variance, covariance, skew, etc.), and that is what you are being asked to figure out how to do.

I'll provide two insights that might help you:

  1. $\epsilon_i$ is centered at $0$, meaning $\epsilon_i = \epsilon_i - \bar \epsilon$.
  2. You can multiply the numerator and denominator by $\frac{1}{n}$, that is, $\frac{1/n}{1/n} = 1$, which seems trivial but is actually critical in converting the numerator and denominator into values that resemble population parameters.

Try using these clues to solve the problem. Let me know if you need more hints.


The original formula you presented was $$\hat\beta_1=\beta_1 + \frac{\sum(D_i - \bar D)\epsilon_i}{\sum (D_i - \bar D)^2}$$

When you replace $\epsilon_i$ with $\epsilon_i - \bar \epsilon$, the numerator is $\sum(D_i - \bar D)(\epsilon_i - \bar \epsilon)$. Hopefully that looks like a familiar formula, though, again, you need the second insight to make the connection. Using the second insight, you can write $\frac{\sum W}{\sum Z}$ as $\frac{\frac{1}{n}\sum W}{\frac{1}{n}\sum Z}$ (here I simply used $W$ and $Z$ as placeholders). So you can turn a ratio of sums into a ratio of means. What are they the means of in this case?


Here's what you are supposed to see:

$$ \sum(D_i - \bar D)\epsilon_i = \sum(D_i - \bar D)(\epsilon_i - \bar \epsilon) $$ $$\begin{aligned} \hat\beta_1 &= \beta_1 + \frac{\sum(D_i - \bar D)\epsilon_i}{\sum (D_i - \bar D)^2} \\ &= \beta_1 + \frac{\sum(D_i - \bar D)(\epsilon_i - \bar \epsilon)}{\sum (D_i - \bar D)^2} \\ &= \beta_1 + \frac{\frac{1}{n}\sum(D_i - \bar D)(\epsilon_i - \bar \epsilon)}{\frac{1}{n}\sum (D_i - \bar D)^2} \\ &= \beta_1 + \frac{\text{Cov}(D, \epsilon)}{\text{Var}(D)} \end{aligned}$$

That's it. $\text{Cov}(D, \epsilon)$ and $\text{Var}(D)$ are moments of the joint distribution of $D$ and $\epsilon$. The usual OLS estimator for univariable regression is $\frac{\text{Cov}(D, Y)}{\text{Var}(D)}$; expanding $Y$ into $Y = \beta_0 + \beta_1 D + \epsilon$ will reduce to the formula I derived above.


Expanding the equation for $Y$ into the usual OLS solution:

$$\begin{aligned} \hat\beta_1 &= \frac{\text{Cov}(D, Y)}{\text{Var}(D)} \\ & = \frac{\text{Cov}(D, \beta_0 + \beta_1 D + \epsilon)}{\text{Var}(D)} \\ & = \frac{\text{Cov}(D, \beta_0) + \text{Cov}(D, \beta_1 D) + \text{Cov}(D, \epsilon)}{\text{Var}(D)} \\ &= \frac{0 + \beta_1 \text{Cov}(D, D) + \text{Cov}(D, \epsilon)}{\text{Var}(D)} \\ &= \frac{\beta_1 \text{Var}(D)}{\text{Var}(D)} + \frac{\text{Cov}(D, \epsilon)}{\text{Var}(D)} \\ &= \beta_1 + \frac{\text{Cov}(D, \epsilon)}{\text{Var}(D)} \end{aligned}$$

$\endgroup$
10
  • $\begingroup$ Thank you for the response. I updated my question to show what I have done thus far. I'm unsure how those two insights factor in, as I end up finding that the bias term cancels out, leaving me only with the true value of đť›˝1. I'm not sure if this is what was supposed to happen, though, and I'm not sure if it's in terms of population moments. It's in terms of expected value, so it is in terms of mean; I assume, then, that it is in terms of population moments? $\endgroup$
    – Student
    Jul 15, 2020 at 18:06
  • $\begingroup$ I made some edits. I think your math is a little incorrect, but trying going down the route I presented. In this case, simplification comes not from expanding terms into their parts but rather from recognizing a complicated formula represents a simple nameable quantity. $\endgroup$
    – Noah
    Jul 15, 2020 at 18:40
  • $\begingroup$ I'm not sure what multiplying the numerator and denominator by 1/n does. I could pull out the terms inside the summation so that I have 1/n multiplied by n multiplied by the mean of all terms inside the summation, but I don't understand what this is giving me. The formula does look similar to the expected value of beta_1 in the univariate case. Why can't I just use this formula except with D instead of X and using Y as usual? $\endgroup$
    – Student
    Jul 16, 2020 at 4:34
  • $\begingroup$ See my last edit. $\endgroup$
    – Noah
    Jul 16, 2020 at 5:11
  • $\begingroup$ Wouldn’t solving this problem by expanding Y result in not having to include the beta_1 term? And isn’t it an issue that the beta_ term is still present? Or should I just write the beta_1 term in terms of expected values? $\endgroup$
    – Student
    Jul 16, 2020 at 15:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.