Significance of coefficient from two different regressions

Question

Question) Researcher A regresses $y$ on a constant and on $x$, obtaining OLS estimates $a$ (intercept), and $b$ (slope) and residual vector $e$. Researcher B, however, regresses $y$ on a constant, on $x$, and on $z$, obtaining OLS estimates $\hat \alpha$, $\hat \beta$, and $\hat \gamma$ as the respective coefficient estimates, and $\hat \epsilon$ as residuals. Explain in detail under what circumstances the following could be true.

(a). $b$ is statistically significant at the 5% level yet $\hat \beta$ is not.

(b). $\hat \beta$ is statistically significant at the 5% level yet $b$ is not.

My guess) I think the problems are related with correlation coefficient between $x$ and $z$. Since $Var(\hat \beta) = Var(b) \times 1/(1-r^2_{xz})$, where $r_{xz}$ is correlation coefficient between $x$ and $z$, $Var(\hat \beta)$ will go to infinity as correlation coefficient reaches 1. So, if there exists high correlation between $x$ and $z$, t-ratio of $\hat \beta$ can be insignificant.

Thus, I think (a) is the case where high correlation relationship exists, and (b) is impossible.

However, I feel that my guess is somewhat flawed and need more concrete proof. Can anyone help me?

Answer 1

As this is a self-study question, it's best to provide some guidance which will help lead to the solution:

In your question, two models are essentially being compared

$$y=\alpha+bx+e$$

$$y=\alpha+\beta x+\gamma z+\epsilon$$

Recall that in a multiple linear regression we estimate $\beta$ as:

$$\hat{\beta}=\frac{\Sigma\hat{r}_{1}y_i}{\Sigma\hat{r}_{1}^2}$$

where $\hat{r}_{i1}$ are the OLS residuals from a simple regression of $x$ on $z$ and can be thought of as the part of $x$ that is uncorrelated with $z$. We can then perform an OLS regression of $y$ on $\hat{r_1}$. As the residuals will have a sample average of 0, the slope of the regression will be equal to $\beta$.

When the model is underspecified, important variables are not included in the model and can lead to biased estimates. Assume that the second model (containing $z$) is the true population model, and that the model satisfies the assumptions of multiple linear regression. To get an unbiased estimate of the effect of $x$ on $y$, we should include $z$. We can derive the expected value of $b$ conditional on the sample values of $x$ and $z$, as $b$ is just the slope from the first model:

$$\hat{b}=\frac{\Sigma(x_i-\bar{x})y_i}{\Sigma(x_i-\bar{x})^2}$$

If the $b$ and $\beta$ differ between the two models, we can express this difference algebraically as

$$b=\hat{\beta}+\hat{\gamma}\tilde{\delta}_1$$

where $\tilde{\delta}_1$ is the slope from a simple regression of $z$ on $x$. Essentially, the confounding term is the partial effect of $z$ on $y$ times the slope in the regression of $z$ on $x$ (it can almost be thought of as the shared effect of $z$ and $x$ on $y$). Note that $\hat{\beta}$ and $\hat{\gamma}$ are the estimated slope parameters from the correctly specified model.

If the multiple linear regression assumptions hold, both $\hat{\beta}$ and $\hat{\gamma}$ are unbiased estimators of $\beta$ and $\gamma$. Therefore as $\tilde{\delta}_1$ only relies on independent variables in the sample (i.e. it can be considered nonrandom):

$$E(b)=E(\hat{\beta}+\hat{\gamma}\tilde{\delta}_1)=E(\hat{\beta})+E(\hat{\gamma})\tilde{\delta}_1=\beta+\gamma\tilde{\delta}_1$$

And the bias can be written as:

$$Bias(b)=E(b)-\beta=\gamma\tilde{\delta}_1$$

For further reading and more information I suggest:

Wooldridge, J. M. 2001. Econometric Analysis of Cross Section and Panel Data, MIT Press.

Wooldridge, J. M. 2006. Introductory econometrics: a modern approach. Mason, OH, Thomson/South-Western.

https://en.wikipedia.org/wiki/Omitted-variable_bias