7
$\begingroup$

Let's say I am running a regression in which my dependent variable is homicide and my variable of interest is access to violent videogames. Let's say that I also throw in the kitchen sink with regard to my control variables-- I have 38 demographic controls, 30 criminological controls that may or may not be relevant, and so on. Some of these controls may even contain fuzzy or bad data (typographical errors, blank cells, and so on). What are some of the negative consequences of these sloppy regressions?

I was told by a grad student in Statistics that these controls will have no effect on the p-value between the dependent variable and the variable of interest, even if the coefficients on the controls will be senseless. But if this were true, why don't all academics just throw in the kitchen sink in their regression? Is it possible to the p-values to become smaller through the addition of junk controls?

$\endgroup$
1
  • 5
    $\begingroup$ Here is one intuition pump: if you have $n$ samples available but add more than $n$ kitchen sink predictors (I love this term, by the way), then your regression will have $R^2=1$ and the coefficient for violent videogames can be anything. You added too many features, leading to overfitting and meaningless result. $\endgroup$
    – amoeba
    Nov 23, 2015 at 16:17

1 Answer 1

4
$\begingroup$

We can write the formula for the standard error of a regression coefficient $\hat \beta_j$ as

$$\sqrt{\frac{\sum_{i=1}^n \hat u^2}{(n-k-1)\sum_{i=1}^n(x_{ij}-\bar x_j)^2(1-R^2_j)}}$$

where $\hat u^2$ are the regression residuals, $n$ is the numbers of observations, $k$ is the number of regressors, and $R^2_j$ is the $R^2$ from a regression of $x_j$ on all other independent variables.

If the additional variables have no effect on the dependent variable the only parts of the formula that will change will be $k$ and $R^2_j$, both of which will increase the standard error, leading to larger p-values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.