5
$\begingroup$

Why do we need to include precision variables in a regression model (i.e., a variable that is associated with the outcome but not the predictor of interest)?

$\endgroup$
3
  • 1
    $\begingroup$ What type of model are we talking about here? Linear regression?! $\endgroup$ Mar 11, 2014 at 3:33
  • 1
    $\begingroup$ Any type of regression. $\endgroup$
    – guestom
    Mar 11, 2014 at 4:21
  • 2
    $\begingroup$ You seem to be asking about what are more usually called covariates. $\endgroup$
    – Glen_b
    Mar 11, 2014 at 7:09

3 Answers 3

5
$\begingroup$

Regression coefficient is often characterisized as a partial correlation coefficient which means it will show effect of particular variable X to the outcome variable Y after effects of other variables, Z, are controlled.

What happens when you omit Z and leave only X? Do coefficient for X change?

If variables Z and X are orthogonal, which rarely happens outside experimental data, these coefficients will not change but in other situations you cannot say that coefficient for X variable measures just effect of variation in X for Y.

In econometrics omission of important variables is called omitted variable bias and it states that marginal effect from the X variable to the Y will no longer be estimated without bias.

Goal of modeling is to find out all relevant variables and to check that residual variation behaves well.

$\endgroup$
5
$\begingroup$

Here is my contribution to Anscombe's Quartet.

The below image shows a regression line on some imaginary data. The estimated regession line is $\hat{y}_{i} = 3 + .5x$, the p-value for the t-test of $H_{0}: \beta=0$ equals 0.002, and the $R^{2}$ for my regression model is 0.67 (just as for the four graphs in Anscombe, 1973).

Regression line of y on x

This regression/regression line says that as $x$ increases by two units, $y$ increases by one unit. But there's something wrong: I have omitted another variable that also explains $y$. This second variable is dichotomous, and indicates which group one is in (the above analysis combined data from both groups). We can model it as $y_{i} = \beta_{0} + \beta_{x}x_{i} + \beta_{z}z_{i} + \varepsilon_{i}$, where $\varepsilon\sim \mathcal{N}\left(0,\sigma_{\varepsilon}\right)$.

Regression of y on x and z

Now we see that actually, as $x$ increases, $y$ actually decreases! This means our first model was not only incorrect, but badly so, because no matter whether one is purple or orange, $y$ has an inverse relation to $x$. In the first model we violated the assumption of no missing variables, and this is why it is important to include covariates (what you label "precision variables") in our model.

$\endgroup$
2
  • 5
    $\begingroup$ Simpson's Paradox is worth reading about, for those not already familiar with it. $\endgroup$
    – R.M.
    Aug 15, 2018 at 17:25
  • $\begingroup$ @TylerWick Wow! Don't know how I goofed, that, and I thank you for the edit! $\endgroup$
    – Alexis
    Oct 27, 2021 at 1:24
2
$\begingroup$

Precision variables would help to reduce the standard errors and hence shrink confidence intervals on the coefficients that you are interested in, and hence make it easier to find significant effects of the variables you are interested in.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.