8
$\begingroup$

Suppose I have a dependent variable that is a test score, $score^*_{i}$. It has been standardised for both age and gender. I want to measure the effect that a binary variable, say Adoption, has on the persons test score. Will it cause any problems if I include age and gender as controls in the regression of score on adoption (given that these were used to standardise the dependent variable)?

That is, I am regressing (using standard OLS):

$score^*_{i} = (score_{i} - \alpha_{1}AGE_{i} - \alpha_{2}Gender_{i}) = \beta_{0} + \beta_{1}Age_{i} + \beta_{2}Gender_{i} + \beta_{3}Adopted_{i} + \epsilon_{i}$
where $score_{i}$ is the raw score (i.e. unstandardized score), $\epsilon_{i}$ is the iid error term. I want to know if $\beta_{3}$ is affected by the inclusion of age and gender?

My thinking is that this may bias the OLS estimates of the age and gender coefficients, but that it would not affect the the estimate of interest--the Adoption estimate.

Is this correct? Or is it wrong altogether to include age and gender as controls in this setting? If so, could you please explain why?

Thanks!

$\endgroup$
2
  • $\begingroup$ How did you come up with $\alpha_1$ and $\alpha_2$? $\endgroup$
    – jbowman
    Commented Dec 20, 2013 at 22:07
  • $\begingroup$ Hi @jbowman. Sorry, I should have been more clear. What I mean by $score^∗_i=(score_i−α_1AGE_i−α_2Gender_i)$ is that the data I have on score is made of some normalization where the test creator has "somehow" transformed the RAW score using age and gender. I don't know the form of this transformation, I just put that as a "possible" example where the raw score is scaled by e.g. subtracting age and gender. I am assuming that $\alpha_1$ and $\alpha_2$ have somehow been chosen by the test maker. $\endgroup$
    – Seb
    Commented Dec 20, 2013 at 22:53

3 Answers 3

3
$\begingroup$

This is permissible. If the standardization has been done on a sample similar to the one you are using, then you would expect that the parameter estimates for age and gender would be small, but I see no reason that cannot be included (except that it may make the model more complex than it needs to be).

The problem you may have is interpretation of the scores. What you will have is not really the effect of age and gender on test score, it is the difference in those effects between your sample and the standardization sample.

Usually, with such scores, the standardization formula is available somewhere; it certainly ought to be available. You say it is not. Then how did you get your raw scores transformed? Or was this all done by computer?

$\endgroup$
2
  • $\begingroup$ Thanks @PeterFlom, I appreciate your comment--it is very helpful. Yes, I am sure the standardization formula is available, I just haven't had been given access to it yet. And I believe the raw scores were transformed by people looking up their standardized equivalent in tables. Thanks, again! $\endgroup$
    – Seb
    Commented Dec 21, 2013 at 1:41
  • $\begingroup$ Just thought to add that this will also affact the interpretation of the effect of adopted on score. $\endgroup$ Commented Dec 21, 2013 at 12:38
3
$\begingroup$

There are potential problems with this approach, besides the obvious question of why not just condition on age and sex in your model.

  • The standardization may have been done on a group of subjects that differ from your target group in a meaningful way, tilting its assessment of the age and sex effects
  • The standardization may have falsely assumed linearity in age and additivity for age and sex
  • The standardization may have been done using an improperly transformed $Y$
  • You are not taken into account the uncertainties associated with the standardization

As you can tell from the above, there are many advantages of avoiding "standardization" and instead doing full conditioning in your analysis.

$\endgroup$
1
$\begingroup$

If $\alpha_1$ and $\alpha_2$ are applied consistently to the entire sample, then you are correct in your thinking that it will bias the OLS estimates of age and gender but not of the adoption estimate, as it is just a linear transformation of scores based on Age and Gender. To boot if you know the value of the $\alpha$'s then you can back transform to get the estimates you are interested in, and if you know that they are correlated with being adopted they should definitely still be included in the model.

Consider that we aren't interested in the effects on the adjusted score, but on the original score;

$score_i = \beta_{01} + \beta_{11} Age + \beta_{21} Gender + \beta_{31} Adopted$

But we only observe $score^*_{i} = score_i - \alpha_1 Age - \alpha_2 Gender$. So we can only estimate the equation;

$score^*_{i} = \beta_{02} + \beta_{12} Age + \beta_{22} Gender + \beta_{32} Adopted$

We can now replace $score^*_{i}$ on the left hand side with the original score in which we are interested in, and then rearrange the equation so the age and gender are only on the right hand side.

$score_i - \alpha_1 Age - \alpha_2 Gender = \beta_{02} + \beta_{12} Age + \beta_{22} Gender + \beta_{32} Adopted$ $score_i = \beta_{02} + (\beta_{12} + \alpha_1) Age + (\beta_{22} + \alpha_2) Gender + \beta_{32} Adopted$

So if we actually know the values of $\alpha_1$ and $\alpha_2$ we can backtransform the estimates of $\beta_{12}$ and $\beta_{22}$ to the original estimates we are interested in. This actually shows that by standardizing the score age and gender should be included in the equation, as it can introduce dependencies that weren't originally there. E.g. $\beta_{11} Age$ and $\beta_{21} Gender$ could originally be zero in the unobserved equation, but are non-zero in the transformed equation (e.g. you would have to have the fortuitous luck that $\alpha_1 = -1 \cdot \beta_{11}$ for the effect of age to be zero in the transformed equation).

Unfortunately my experience with vendors who adjust scores like this is that they don't release the $\alpha$'s and the $\alpha$'s aren't applied uniformily to the entire sample (e.g. girls aged 10 would have different $\alpha$'s than boys aged 12). I don't believe this logic applies in those circumstances.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.