I've asked this question before, in the context of logistic-regression, but I'd like to pose it in a broader context here.

There are a lot of variables with interval or ratio-type values for some subjects, and undefined for the rest of subjects (typically based on a grouping variable.) An example would be the age at menarche, which has ratio-type values for females and is undefined for males. For lack of a better word, I'll call these type of variables semi-numerical. In my previous post, @jem77bfp showed me how to deal with these variables in a regression: by creating an interaction term. My question is, how can we code these variables into something numerical for general use? For instance, creating an interaction is not an option if you'd like to do a principal component analysis of several variables, some of which are of this type.

I could come up with one method of doing this, but it's far from ideal: break the numerical part into categories (say, "early", "standard", and "late" for age at menarche for females), assign a value of "undefined" for the other subjects, and use dummy coding to transform the resulting $k$ categories into $k-1$ binary variables.

The problem with this approach is that it comes with a loss of information in the numerical part of the variable. Before, we knew that $ageAtMenarche = 12$ is a smaller value than $ageAtMenarche = 15$ (by 3 years), but that information is lost with dummy coding.

I'd love to hear how other people deal with this problem. Are there any standard methods? Thanks!

  • 1
    $\begingroup$ You can apply the answer at stats.stackexchange.com/questions/1444/…, mutatis mutandis, to this situation. $\endgroup$
    – whuber
    Jan 25 '13 at 15:27
  • $\begingroup$ Thank you @whuber. I always learn a lot by reading your excellent responses. I'm somewhat lost here though. I'm probably missing something, but I can't see how I can apply the situation with zero values to here. $\endgroup$
    – user765195
    Jan 26 '13 at 22:30
  • $\begingroup$ Very closely related: stats.stackexchange.com/questions/6563/…. $\endgroup$
    – whuber
    Jan 28 '13 at 18:36

You do not need to lose information with dummy coding. Let's take age at menarche as an example of a numerical independent variable that is meaningless for a subpopulation. The intention (if I understand correctly) is to have two models, one for boys and the other for girls, but to fit them both simultaneously using all the data possible. For logistic regression the one for girls will have the form

$$\text{logit}(\Pr(Y=1) \mid \text{girl}) = \color{red}{\beta_0 X} + \color{gray}{(\beta_1 X_1 + \cdots + \beta_p X_p)}$$

where $X$ is age at menarche--expressed however you like--and $X_1, \ldots, X_p$ are all the other variables. (The form of the left hand side might change for other forms of regression, but that's incidental to the topic of variable encoding.) The one for boys is to have the form

$$\text{logit}(\Pr(Y=1) \mid \text{boy}) = \color{blue}{\alpha_0} + \color{gray}{(\beta_1 X_1 + \cdots + \beta_p X_p)}$$

where $X$ no longer enters and $\alpha_0$ is any gender effect. If we define $Z$ to be a dummy for boys (equal to $1$ for them and $0$ otherwise), then both models are subsumed under the general form

$$\text{logit}(\Pr(Y=1)) = \color{blue}{\alpha_0} Z + \color{red}{\beta_0 X}(1-Z) + \color{gray}{(\beta_1 X_1 + \cdots + \beta_p X_p)},$$

as you can readily check by seeing that setting $Z=0$ removes the $\color{blue}{\alpha_0}$ term and reduces the second to $\color{red}{\beta_0 X}$ (the first model) while setting $Z=1$ removes the $\color{red}{\beta_0 X}$ term and keeps the $\color{blue}{\alpha_0}$ term (the second model). The second term, $\color{red}{\beta_0 X}(1-Z)$, is recognizable as an interaction (between gender and age at menarche), making it a familiar construct susceptible to the usual model-building techniques and regression diagnostics.

When performing model-building exercises, you cannot include both $Z$ and $1-Z$ (at least not when the model includes a constant): they are collinear with the constant. This is a case where you must ignore the oft-heard advice to include every variable that is involved in any interaction. The interaction variable $\color{red}{X}(1-Z)$ is easily calculated manually. Just be sure to treat $X$ and $Z$ as a nested pair of variables: it makes no sense to include $X$ without $Z$ (that's the problem of defining age at menarche for boys), although it does make sense to include $Z$ without $X$ (that's a model which includes a gender difference but has dropped the age-at-menarche variable). In effect, to account for age at menarche you must include gender in the model, and then you can decide whether age at menarche helps improve the model for girls alone; it will have no effect on the boys.

A helpful reader (with insufficient reputation to comment) pointed out a potential pitfall in algebraically computing $X(1-Z)$ if $X$ has been recorded as a missing or null value (such as NA in R) for the boys, because this could cause $X(1-Z)$ to be missing rather than $0$ as intended (depending on how your computing platform handles missing-value codes). The variable "$X(1-Z)$", which is intended to equal $0$ for boys and $X$ for girls, should be computed with a conditional expression like ifelse(Z==1,0,X). This, I hope, further clarifies what I meant by the reference to "nested variables."

  • $\begingroup$ Thank you @whuber. As always, it's a real pleasure to read and learn from your excellent responses. Many thanks! $\endgroup$
    – user765195
    Jan 29 '13 at 4:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.