How do you deal with “nested” variables in a regression model?

Consider a statistical problem where you have a response variable that you want to describe conditional on an explanatory variable and a nested variable, where the nested variable only arises as a meaningful variable for particular values of the explanatory variable. In cases where the explanatory variable does not admit a meaningful nested variable, the latter is usually coded either as NA in the data set, or if it is coded with a value, that value is merely a placeholder that does not have any meaningful interpretation.

This situation tends to arise whenever you have an explanatory variable indicating the existence of a thing, and one or more nested variables describing the characteristics of that thing. Some examples of this kind of situation in statistical problems are the following:

• The explanatory variable is an indicator of whether a survey participant is married, and the nested variable is some characteristic of the spouse (e.g., education, age, etc.);

• The explanatory variable is an indicator of the presence of an item in a space, and the nested variable is a measure of some characteristic of the item (e.g., size, distance, etc.);

• The explanatory variable is an indicator of the occurrence of an event and the nested variable is a description of some characteristic of the event (e.g., duration, magnitude, etc.).

In these kinds of situations, we often want to build a regression-type model (in the broad sense that includes GLMs, GLMMs, etc.) describing the relationship between the response variable and the other variables. It is not obvious how to deal with the nested variable in this type of model.

Question: How do we deal with the nested variable in this type of model?

Note: This question is designed to give a generalised answer to a recurring question on CV.SE regarding nested variables in regression (see e.g., here, here, here and here). This question is designed to give a generalised context-independent example of this problem.

• I would have left this as a comment but I do not have enough reputation. I am having trouble using this solution in R - glm() or lm(). I am using the model: y ~ x1 + x1:x2 Unfortunately if I encode the missing data as NA the default na.action removes the rows with NAs and leaves x1 with only one level - making the model equivalent to just: y ~ x2 If I use argument to glm: na.action = na.pass I get an error: Error in glm.fit(x = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, : NA/NaN/Inf in 'x' If instead I encode the missing variable as 0, the nested model: y ~ x1 + x1:x2 Gives the exact – Adam Waring Jan 14 at 13:10

Meaningless values of nested variables must not affect your model: The crucial desideratum with this type of data analysis is that the nested variable must not impact the model if the original explanatory variable does not admit it as a meaningful variable. In other words, the model must be of a form that ignores meaningless values of the nested variable. This is a crucial requirement for a valid model with nested variables, since it ensures that the model output is not affected by arbitrary coding choices.

Modelling with nested variables: This requirement is achieved by putting the nested variable into the model only as an interaction with the original explanatory variable, without including it as a main effect. (More specifically, the nested variable should be interacted with a logical statement on the explanatory variable indicating that it is a meaningful variable.) Note that this is an exception to the general rule that terms should not be included as interactions without a main effect term.

Consider the general case where the nested variable is only meaningful when the explanatory variable is in some set of values A. In that case, you would use a model form like this:

response ~ 1 + explanatory + (explanatory %in% A):nested + ...

In the common case where your explanatory variable is an indicator variable (with a value of one giving rise to a meaningful nested variable), this model form simplifies to this:

response ~ 1 + explanatory + explanatory:nested + ...

Observe that in these model statements there is no main effect term for the nested variable. This is by design --- the nested variable should not have a main effect term, since it is not a meaningful variable in the absence of a condition on the explanatory variable. With this type of model form you will get an estimate for the effect of the explanatory variable and another estimate for the effect of the nested variable.

Coding nested variables in your data: When dealing with data-frames that list the variables for the regression, it is good practice for the values of the nested variable to be coded as NA in cases where it does not meaningfully arise from the explanatory variable. This tells the reader that there is no meaningful variable here. Some analysts code these variables with other values, like zero, but that is generally bad practice, since it can be mistaken for a meaningful quantity.

Mathematically, if you multiply any real number by zero, you get zero. However, if you are coding in R you have to be careful here because the program multiplies 0:NA to give NA instead of 0. This means that you may need to re-code the NA values to zero for the purposes of model-fitting, or construct the design-matrix for the model so that these values are set to zero.

• What does the design matrix $X$ looks like? Here you mentioned that NA can be used. But I think the software converts NA to some kind of code, because $X$ does not accept missing value. – user158565 Oct 17 '18 at 14:02
• Since I have not specified any particular software (but I am using the syntax of R) it is not clear to me why NA values would not be acceptable. In R you can certainly have NA values in your data-frames. – Ben Oct 17 '18 at 20:54
• Suppose there are NAs in $X$, how to calculate $(X'X)^{-1}$ ? – user158565 Oct 17 '18 at 21:15
• With the models used in this answer, the NA values occur in the data-frame for the variables, but they don't appear in the design matrix, since the nested variable only enters the model through an interaction. – Ben Oct 17 '18 at 21:22
• That is my original question: What does the design matrix look like? In fact, I want to do it in SAS, but missing value cannot be in design matrix. – user158565 Oct 17 '18 at 21:34