How do you deal with "nested" variables in a regression model?

Question

Consider a statistical problem where you have a response variable that you want to describe conditional on an explanatory variable and a nested variable, where the nested variable only arises as a meaningful variable for particular values of the explanatory variable. In cases where the explanatory variable does not admit a meaningful nested variable, the latter is usually coded either as NA in the data set, or if it is coded with a value, that value is merely a placeholder that does not have any meaningful interpretation.

This situation tends to arise whenever you have an explanatory variable indicating the existence of a thing, and one or more nested variables describing the characteristics of that thing. Some examples of this kind of situation in statistical problems are the following:

• The explanatory variable is an indicator of whether a survey participant is married, and the nested variable is some characteristic of the spouse (e.g., education, age, etc.);

• The explanatory variable is an indicator of the presence of an item in a space, and the nested variable is a measure of some characteristic of the item (e.g., size, distance, etc.);

• The explanatory variable is an indicator of the occurrence of an event and the nested variable is a description of some characteristic of the event (e.g., duration, magnitude, etc.).

In these kinds of situations, we often want to build a regression-type model (in the broad sense that includes GLMs, GLMMs, etc.) describing the relationship between the response variable and the other variables. It is not obvious how to deal with the nested variable in this type of model.

Question: How do we deal with the nested variable in this type of model?

---

Note: This question is designed to give a generalised answer to a recurring question on CV.SE regarding nested variables in regression (see e.g., here, here, here and here). This question is designed to give a generalised context-independent example of this problem.

Answer 1

Meaningless values of nested variables must not affect your model: The crucial desideratum with this type of data analysis is that the nested variable must not impact the model if the original explanatory variable does not admit it as a meaningful variable. In other words, the model must be of a form that ignores meaningless values of the nested variable. This is a crucial requirement for a valid model with nested variables since it ensures that the model output is not affected by arbitrary coding choices.

---

Modelling with nested variables: This requirement is achieved by creating an indicator variable that determines when your nested variable is meaningful, and putting the nested variable into the model only as an interaction with this indicator, without including it as a main effect. Note that this is an exception to the general rule that terms should not be included as interactions without a main effect term.

Consider the general case where the nested variable is only meaningful when the explanatory variable is in some set of values A. In that case, you would use a model form like this:

response ~ 1 + explanatory + (explanatory %in% A) + (explanatory %in% A):nested + ...

This assumes that the explanatory variable is continuous; if it is already a factor variable then the (explanatory %in% A) term will be redundant and can be removed. In the common case where your explanatory variable is an indicator variable (with a value of one giving rise to a meaningful nested variable), this model form simplifies to this:

response ~ 1 + explanatory + explanatory:nested + ...

Observe that in these model statements there is no main effect term for the nested variable. This is by design --- the nested variable should not have a main effect term, since it is not a meaningful variable in the absence of a condition on the explanatory variable. With this type of model form you will get an estimate for the effect of the explanatory variable and another estimate for the effect of the nested variable.

---

Coding nested variables in your data: When dealing with data-frames that list the variables for the regression, it is good practice for the values of the nested variable to be coded as NA in cases where it does not meaningfully arise from the explanatory variable. This tells the reader that there is no meaningful variable here. Some analysts code these variables with other values, like zero, but that is generally bad practice since it can be mistaken for a meaningful quantity.

Mathematically, if you multiply any real number by zero, you get zero. However, if you are coding in R you have to be careful here because the program multiplies 0:NA to give NA instead of 0. This means that you may need to re-code the NA values to zero for the purposes of model-fitting, or construct the design-matrix for the model so that these values are set to zero.

---

Cases where the base variable is a function of the nested variable: One situation that occasionally arises in regression analysis involving nested variables is the case where the nested variable has a sufficient amount of detail that it fully determines the initial explanatory variable that it arises from --- i.e., the original explanatory variable is a function of the nested variable. An example of this occurs in this question, where the analyst has an indicator variable DrugA for whether or not a drug has been taken, and a nested variable DrugA_Conc for the concentration of the drug. In this example, the latter variable allows a concentration value of zero, which is equivalent to the drug not being taken, and so DrugA is equivalent to DrugA_Conc != 0.

In these types of cases, the interaction term between the explanatory variable and the nested variable is functionally equivalent to the nested variable, and so it is possible (and usually desirable) to remove the initial explanatory variable from the model altogether, and simply use the nested variable on its own. This is legitimate in this case, because the values in the nested variable determine the value of the initial explanatory variable. We have noted above that it is often appropriate to code nested variables as NA when the conditions for them are not applicable. If the condition arises from an explanatory variable that is an indicator, and the indicator corresponds to use of the nested variable, then the event nested != NA is equivalent to explanatory. In such cases, it is possible to recode the nested variable so that the initial explanatory variable is not required in the model at all.

Note that care must be taken when looking at this situation. Even in the case where you are using an initial explanatory variable that is an indicator variable, it may be useful for interpretive purposes not to merge the explanatory variable and the nested variable. Moreover, in cases where the explanatory variable is not an indicator variable, it will usually contain information not contained in the nested variable, and so it cannot be removed.

---

You should consider hierarchical or linear mixed models: The above method ensures that your nested variables do not contribute to the regression in cases where they are meaningless. However, the use of OLS estimation with a standard regression model still assumes that the "error terms" in the model are uncorrelated. In cases where you have nested variables, this may give rise to correlated errors that are best represented by a hierarchical model or a linear mixed model. Consequently, when you have nested variables in your regression, you should consider whether or not outcomes for data points in the same nested "group" will have outcomes that are correlated (conditional on the other regressors) or not.

The present answer will not go into detail on hierarchical models and linear mixed models. They are both broad model classes with substantial statistical literature. Gelman and Hill (2007) gives a good overview of the subject, starting with standard linear regression and proceeding into multilevel hierarchical modelling. It also gives details on implementation in R.