Factor for rows without attribute and rows with attribute and additional characteristic

Question

I have run an experiment in which I've measured some metric X, and collected an associated attribute attr's value if and only if the value of X exceeds some threshold t. If a case's X value doesn't exceed t, then attr is not captured. Here's what my data look like:

ID  attr_val  no_attr  outcome
 0       NaN        1        0
 1       NaN        1        1
 2         0        0        1
 3         4        0        1
 4         2        0        1
 5         1        0        0

Where ID is a unique identifier for each case, no_attr indicates whether the attr value was captured for that particular case (i.e. whether the X value exceeded t), and the binary outcome outcome is shown.

Now, I want to predict outcome based on the attribute value if a case's X exceeded t, and I also want to measure whether a case's X not exceeding t is predictive, as well.

In order to keep IDs 0 and 1 in the model, attr_val will need to be populated with some value, not just nulls. But I don't really feel comfortable filling in 0, for example, because IDs 0 and 1 didn't have a chance to give their attr_val because their X values didn't exceed t. However, this X-exceeding-t criterion is very important to my experiment, so I can't just take the attr_vals for rows 0 and 1 anyway.

Running a logistic regression of the form outcome ~ attr_val + no_attr currently would make the design matrix singular, as just zeroes are included for no_attr if I haven't filled in any nulls in attr_val. Is the right approach here to augment no_attr by 1 so we're not multiplying by zeroes down the line? Or is there a better way to encode this problem?

Answer 1

Question of this sort are easily resolved by writing down your model.

Suppose you did use some number, say $a$, to represent the missing values of attr_val. Then your model would be in the form

$$\Pr(\text{outcome}) = h(\beta_0 + \beta_{\text{attr_val}} [\text{attr_val}] + \beta_{\text{no_attr}} [\text{no_attr}])$$

for your chosen link function $h$. Since the probabilities depend on the argument of $h$, we will focus on its values.

Consider two cases:

1. attr_val is not missing. Then no_attr is zero and $$\Pr(\text{outcome}) = h(\beta_0 + \beta_{\text{attr_val}} [\text{attr_val}] ).$$

2. attr_val is missing. Then attr_val is set to the value $a$, no_attr equals $1$, and the formula becomes $$\Pr(\text{outcome}) = h(\beta_0 + \beta_{\text{attr_val}} a + \beta_{\text{no_attr}} ).$$

You may interpret $(1)$ as a model for the output when attr_val is not missing and $(2)$ determines the fit when attr_val is missing. If you conveniently choose $a=0$, this simplifies further: $\beta_0 + \beta_{\text{no_attr}}$ determines the fit in such cases, allowing you to interpret $\beta_{\text{no_attr}}$ as a change in the intercept $\beta_0$ attributed to the circumstances $X\le t$.