I have run an experiment in which I've measured some metric X, and collected an associated attribute
attr's value if and only if the value of X exceeds some threshold t. If a case's X value doesn't exceed t, then
attr is not captured. Here's what my data look like:
ID attr_val no_attr outcome 0 NaN 1 0 1 NaN 1 1 2 0 0 1 3 4 0 1 4 2 0 1 5 1 0 0
ID is a unique identifier for each case,
no_attr indicates whether the
attr value was captured for that particular case (i.e. whether the
X value exceeded t), and the binary outcome
outcome is shown.
Now, I want to predict outcome based on the attribute value if a case's X exceeded t, and I also want to measure whether a case's X not exceeding t is predictive, as well.
In order to keep IDs 0 and 1 in the model,
attr_val will need to be populated with some value, not just nulls. But I don't really feel comfortable filling in 0, for example, because IDs 0 and 1 didn't have a chance to give their
attr_val because their X values didn't exceed t. However, this X-exceeding-t criterion is very important to my experiment, so I can't just take the
attr_vals for rows 0 and 1 anyway.
Running a logistic regression of the form
outcome ~ attr_val + no_attr currently would make the design matrix singular, as just zeroes are included for
no_attr if I haven't filled in any nulls in
attr_val. Is the right approach here to augment
no_attr by 1 so we're not multiplying by zeroes down the line? Or is there a better way to encode this problem?