I have run an experiment in which I've measured some metric X, and collected an associated attribute attr
's value if and only if the value of X exceeds some threshold t. If a case's X value doesn't exceed t, then attr
is not captured. Here's what my data look like:
ID attr_val no_attr outcome
0 NaN 1 0
1 NaN 1 1
2 0 0 1
3 4 0 1
4 2 0 1
5 1 0 0
Where ID
is a unique identifier for each case, no_attr
indicates whether the attr
value was captured for that particular case (i.e. whether the X
value exceeded t), and the binary outcome outcome
is shown.
Now, I want to predict outcome based on the attribute value if a case's X exceeded t, and I also want to measure whether a case's X not exceeding t is predictive, as well.
In order to keep IDs 0 and 1 in the model, attr_val
will need to be populated with some value, not just nulls. But I don't really feel comfortable filling in 0, for example, because IDs 0 and 1 didn't have a chance to give their attr_val
because their X values didn't exceed t. However, this X-exceeding-t criterion is very important to my experiment, so I can't just take the attr_val
s for rows 0 and 1 anyway.
Running a logistic regression of the form outcome ~ attr_val + no_attr
currently would make the design matrix singular, as just zeroes are included for no_attr
if I haven't filled in any nulls in attr_val
. Is the right approach here to augment no_attr
by 1 so we're not multiplying by zeroes down the line? Or is there a better way to encode this problem?
[0, inf)
. To your last question, no; the inverse of theno_attr
(has_attr
?) is essentially a link for a particular case's being able to have a value forattr_val
. $\endgroup$ – blacksite Dec 18 '17 at 13:35