I working with some data to perform some basic logistic regression. The data looks something like this (simplified here):

userid   eventid   eventdate       label
1        A         2016-10-15      1
2        A         2016-10-15      0   
5        B         2016-09-15      1
1        B         2016-09-15      0
2        C         2016-09-10      0
1        C         2016-09-10      0

So, one of features I'm trying to create is the number of days since the last event, last event before that and so on. This would lead to something like this.

userid   eventid   eventdata    days_last_event   days_last2_event    label
1        A         2016-10-15   30                35                  1
2        A         2016-10-05   35                NA                  0   
5        B         2016-09-30   NA                NA                  1
1        B         2016-09-15   5                 NA                  0
2        C         2016-09-10   NA                NA                  0  
1        C         2016-09-10   NA                NA                  0

So, as you can see the feature columns created will have a lot of NA rows depending upon if the userid has occured before or not. I think having these replaced with 0 would imply something different than what NA would. And ignoring these rows isn't an option.

So, what is good practice to handle these kinds of cases? Should it work if I just replaced 0 for NA?


Should it work if I just replaced 0 for NA?

Probably not! Logistic regression works by finding a linear boundary between the classes. By just plugging in 0, there's a chance you're creating a dataset where such a linear boundary does not exist.

Asides from that, the question you're discussing is fairly well-researched. See, for examle this review lecture from Notre Dame.

There is an important question to consider, which is unclear from your question. Why is user B (for example) missing the days from the last event? Consider two different scenarios:

  1. User B always had events happening (or at least had events happening for a long time). At some random point, you decided to start recording the events of the user, so there are quite a few past events for user B, but you just don't have them.

  2. User B made some change, and now has events. This could happen, for example, if the user registered to some site, and now is making purchases (each purchase is an event). In this case, past events for user B simply don't exist.

As noted in the paper above, most schemes involve imputing missing values, even though this is known to be problematic. E.g., you can replace each missing value by the mean of the non-missing values for this column. More sophisticated schemes might build successive regression models to fill up missing values from existing values.

Going back to the above point, though - regardless of the imputation you choose - if the reason for the missing values is 2, then you can add indicator features describing whether the data has been imputed.

So, for example, if this is the case, your columns could include days_last_event_missing and days_last_two_event_missing. These would be 1 if the values were imputed, and 0 otherwise. You would feed these columns to the logistic regression algorithm along with the other ones.

If the reason for the missing values is not 2, then current wisdom is not to use these indicator variables.

  • $\begingroup$ Hi @Ami, thank you for the excellent answer. My use case would would be 1 because I have data from a certain time period but events happened well before that as well. I will impute likewise. $\endgroup$ – sfactor Oct 17 '16 at 5:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.